Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
OHLC Resampling Dilemma

Lets say trading hours are from 9:00am to 3:30PM. If I resample OHLC every minute, what happens to the 3:30 data?

For 9:00 am the first minute ends at 9:00:59 and the next OHLC starts at 9:01:00-9:01:59?

1 response

@James Liu -Interesting question. Different exchanges and data aggregators do things differently.

First off, I assume this is regarding tick data or data which represents an order fill timestamped at the time it was executed? Pragmatically, there are no conventional trades which execute either at the open (in this case 9:00) or at the close (in this case 3:30). Not aware of rules for all exchanges but typically orders are only executed during market hours. This means after the open and before the close. However, there are some special executions which do happen at market open and close. Not all exchanges support these but one can often place an order before the open to execute 'at open'. These types of orders don't go through the typical fill rules and are sometimes referred to as an 'auction'. (see https://www.investopedia.com/articles/investing/091113/auction-method-how-nyse-stock-prices-are-set.asp ). These executions may be marked at the open or close time but will typically also have a separate flag indicating open or close pricing. These are special executions.

If one has tick data with a 3:30 timestamp it should probably be included in the final minute.

However, and this is a big however, this only applies to tick data which are streamed executions each with a timestamp. Most retail data (and all data on Quantopian) is bar data. The data provider has already hashed out any nuances of the tick data and aggregated it into bars. The Quantopian minute data is labeled as of the end of the minute. Therefore the first bar would be labeled 9:01 (actually for US markets it would be 9:31 ET). The last bar on a full trading day would be labeled 3:00 (actually for US markets it would be 4:00 PM ET). If there are 390 minutes in a trading day then there are 390 bars. The tick resampling ambiguity is resolved. There isn't a single 'correct' way to resample but here is my suggested method using pandas. Use the parameters closed='right', label='right'. This will return the last non-NaN price in each 5 minute bucket. The buckets start at times 01, 06, 11, 16 etc and end at times 05, 10, 15, 20 etc. They are labeled with the end time.

    close_1m = data.history(my_stocks, 'close', my_timeframe, '1m')  
    close_5m = close_1m.resample('5T', closed='right', label='right').last()

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.