Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
How to get "1 hour" frequency in history(bar_count, frequency, field)?

Hi, I'm new here, and I already love this place!

For my algorithm I'm interested to get hourly data. However if I understood correctly history(bar_count, frequency, field) can only give data per day or per minute.
Am I missing something? Because it seems like getting hourly bar is pretty basic...
Maybe there is a function to transform the "1 minute bars" into "1 hour bars"?

Thanks a lot!

5 responses

You are correct in that we use data at a frequency of 1 minute or 1 day. I actually just wrote a function for another user that chunks the data into specified intervals. I didn't spend too much time on it so it probably isn't super efficient. See if this code does what you want...

import numpy as np  
import pandas as pd  
def group_by_minutes(data, minutes=15):  
    grouped_sums = pd.DataFrame()  
    for security, ts in data.iteritems():  
        minute_n_sum = pd.Series()  
        for dates, group in zip(np.array_split(ts.index, len(ts.index) / minutes), np.array_split(ts.values, len(ts.values) / minutes)):  
            minute_n_sum["%s:%s-%s:%s" % (dates[0].hour, dates[0].minute, dates[-1].hour, dates[-1].minute)] = np.sum(group)  
        grouped_sums[security] = minute_n_sum  
    return grouped_sums  
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Can't one just use pandas resample?

Simon,

You're probably right that would probably be the way to do what this guy is asking. The above code snippet was adapted from a user who wanted to have custom index values.

Using resample...

vol_history = history(390, "1m", "volume")  
hourly_vol_history = vol_history.resample('60Min', how='sum')

Documentation on resample.

James,
Hi, I am new to python. I tried to use your custom function to convert the data of 1 minute in get_pricing into 15 minutes but I received the error of :

AttributeErrorTraceback (most recent call last)
in ()
----> 1 group_by_minutes(testData, 15)

in group_by_minutes(data, minutes)
3 for security, ts in data.iteritems():
4 minute_n_sum = pd.Series()
----> 5 for dates, group in zip(np.array_split(ts.index, len(ts.index) / minutes), np.array_split(ts.values, len(ts.values) / minutes)):
6 minute_n_sum["%s:%s-%s:%s" % (dates[0].hour, dates[0].minute, dates[-1].hour, dates[-1].minute)] = np.sum(group)
7 grouped_sums[security] = minute_n_sum

AttributeError: 'numpy.float64' object has no attribute 'index'

@James when we call history function at the early market open, it includes data from yesterday's data after market close or data from before market open. How to get history data just for today.(only data after market open is needed)