@Satya,

I've learned that the pandas dataframe has what is called a resample method. It takes a dataframe (what comes out of the history method) and works its magic.

This converts days to weeks:

closes   = history(126, "1d", "close_price").resample("1w")  

And this would convert single minutes to 15 minute intervals

closes   = history(390 * 30, "1m", "close_price").resample("15min")  

Note that these would be a rolling 15 minute windows (a recalc on minute 1, 2, 3, etc. going back 15 minutes at each recalculation.)

If you want consistent on-the-hour 15 minute bars then you'll have to either monitor the time as it is presented on each minute bar, and trigger accordingly. Or you could set a few schedule_function methods to trigger on your 15 minute thresholds.

Thanks Market Tech. I will give it a try.

Hello Satyapravin,

Note that Pandas also supports a variety of rolling statstics (see http://pandas.pydata.org/pandas-docs/version/0.15.0/computation.html). If you re-sample prices, you'll just be reducing the sampling rate of the minutely data stream, ignoring most of the data. Instead, you could try:

import pandas as pd

prices = history(10800, '1m', 'price')  
prices_smoothed = pd.ewma(prices, span = 30)  

You can treat the span of the ewma as a free parameter to be optimized (e.g. try span = 15, 30, 45, 60, ...). And you could still re-sample after the smoothing.

One problem with this approach is that at market open, you'll be averaging data from the prior day. So, you might want to wait until the market has been open for awhile before making trading decisions.

Grant

Hello Grant,
Why are you using exponential weight to smooth the data, and how does this differ from the resample method? Also, great job in the contest!

Regards,
Calvin

Hi Calvin,

The idea is to use all of the data, averaging it on a rolling basis, so that the noise is down to an appropriate level for the frequency of trading. Keep in mind that the minutely bars are constructed by sampling individual recorded trades; my understanding is that there is no averaging. So, using Pandas ewma will beat down the noise, weighting more recent data the most.

I'm not so familiar with resample. It appears to default to take the mean for downsampling (see http://pandas.pydata.org/pandas-docs/dev/timeseries.html#up-and-downsampling and http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.resample.html). So, there would be a smoothing effect, as well.

Personally, I don't see any reason to fiddle around with the timestamps. Pandas ewma (or rolling_mean) performs the averaging on the trailing window, and applies the minutely timestamp when the calculation was performed (which corresponds to the timestamp of the current OHLCV minutely bar). Perhaps the interest in generating bars at other frequencies is to port algos written for other data feeds to Quantopian?

Regarding the contest, there is a lot of luck involved. I am one of many monkeys on typewriters.

Grant

Grant,

Does that info go into the initialize section? I am confused as to where ti goes.

Thank you,
Dante Migale

Dante,

I think you are only allowed to call history() from handle_data() and any scheduled function. If you call it from initialize() you'll get:

HistoryInInitialize: history() should only be called in handle_data()  
There was a runtime error on line 5.  

And if you put it in before_trading_start() you'll get:

IndexError: index 0 is out of bounds for axis 0 with size 0  
There was a runtime error on line 8.  

Which is weird...Q support, is this a bug? Or maybe the error message just needs to be updated?

See attached (you can clone it to add a scheduled function, but I know history() works in this case).

Grant