Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Resample and Indicators
Tim Trice
posted Jun 23, 2015

I'm struggling, hoping someone can help.

This is inside a scheduled function. The bar_count is high because frankly nothing lower is working. When I access ema9[s][-1] I get nan. In fact, most the ema9[s] is nan except for a few bars somewhere in the middle.

Can you not run indicators based off resampled history data?

def OnFiveMinutes(context, data):

    c = history(bar_count=5000,  
                frequency='1m',  
                field='close_price').resample('5min')

    for s in context.stocks:

        # Get EMA(9)  
        ema9 = c.apply(ExponentialMovingAverage,  
                       length=9)
5 responses
Jeremy Steele
Jun 24, 2015

I think it is either a problem in your ExpoentialMovingAverage function or the stocks in your universe don't have data.

The following works fine.

import pandas  
def initialize(context):  
    set_symbol_lookup_date('2015-11-01')  
    context.stocks = symbols('AAPL','GOOG','T')

# Will be called on every trade event for the securities you specify.  
def handle_data(context, data):  
    # Implement your algorithm logic here.

    # data[sid(X)] holds the trade event data for that security.  
    # context.portfolio holds the current portfolio state.  
    c = history(bar_count=5000,  
                frequency='1m',  
                field='close_price').resample('5min')  
    for stock in data:  
        ema9 = pandas.ewma(c[stock],com=4)[-1] #com is center of mass = (s-1)/2 or com =4 where periods = 9)  
        print ema9
Tim Trice
Jun 24, 2015

Jeremy, you may be correct but I'm not sure how. And as I'm trying to stick to TALIB I would like to figure out this issue before I abandon it.

The EMA works great if I don't resample the history data. Here is the 5-minute closing data:

2015-05-19 14:40:00+00:00             73.282333  
2015-05-19 14:45:00+00:00             73.302000  
2015-05-19 14:50:00+00:00             73.303200  
2015-05-19 14:55:00+00:00             73.370000  
2015-05-19 15:00:00+00:00             73.461000  
2015-05-19 15:05:00+00:00             73.467000

Nothing abnormal. Here is the EMA based on that 5-minute data:

 Timestamp('2015-05-19 15:10:00+0000', tz='UTC', offset='5T'): nan  
Timestamp('2015-05-19 15:15:00+0000', tz='UTC', offset='5T'): nan  
Timestamp('2015-05-19 15:20:00+0000', tz='UTC', offset='5T'): 73.4083925926  
[some data eradicated to save space]
Timestamp('2015-05-19 20:00:00+0000', tz='UTC', offset='5T'): 73.4060464978  
Timestamp('2015-05-19 20:05:00+0000', tz='UTC', offset='5T'): nan  
Timestamp('2015-05-19 20:10:00+0000', tz='UTC', offset='5T'): nan

All above and below are nan.

If I do not resample the data my closes are just fine only at one-minute; expected.

My EMA, however, works perfectly (no nan's).

Tim Trice
Jun 24, 2015

So, as I was looking for the resampled history data in my post above I noticed an issue that is generating the problem.

When I resample the history data, for whatever reason, it is filling in non-market hours, too. This is the data from c:

2015-06-05 19:10:00+00:00               78.1400  
2015-06-05 19:15:00+00:00               78.1610  
2015-06-05 19:20:00+00:00               78.1510  
2015-06-05 19:25:00+00:00               78.1140  
2015-06-05 19:30:00+00:00               78.1290  
2015-06-05 19:35:00+00:00               78.1086  
2015-06-05 19:40:00+00:00               78.0480  
2015-06-05 19:45:00+00:00               78.0430  
2015-06-05 19:50:00+00:00               78.1060  
2015-06-05 19:55:00+00:00               78.1900  
2015-06-05 20:00:00+00:00               78.1400  
2015-06-05 20:05:00+00:00                   NaN  
2015-06-05 20:10:00+00:00                   NaN  
2015-06-05 20:15:00+00:00                   NaN  
2015-06-05 20:20:00+00:00                   NaN  
2015-06-05 20:25:00+00:00                   NaN  
2015-06-05 20:30:00+00:00                   NaN  
2015-06-05 20:35:00+00:00                   NaN  
2015-06-05 20:40:00+00:00                   NaN  
2015-06-05 20:45:00+00:00                   NaN

The resample function is adding new data into the closing fields and this is causing my issue. So my next question is, why and how can I correct it?

Jeremy Steele
Jun 24, 2015

ema9.dropna(inplace=True)

Tim Trice
Jun 24, 2015

Thanks Jeremy. I'll have to save that for future reference.

I did find what I think is another solution:

    c = history(bar_count=50,  
                frequency='1m',  
                field='close_price').groupby(lambda d: d.date()).resample('5min')

And the data:

2015-06-05 2015-06-05 19:10:00+00:00                   NaN  
           2015-06-05 19:15:00+00:00                   NaN  
           2015-06-05 19:20:00+00:00                   NaN  
           2015-06-05 19:25:00+00:00                   NaN  
           2015-06-05 19:30:00+00:00                   NaN  
           2015-06-05 19:35:00+00:00                   NaN  
           2015-06-05 19:40:00+00:00                   NaN  
           2015-06-05 19:45:00+00:00                   NaN  
           2015-06-05 19:50:00+00:00             78.111178  
           2015-06-05 19:55:00+00:00             78.126942  
           2015-06-05 20:00:00+00:00             78.129554

I'm not sure what unexpected issues may arise from this but I'm going to give it a shot. I'm not a python guru; I've no idea what lambda is or what any of that code does; I'll have to figure it out later.

On the first minute of the trading day I do get nan for the latest value:

2015-06-05 2015-06-05 19:10:00+00:00                   NaN  
           2015-06-05 19:15:00+00:00                   NaN  
           2015-06-05 19:20:00+00:00                   NaN  
           2015-06-05 19:25:00+00:00                   NaN  
           2015-06-05 19:30:00+00:00                   NaN  
           2015-06-05 19:35:00+00:00                   NaN  
           2015-06-05 19:40:00+00:00                   NaN  
           2015-06-05 19:45:00+00:00                   NaN  
           2015-06-05 19:50:00+00:00             78.111178  
           2015-06-05 19:55:00+00:00             78.126942  
           2015-06-05 20:00:00+00:00             78.129554  
2015-06-08 2015-06-08 13:30:00+00:00                   NaN

However, when I execute the function again at 13:36 (9:35 EST) 13:30 and 13:35 fill with data:

2015-06-05 2015-06-05 19:15:00+00:00                   NaN  
           2015-06-05 19:20:00+00:00                   NaN  
           2015-06-05 19:25:00+00:00                   NaN  
           2015-06-05 19:30:00+00:00                   NaN  
           2015-06-05 19:35:00+00:00                   NaN  
           2015-06-05 19:40:00+00:00                   NaN  
           2015-06-05 19:45:00+00:00                   NaN  
           2015-06-05 19:50:00+00:00                   NaN  
           2015-06-05 19:55:00+00:00             78.116622  
           2015-06-05 20:00:00+00:00             78.121298  
2015-06-08 2015-06-08 13:30:00+00:00             78.170288  
           2015-06-08 13:35:00+00:00             78.229231

Obviously the 13:30 should not be in there so I'll have to play around with my scheduling and see if I can get out of that. I'll try and verify those numbers shortly, too.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.