Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Adjusting a Builtin Pipeline Factor
Boone Baker
posted Jan 30, 2017

First, thanks for this awesome platform! I've gone through the tutorials and am very eager to get started. Unfortunately, I've hit a snag on what I think is a very basic issue.

In my pipeline, I'd like to run various moving averages on the Previous Day's data.

I've got the EMA 5-day as:
EMA5 = ExponentialWeightedMovingAverage(inputs=[USEquityPricing.close],window_length=5, decay_rate=.94, mask=base_universe)

Now I'd like to also have the Previous Day's EMA 5-day... I've tried a few different things, but can't seem to figure it out.
First I tried a few versions of the following, adjusting the "[-1]" around for different:
prevEMA5 = ExponentialWeightedMovingAverage(inputs=[USEquityPricing.close[-1],window_length=5, decay_rate=.94, mask=base_universe)

This was clearly wrong. So then I tried making a custom factor...
class PrevEMA(ExponentialWeightedMovingAverage):
def compute(self, today, asset_ids, out, data, window_length):
out[:] = ExponentialWeightedMovingAverage(data[:-1], window_length)

And calling it through:
pema5 = PrevEMA(inputs=[USEquityPricing.close], window_length=5)

I'm now getting an error saying that PrevEMA expected a 'decay_rate' parameter. If I add that to the call, then it says it's unexpected. If I add decay_rate into the compute definition, my error is: TypeError: compute() takes exactly 7 arguments (6 given)

I think it's possible to do what I'm attempting, I just don't know the way to go about it. If somebody could kindly help a rookie out, I think I'll be able to run with it and adapt to my needs down the road. Thanks!

9 responses
Jamie McCorriston
Jan 30, 2017

Hi Boone,

Welcome to Quantopian!

It looks like you're off to a good start with Pipeline. I understand what you are trying to do. To implement the 5-day moving average from yesterday, you'll need to define a CustomFactor, but you won't be able to use the builtin ExponentialWeightedMovingAverage. The ExponentialWeightedMovingAverage factor requires BoundColumns as inputs (doesn't actually have data yet), but in the compute function, data is an array of data. For a 1-day delayed EWMA, you'll need to define the entire computation in the compute function. I would recommend looking at the implementation of ExponentialWeightedMovingAverage in Zipline as a starting point.

Let me know if this helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Boone Baker
Jan 30, 2017

Thank you, Jamie!

I have written a custom factor for PrevEMA as follows (it's just the EWMA with a data[-1] tweak):
class PrevEMA(ExponentialWeightedFactor):
def compute(self, today, assets, out, data, decay_rate):
out[:] = average(
data[-1],
axis=0,
weights=exponentialweights(len(data[-1]), decay_rate),
)

but then I get an error that _ExponentialWeightedFactor is not defined. So upon search of Zipline - I found that definition and pasted that in...

then I got a SingleInputMixin is not defined error - searching led me to the "from zipline.pipeline.mixins import SingleInputMixin" line, but when I pasted that into my code, it raised an ImportError.

So what I'm getting at, is there an import for _ExponentialWeightedFactor or SingleInputMixin?

Boone Baker
Feb 1, 2017

I was able to Google a class definition for SingleInputMixin, but I'm not totally sure it's the same one, I got it from https://scrutinizer-ci.com/g/quantopian/zipline/code-structure/master/py-class/zipline.pipeline.SingleInputMixin. After including it in my notebook, I now get an "Insecure built-in function 'super'" error. No idea what that means, just that my SingleInputMixin class that I found returns a super...

Jamie McCorriston
Feb 1, 2017

Boone,

The part of the EWMA definition that you should be looking at is what's in compute. You will probably want to use the same definition of exponential_weights, but I don't think you need to inherit from the ExponentialWeightedFactor class. Instead, inherit from the CustomFactor class:

class PrevEMA(CustomFactor):

The reference to Zipline was meant to be so that you could see what the definition was for EWMA and use that as a starting point. I apologize if I was not clear earlier.

Boone Baker
Feb 1, 2017

Jamie,

Thanks for bearing with me. I'm still BEYOND confused. I've stripped out most of the excess stuff that I was trying to accommodate for, but if I'm understanding correctly (which is a big IF) - I need to inherit ExponentialWeightedFactor to pass the decay_rate.

I've stripped out pretty much everything except for the pieces directly tied to my calculation to illustrate.

Jamie McCorriston
Feb 2, 2017

Hi Boone,

No need to inherit ExponentialWeightedFactor. You can just add the decay_rate as a parameter to the CustomFactor. I got the notebook working. Let me know what you think!

Boone Baker
Feb 2, 2017

Thank you, Jamie! It was the params piece that I was missing. That clarification will help me SOOO much.

Now my concern is that all of the PrevEMA5 values are either NaN or 39.178706 - which makes me think it's not calculating correctly, or better worded- as expected.

For example: if I run EMA5 on '2015-05-04', AAPL has an EMA5 value of 129.061398 --- this is what I would expect to see from PrevEMA5 when run on '2015-05-05', but instead I'm getting 35.001484. (EDIT: I'm using a 0.94 decay_rate)

Is it possible that data[-1] isn't actually what I'm needing?

Jamie McCorriston
Feb 3, 2017

Nice distinction between 'expected' and 'correct'!

In this case, however, you're right to use 'correct'. You should actually be using data[:-1] (the : means everything up to the last (-1) index). You also want to extend the window_length to be 6. I made those changes in this version of the notebook.

I hope this helps.

Boone Baker
Feb 3, 2017

Thank you, Jamie! This helps immensely. It all makes sense to me know, so hopefully I should be able to run with this new understanding and start building some solid pipelines. I'm sure I'll be back with more questions soon, but this should get me on my feet and running.

I went ahead and expanded the date range a little, and screened it down to just 3 stocks to verify that the results were what I was expecting, and they absolutely are. Thank you again!

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.