Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Combining filters in pipeline

Hello

I have three factors in a pipeline.

lower_band,
lower_band_tplus1,
lower_band_tminus2.

Note. The factor lower_band is calculated from past data that does not include t-1, so it is possible to plot it into the future t+1.

The 2nd and 3rd factors are just lagged copies of the 1st factor (lagged t+1 and t-1).

Use of factors in filters:
I use these factors in filters to determine if the value of today's lower_band is a maximum (greater than t+1 and t-1).

I have two questions please:

1) I currently achieve this by using three different factors , with three different "custom factors" which are identical apart from being lagged forward and backward 1 day. Is it possible to determine if today's value for lower_band is a maximum with just a single custom factor and the use of something like .shift()?

2) Later, in the logic of the pipeline, I'm trying to determine if is a max with the following statement:

Approach 1

arrows_up =  lower_band >= max(lower_band_minus1, lower_band_plus1) 

longs = arrows_up  

However, when i check the output this is not behaving as expected.

When I break the statement down into two steps, I get the correct output:

Approach 2

arrows_up_step1 =  lower_band >= lower_band_minus1  
arrows_up_step2 =  lower_band >= lower_band_plus1

longs = arrows_up_step1  & arrows_up_step2  

I can't understand why Approach 1 and Approach 2 get such different results? Approach 1 gives incorrect results (the lower_band value is not the max).

Thanks in advance

7 responses

Any hints tips appreciated! I've stalled! :-)

Just stumbled upon this - looks like can be used to solve problem 1.

https://www.quantopian.com/posts/simple-moving-average-n-days-ago

If anybody knows the solution to problem 2 - be great to hear!

Hmmm. OK, digging deeper into this custom factor:

def offset_sma_factor(days, **kwargs):  
    kwargs['window_length'] += days  
    class OffsetSMA(CustomFactor):  
        def compute(self, today, assets, out, values):  
            out[:] = np.nanmean(values[:-days])  
    return OffsetSMA(**kwargs)  

It was suggested by Q in a thread here:
https://www.quantopian.com/posts/simple-moving-average-n-days-ago

It looks like a great way to lag a custom factor. The only problem I've noticed is that when you pass to the parameter that controls the lag ("days") the value of zero, it returns NAN. This is a real pain! :-)

Could somebody who really knows there way around custom factors comment?
If this is the case then i'm going to need two versions of the custom factor, one with the ability to be lagged, and one simple version.

Tks

Maybe try this

def offset_sma_factor(days, **kwargs):  
    kwargs['window_length'] += days  


    class OffsetSMA(CustomFactor):  
        def compute(self, today, assets, out, values):  
            out[:] = np.nanmean(values[:self.window_length-days], axis=0)  
    return OffsetSMA(**kwargs)  

Note also the addition of 'axis=0'. Without that the function takes the mean of the flattened array. With it will take the mean of each column (ie each security)

See attached for it in action. Works with 0. Also note the sma200 values are offset correctly.

Good luck.

Thanks Dan - That code of yours works perfectly for me.

Could i pos also ask you about this issue?

In the logic of the pipeline, I'm trying to determine if is a max with the following statement:

Approach 1



arrows_up =  lower_band >= max(lower_band_minus1, lower_band_plus1) 

longs = arrows_up  

However, when i check the output this is not behaving as expected.

When I break the statement down into two steps, I get the correct output:

Approach 2

arrows_up_step1 =  lower_band >= lower_band_minus1  
arrows_up_step2 =  lower_band >= lower_band_plus1

longs = arrows_up_step1  & arrows_up_step2  

I can't understand why Approach 1 and Approach 2 get such different results? Approach 1 gives incorrect results (the lower_band value is not the max).

Thanks in advance

There are only a limited number of operators which can be used with factors and max isn't one of them. See the docs at https://www.quantopian.com/help#quantopian_pipeline_filters_Filter and look just before the 'filter' section. That is why version 1 doesn't work. Version 2 uses comparison operators which are implemented.

I have always found it odd there isn't an error when trying to use any unsupported operators. It appears everything worked, but as you noticed, the results are just wrong. A common mistake is to use the words or and rather than the symbols | &. This again isn't supported but there is no error. Just the wrong result.

Thank you