How can I use the output of a custom factor as a mask for other custom factors?

Back to Community

edited Jul 21, 2020

In my pipeline, I have a custom factor that assesses whether a particular long term trend is detected.

It returns nan if the trend is not detected and a float if detected. My customfactors take quite a bit of computation time. However, I only need to run the next custom factors for those assets for which a value was returned by the first custom factor. In fact, the other factors take the output of the first factor as input, so running it for assets for which np.nan was returned does not make any sense....

So I was hoping that I could simply create a list of assets for which the first custom factor returned a value that was not null and use that list as a mask going forward for subsequent computations in my pipeline. However, this does not work.... The code below produces an error

So my question is... is there a way to create a mask and limit the assets on which computations are run, by the output of another factor? And if so, how should this be done?

def make_pipeline():  
#    test_securities = symbols(['AAPL', 'TSLA', 'AMZN'])  
#    base_universe = StaticAssets(test_securities)  
    base_universe = Q500US()

    universe = base_universe  
# Here we limit out universe to four assets  
    weeklySignal = weeklySignals(window_length = 400, mask = base_universe)  
    df1 = weeklySignal  
    df2=[pd.notnull(df1['Weekly signal'])]  
    maskResults = (pd.notnull(df2['Weekly signal']) == True)  
    maskInput = list(maskResults.index.get_level_values(1))  
    # maskInput is a list of assets for which subsequent computations should take place

    weeklyConfirmations = weeklyConfirmations(inputs = [weeklySignal], window_length = 400, mask = maskInput)  
    weeklyOBV = weeklyOBV(inputs = [weeklySignal], window_length = 400, mask = maskInput)

#    return Pipeline(columns={'Weekly signal': weeklySignal,  
#                            'Weekly confirmations': weeklyConfirmations,  
#                            'weeklyOBV': weeklyOBV  
#                            }, screen=base_universe)  
    return Pipeline(columns={'Weekly signal': weeklySignal,  
                            'Weekly confirmations': weeklyConfirmations,  
                            'weekly OBV': weeklyOBV}, screen = base_universe)

result = run_pipeline(make_pipeline(), '2015-01-05', '2015-01-08')  
result

2 responses

Dan Whitnable

Jul 22, 2020

Factors and custom factors can easily be made into filters which can then be used as a mask for other subsequent factors. In this way, the results of one factor can be used to limit the assets in another factor.

The first issue with the above code is that weeklySignal is not a dataframe. It's not the actual data and cannot be manipulated as such. It's a factor and only factor operations and methods can be applied to it. Therefore the code below won't work

    weeklySignal = weeklySignals(window_length = 400, mask = base_universe)  
    df1 = weeklySignal  
    df2=[pd.notnull(df1['Weekly signal'])]  
    maskResults = (pd.notnull(df2['Weekly signal']) == True)  
    maskInput = list(maskResults.index.get_level_values(1))

The pandas methods don't apply to factors. However, factors have their own methods to accomplish this.

   weekly_signal_not_null = weeklySignal.notnull()

See the documentation for other factor methods which create filters (https://www.quantopian.com/docs/api-reference/pipeline-api-reference#methods-that-create-filters). The above filter can then be used as a mask for subsequent factors, or the factor with the notnull method can be used directly

weeklyConfirmations = weeklyConfirmations(inputs=[weeklySignal], window_length=400, mask=weeklySignal.notnull())  
weeklyOBV = weeklyOBV(inputs = [weeklySignal], window_length=400, mask=weeklySignal.notnull())

That should work as you intended. Good luck.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Tristan Arkesteijn

Jul 23, 2020

Thank you Dan! This is exactly the help I needed.

You've successfully submitted a support ticket.

Our support team will be in touch soon.