Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
add new column after pipeline run - deciles accounting for industries
Peter Longauer
posted Dec 6, 2017
API Seeking Help Getting Started Pipeline

Hi, I am quite new to python and dont fully understand limits and capabilities of the language.
I am trying to run a backtest where I look at my fundamental factor which is a ratio computed across Q1500 stocks within their industries and want to go short 10th deciles and long 1st deciles stocks.
It wouldnt be that complicated if I looked at the fundamentals across whole Q1500, but i want to account for industries because top/bottom decile might be skewed towards some particular industry.
I came up with this code but obviously it doesnt work, its breaking after return pipe.
Any idea/solution would be much appreciated, thanks.

    pipe = Pipeline(  
        columns={  
            'n1': NI1,  
            'n2': NI2,  
            'ts': TD,  
            'recent_indebt' : recent_indebt,  
            'sectors' : sectors  
        },  
        screen = (sector)  
        #mask = universe  
    )  
    return pipe

    dec = pipe.groupby(['sectors'])['ts'].transform(  
                     lambda x: pd.qcut(x, 10, labels=range(1,11)))  
    pipe.add(dec, 'dec')  
    low_returns = pipe['dec'].eq('10')  
    pipe.add(low_returns, 'low_returns')  
    pipe['high_returns'] = pipe['dec'].eq('1')  
    pipe.add(high_returns, 'high_returns')
5 responses
Luca
Dec 6, 2017

You can easily compute the deciles from the factor itself (pipeline factors have some useful methods like .deciles) and then you can add the results to the pipeline itself:

from quantopian.pipeline.classifiers.morningstar import Sector

    pipe = Pipeline(  
        columns={  
            'n1': NI1,  
            'n2': NI2,  
            'ts': TD,  
            'ts_dec': TD.deciles(groupby=Sector()),  
            'recent_indebt' : recent_indebt,  
            'sectors' : sectors  
        },  
        screen = (sector)  
        #mask = universe  
    )  
    return pipe
Peter Longauer
Dec 6, 2017

thanks Luca, I get the below error. Am I missing something?

TypeError: deciles() got an unexpected keyword argument 'groupby'
...

Ernesto Perez
Dec 7, 2017

Hello Peter,

The .deciles() attribute method does not have a 'groupby' parameter. An alternative would be to first zscore securities grouping them by sector, and then apply the deciles method. Here is an example using Luca's code from above:

from quantopian.pipeline.classifiers.morningstar import Sector

    pipe = Pipeline(  
        columns={  
            'n1': NI1,  
            'n2': NI2,  
            'ts': TD,  
            'ts_dec': TD.zscore(groupby=Sector()).deciles(),  
            'recent_indebt' : recent_indebt,  
            'sectors' : sectors  
        },  
        screen = (sector)  
        #mask = universe  
    )  
    return pipe

This should give you a result close to what you are looking for. I hope it helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Luca
Dec 7, 2017

oh, I was too optimistic, I expected .deciles method to have the same arguments as .zscore and .rank. Actually, I am not sure why 'groupby' is not in .deciles.
Thanks Ernesto for the correction and workaround.

Peter Longauer
Dec 10, 2017

Thanks both for the advice

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.