Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Custom Industry Factor Issue

I am attempting to write a custom factor to compute the Gain Percent of each stock's industry. I need to be able to offset the window so that I can return (for example) the 1 week gain for the industry for the previous week. I am getting the following error when I try to execute it:

TypeError: zipline.pipeline.pipeline.add() expected a value of type zipline.pipeline.term.Term for argument 'term', but got abc.ABCMeta
instead. There was a runtime error on line 68.

Any help in figuring out how to do this would be appreciated.

FYI: Line 68 is:

 pipe.add(gainpctind_off0, name='gainpctind_off0')  

Thanks.


# Import the libraries we will use here  
import numpy as np  
from numpy import ma  
import pandas as pd  
import talib as ta  
from quantopian.pipeline.factors import SimpleMovingAverage  
from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline import Pipeline, CustomFactor  
from quantopian.pipeline.data.builtin import USEquityPricing  
import quantopian.pipeline.data.morningstar as ms

def GainPct(offset=0, nbars=2):  
    class GainPctFact(CustomFactor):  
        window_length = nbars + offset  
        inputs = [USEquityPricing.close]  
        def compute(self, today, assets, out, close):  
            out[:] = (close[-1] - close[offset]) / close[offset] * 100  
    return GainPctFact

def GainPctInd(offset=0, nbars=2):  
    class GainPctIndFact(CustomFactor):  
        window_length = nbars + offset  
        inputs = [USEquityPricing.close, ms.asset_classification.morningstar_industry_code]  
        def compute(self, today, assets, out, close, industries):  
            # For each industry, build a list of the per-stock gains over the given window  
            gains_by_industry = {}  
            for i in range(0, len(industries)):  
                industry = industries[i]  
                asset_gainpct = (close[-1] - close[offset]) / close[offset] * 100  
                if industry in assets_by_industry:  
                    gains_by_industry[industry].append(asset_gainpct)  
                else:  
                    gains_by_industry[industry] = [asset_gainpct]

            # Loop through each stock's industry and compute a mean value for that  
            # industry (caching it for reuse) and return that industry mean for  
            # that stock  
            mean_cache = {}  
            for i in range(0, len(industries)):  
                industry = industries[i]  
                if not industry in mean_cache:  
                    mean_cache[industry] = np.mean(gains_by_industry[industry])  
                out[i] = mean_cache[industry]  
    return GainPctIndFact    

# The initialize function is the place to set your tradable universe and define any parameters.  
def initialize(context):  
    pipe = Pipeline()  
    attach_pipeline(pipe, name='my_pipeline')

    gainpctind_off0 = GainPctInd()  
    gainpctind_off1 = GainPctInd(1)  
    gainpctind1wk = GainPctInd(0, 5)  
    gainpctindprevwk = GainPctInd(5, 5)  
    pipe.add(gainpctind_off0, name='gainpctind_off0')  
    pipe.add(gainpctind_off1, name='gainpctind_off1')  
    pipe.add(gainpctind1wk, name='gainpctind1wk')  
    pipe.add(gainpctindprevwk, name='gainpctindprevwk')  

def before_trading_start(context, data):  
    results = pipeline_output('my_pipeline')  
    print results.head(5)  
    update_universe(results.sort('gainpctind1wk').index[:10])  

# The handle_data function is run every bar.  
def handle_data(context,data):  
    # Record and plot the leverage of our portfolio over time.  
    record(leverage = context.account.leverage)

    # We also want to monitor the number of long and short positions  
    # in our portfolio over time. This loop will check our positition sizes  
    # and add the count of longs and shorts to our plot.  
    longs = shorts = 0  
    for position in context.portfolio.positions.itervalues():  
        if position.amount > 0:  
            longs += 1  
        if position.amount < 0:  
            shorts += 1  
    record(long_count=longs, short_count=shorts)  
5 responses

As a follow-up, I also tried this (and every variation of it I could think up):


def GainPctInd(offset=0, nbars=2):  
    def GainPctIndCompute(self, today, assets, out, close, industries):  
        # For each industry, build a list of the per-stock gains over the given window  
        gains_by_industry = {}  
        for i in range(0, len(industries)):  
            industry = industries[i]  
            asset_gainpct = (close[-1] - close[offset]) / close[offset] * 100  
            if industry in assets_by_industry:  
                gains_by_industry[industry].append(asset_gainpct)  
            else:  
                gains_by_industry[industry] = [asset_gainpct]

        # Loop through each stock's industry and compute a mean value for that  
        # industry (caching it for reuse) and return that industry mean for  
        # that stock  
        mean_cache = {}  
        for i in range(0, len(industries)):  
            industry = industries[i]  
            if not industry in mean_cache:  
                mean_cache[industry] = np.mean(gains_by_industry[industry])  
            out[i] = mean_cache[industry]  
    return type('GainPctIndFact', (CustomFactor,), {  
        'window_length': nbars + offset,  
        'inputs': [USEquityPricing.close, ms.asset_classification.morningstar_industry_code],  
        'compute': GainPctIndCompute  
    })  

But, I get this when I try it:

ValueError: Cannot construct types dynamically There was a runtime
error on line 66.

Line 66 when doing this is:

gainpctind_off0 = GainPctInd()  

To answer my own question, it seems all I needed was to put () at the end of the GainPctIndFact that I was returning:

return GainPctIndFact()  

My current incarnation of this factor now looks like this (after fixing a few other issues):

def GainPctInd(offset=0, nbars=2):  
    class GainPctIndFact(CustomFactor):  
        window_length = nbars + offset  
        inputs = [USEquityPricing.close, ms.asset_classification.morningstar_industry_code]  
        def compute(self, today, assets, out, close, industries):  
            # Compute the gain percents for all stocks  
            asset_gainpct = (close[-1] - close[offset]) / close[offset] * 100  

            # For each industry, build a list of the per-stock gains over the given window  
            gains_by_industry = {}  
            for i in range(0, len(industries)):  
                industry = industries[0,i]  
                if industry in gains_by_industry:  
                    gains_by_industry[industry].append(asset_gainpct[i])  
                else:  
                    gains_by_industry[industry] = [asset_gainpct[i]]

            # Loop through each stock's industry and compute a mean value for that  
            # industry (caching it for reuse) and return that industry mean for  
            # that stock  
            mean_cache = {}  
            for i in range(0, len(industries)):  
                industry = industries[0,i]  
                if not industry in mean_cache:  
                    mean_cache[industry] = np.mean(gains_by_industry[industry])  
                out[i] = mean_cache[industry]  
    return GainPctIndFact()  

If anyone has a clue on how to use NumPY to eliminate the two FOR loops while generating the correct results, I'm all ears.

Hi Eliot,

You beat me to it! I was typing out an answer and it seems you also caught the reference to assets_by_industry. I'm assuming you saw the discussion on creating parametrized custom factors here, but I linked it for your convenience in case you didn't see it.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

I had not seen that, but thank you for the link. I'm familiar enough with closures and while I'm rather green at Python still, I did know it could do closures, so I just Googled to make sure I could put a closure around a class definition and return it. The examples I saw, however, did not have the () on the return. I still would like to know how to remove two FOR loops and use numpy to perform the computations and copies more efficiently.

For anyone following this thread, the discussion was continued here: https://www.quantopian.com/posts/identifying-industry-average-values-for-fundamentals?utm_campaign=identifying-industry-average-values-for-fundamentals&utm_medium=email&utm_source=forums.