Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
having trouble working with pipeline

Hello,

I was following the pipeline tutorial and tried my hand at making my own algorithm using a pipeline. I tried making it similar to the tutorial but i also wanted to tweak a few things in order to learn. Anyways, I keep getting an error: "TimeoutException: Call to before_trading_start timed out
There was a runtime error on line 59." I have tried messing with it for a few hours but I can't figure out what i'm doing wrong. Can someone help point me in the right direction? The line that causes the error is in "before_trading_start."

from quantopian.pipeline import Pipeline  
from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline.factors import RollingLinearRegressionOfReturns

import pandas as pd  
import numpy as np  
from scipy import stats

def initialize(context):  
    schedule_function(balance, date_rules.week_start(), time_rules.market_open(hours=1))  
    schedule_function(my_record_vars, date_rules.every_day(), time_rules.market_close())

    my_pipe = make_pipeline()  
    attach_pipeline(my_pipe, name='my_pipeline')  
def make_pipeline():  
    rolling_correlations = RollingLinearRegressionOfReturns(  
        target=sid(8554),  
        returns_length=70,  
        regression_length = 35,  
    )  
    beta = rolling_correlations.beta  
    r = rolling_correlations.r_value  
    rsquared = r**2  
    shouldbuy_beta =  beta > 1.1  
    shouldbuy_r = rsquared > 0.9  
    securities_to_trade = (shouldbuy_beta & shouldbuy_r)  
    return Pipeline(  
        columns={  
            'beta':beta,  
            'rsquared':rsquared,  
            'should buy beta':shouldbuy_beta,  
            'shouldbuy r':shouldbuy_r,  
            'securities_to_trade':securities_to_trade  
        },  
        screen = (securities_to_trade)  
    )

def compute_weights(context):  
    if len(context.longs) != 0:  
        long_weight = 1/len(context.longs)  
    else:  
        long_weight = 0  
    print long_weight  
    return long_weight

def before_trading_start(context, data):  
    context.output = pipeline_output('my_pipeline') ## <<<<<------THE LINE THAT TIMES OUT  
    context.longs = context.output[context.output['securities_to_trade']].index.tolist()  
    context.long_weight = compute_weights(context)  
def balance(context, data):  
    for security in context.portfolio.positions:  
        if security not in context.longs and data.can_trade(security):  
            order_target_percent(security, 0)  
    for security in context.longs:  
        if data.can_trade(security):  
            order_target_percent(security, context.long_weight)

def my_record_vars(context, data):  
    """  
    Record variables at the end of each day.  
    """  
    longs = 0  
    for position in context.portfolio.positions.itervalues():  
        if position.amount > 0:  
            longs += 1  
    record(leverage=context.account.leverage, long_count=longs)





2 responses

The 'RollingLinearRegressionOfReturns factor is notoriously calculation intensive and takes a lot of time. An algorithm only gets so much time before it times out. One option is to limit your universe of stocks to a subset of the total and then use that filter as a mask to the factor. Using a mask will bypass the calculation for those securities which the mask does not return true. The value for those securities will be set to NaN but ostensively you don't care about those anyway. There are probably other ways to speed up your pipeline definition but this at least gets it working.

I've attached a backtest where I've added the built in Q500US universe (see https://www.quantopian.com/help#built-in-filters ) as a mask and (while still slow) it runs.

Dan

Thanks Dan!