Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Basic Q - combining custom factors with existing factors

Hi,

I am new to the Quantopian community! Trying to get comfortable with the layout and am having trouble properly compiling a basic algorithm which combines a basic custom volatility factor with some fundamental factors. Am I adding the custom factor incorrectly into the Pipe? Any help would be appreciated!

from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline import Pipeline  
from quantopian.pipeline.filters.morningstar import Q1500US  
from quantopian.pipeline.data.morningstar import operation_ratios  
from quantopian.pipeline.data.morningstar import valuation_ratios  
from quantopian.pipeline.data.builtin import USEquityPricing  
from quantopian.pipeline.factors import AverageDollarVolume, CustomFactor  
import pandas as pd  
import numpy as np

class Volatility(CustomFactor):  
    inputs = [USEquityPricing.close]  
    window_length = 63  
    def compute(self, today, assets, out, close):  
        close = pd.DataFrame(data=close, columns=assets)  
        out[:] = 1 / np.log(close).diff().std() ##ranking lowest as best  
def initialize(context):  
    my_pipe = make_pipeline()  
    attach_pipeline(my_pipe, 'my_pipeline')  
    pipe.add(Volatility(), "volatility")  
    schedule_function(my_rebalance, date_rules.month_start(days_offset=0),time_rules.market_open(minutes=1))  
    schedule_function(my_record_vars, date_rules.every_day(), time_rules.market_close())

def make_pipeline():  
    test_factor1=operation_ratios.roic.latest  
    test_factor2=valuation_ratios.dividend_yield.latest  
    test_factor3=valuation_ratios.cash_return.latest  
    test_factor4=volatility  
    universe=(Q1500US() &  
              test_factor1.notnull() &  
              test_factor2.notnull() &  
              test_factor3.notnull() &  
              test_factor4.notnull())  
    testing_factor1=test_factor1.rank(mask=universe, method='average')  
    testing_factor2=test_factor2.rank(mask=universe, method='average')  
    testing_factor3=test_factor3.rank(mask=universe, method='average')  
    testing_factor4=test_factor3.rank(mask=universe, method='average')  
    factor_test=testing_factor1 + testing_factor2 + testing_factor3 + testing_factor4  
    quantile_test=factor_test.quantiles(10)  

    pipe=Pipeline(columns={  
            'factor_test':factor_test,  
        'shorts' :quantile_test.eq(0),  
        'longs' :quantile_test.eq(9)},  
                  screen=universe)  
    return pipe  
def before_trading_start(context, data):  
    try:  
        """  
        Called every day before market open.  
        """  
        context.output = pipeline_output('my_pipeline')

        context.security_list = context.output.index.tolist()  
    except Exception as e:  
        print(str(e))  

def my_rebalance(context,data):  
    """  
    Place orders according to our schedule_function() timing.  
    """  
    # Compute our portfolio weights.  
    long_secs = context.output[context.output['longs']].index  
    long_weight = 0.75 / len(long_secs)  
    short_secs = context.output[context.output['shorts']].index  
    short_weight = -0.25 / len(short_secs)

    for security in long_secs:  
        if data.can_trade(security):  
            order_target_percent(security, long_weight)  
    for security in short_secs:  
        if data.can_trade(security):  
            order_target_percent(security, short_weight)

    for security in context.portfolio.positions:  
        if data.can_trade(security) and security not in long_secs and security not in short_secs:  
            order_target_percent(security, 0)  

def my_record_vars(context, data):  
    """  
    Plot variables at the end of each day.  
    """  
    long_count = 0  
    short_count = 0

    for position in context.portfolio.positions.itervalues():  
        if position.amount > 0:  
            long_count += 1  
        if position.amount < 0:  
            short_count += 1  
    # Plot the counts  
    record(num_long=long_count, num_short=short_count, leverage=context.account.leverage)  
3 responses

Really appreciate you getting back to me so quickly.

Follow-up basic Q: Am I approaching the Long/Short setup correctly?
The quantile format is decile
I want to be long the top decile and short the bottom decile

Is this code correct?

def make_pipeline():  
    test_factor1=operation_ratios.roic.latest  
    test_factor2=valuation_ratios.dividend_yield.latest  
    test_factor3=valuation_ratios.cash_return.latest  
    test_factor4=volatility  
    universe=(Q1500US() &  
              test_factor1.notnull() &  
              test_factor2.notnull() &  
              test_factor3.notnull() &  
              test_factor4.notnull())  
    testing_factor1=test_factor1.rank(mask=universe, method='average')  
    testing_factor2=test_factor2.rank(mask=universe, method='average')  
    testing_factor3=test_factor3.rank(mask=universe, method='average')  
    testing_factor4=test_factor3.rank(mask=universe, method='average')  
    factor_test=testing_factor1 + testing_factor2 + testing_factor3 + testing_factor4  
    quantile_test=factor_test.quantiles(10)  

    pipe=Pipeline(columns={  
            'factor_test':factor_test,  
        'shorts' :quantile_test.eq(0),  
        'longs' :quantile_test.eq(9)},  
                  screen=universe)  
    return pipe  
def before_trading_start(context, data):  
    try:  
        """  
        Called every day before market open.  
        """  
        context.output = pipeline_output('my_pipeline')

        context.security_list = context.output.index.tolist()  
    except Exception as e:  
        print(str(e))  

def my_rebalance(context,data):  
    """  
    Place orders according to our schedule_function() timing.  
    """  
    # Compute our portfolio weights.  
    long_secs = context.output[context.output['longs']].index  
    long_weight = 0.5 / len(long_secs)  
    short_secs = context.output[context.output['shorts']].index  
    short_weight = -0.5 / len(short_secs)

    for security in long_secs:  
        if data.can_trade(security):  
            order_target_percent(security, long_weight)  
    for security in short_secs:  
        if data.can_trade(security):  
            order_target_percent(security, short_weight)

    for security in context.portfolio.positions:  
        if data.can_trade(security) and security not in long_secs and security not in short_secs:  
            order_target_percent(security, 0)  

A couple troubleshooting tips -

  1. It's easier to help when we're working with "cloneable" backtests and notebooks. The cut-and-paste leads to formatting problems and missing important parameters like date ranges.
  2. Pipelines are easiest to get going in the research environment. Much faster debug cycle and visualization. Plus you can then run Alphalens on the result.

That said, I think you weren't quite calling your CustomFactor correctly. Try this version, in the attached notebook. You can also analyze your results to see if you're generating the long/short list that you meant to.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Dan....my question about the decile is more along the lines of whether you specify bottom decile with 0 and top decile with 9? I am used to saying 1 vs 10 but saw an example in a training video where it was done the way I coded in the algorithm. Does that look right or am I missing something?

Thanks!