Min Correlation

Back to Community

Min Correlation

posted

Hello Quantopians,

Could someone please help me finish this algo. This is an attempt to re-create the minimum correlation algo based on the post here. I have been working on fixing the error for 5 hours now but still no luck. It throws me an error with the list of all the stocks selected.

I am not sure why I am not able to see the code in the posted backtest. So I am pasting all the code here.

from quantopian.pipeline.filters import Q500US  
from quantopian.pipeline.factors import DailyReturns  
from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline import Pipeline, CustomFactor  
from quantopian.pipeline.data import Fundamentals, USEquityPricing  
import numpy as np  
import math  
import scipy


def initialize(context):  
    schedule_function(trade, date_rules.every_day(), time_rules.market_close(minutes = 30))  
    attach_pipeline(make_pipeline(), 'pipeline')  
def make_pipeline():  
    m = Q500US()  
    ret = DailyReturns()  
    quality = ret  
    good = quality.top(20, mask = m)  
    bad = quality.bottom(20, mask = m)  
    pipe = Pipeline(columns={'good':good, 'bad':bad}, screen = (good | bad))  
    return pipe

def trade(context, data):  
    output = pipeline_output('pipeline')  
    longlist = output[output.good].index  
    shortlist = output[output.bad].index  
    prices = np.log(history(30, '1d', 'price').dropna(axis=1))  
    daily_R = prices.pct_change().dropna()  
    longs = get_reduced_correlation_weights(daily_R[longlist])  
    shorts = get_reduced_correlation_weights(daily_R[shortlist])

    for stock in data:  
        if stock in shortlist:  
            order_target_percent(stock, -0.5 * shorts[stock])  
        elif stock in longlist:  
            order_target_percent(stock, 0.5 * longs[stock])  
        else:  
            order_target(stock, 0)  

def get_reduced_correlation_weights(returns, risk_adjusted=True):  
    correlations = returns.corr()  
    adj_correlations = get_adjusted_cor_matrix(correlations)  
    initial_weights = adj_correlations.T.mean()  
    ranks = initial_weights.rank()  
    ranks /= ranks.sum()  
    weights = adj_correlations.dot(ranks)  
    weights /= weights.sum()  
    if risk_adjusted:  
        weights = weights / returns.std()  
        weights /= weights.sum()  
    return weights  
def get_adjusted_cor_matrix(cor):  
    values = cor.values.flatten()  
    mu = np.mean(values)  
    sigma = np.std(values)  
    distribution = scipy.stats.norm(mu, sigma)  
    return 1 - cor.apply(lambda x: distribution.cdf(x))

2 responses

Dan Whitnable

Here are a few comments...

First, Everything looks fine with the imports, and the initialize and make_pipeline functions. From a code execution perspective, the first suspicious lines of code are

   longlist = output[output.good].index  
   shortlist = output[output.bad].index  
   prices = np.log(history(30, '1d', 'price').dropna(axis=1))

One should use the data.history method to fetch pricing data. Also, the assets must be specified (see the docs https://www.quantopian.com/docs/api-reference/algorithm-api-reference#quantopian.algorithm.interface.BarData.history). Is this what was intended?

    longlist = output[output.good].index.tolist()  
    shortlist = output[output.bad].index.tolist()  
    prices = np.log(data.history(assets=longlist+shortlist,  
                                 fields='price',  
                                 bar_count=126,  
                                 frequency='1d'  
                                ).  
                    dropna(axis=1)  
                   )

Notice the use of the tolist() method which will turn the index into a true list. Generally a good idea if one is calling something a list to ensure it's actually a list. This also makes concatenating the two easier (simply use the + operator).

The next issue is the following line

    daily_R = prices.pct_change().dropna()

Is the intention that 'daily_R ' are the daily log returns? If so, a better way to calculate this would be something like the following.

    longlist = output[output.good].index.tolist()  
    shortlist = output[output.bad].index.tolist()  
    prices = data.history(assets=longlist+shortlist,  
                                 fields='price',  
                                 bar_count=126,  
                                 frequency='1d'  
                                )  
    daily_R = np.log(1 + prices.pct_change()).dropna()

Finally, the following line iterates over all assets in data. This isn't really supported and, at the very least, is doing a lot of unnecessary work.

    for stock in data:

Consider iterating over just the long and short stocks something like this

    for stock in longlist+shortlist:  
        if stock in shortlist:  
            order_target_percent(stock, -0.5 * shorts[stock])  
        elif stock in longlist:  
            order_target_percent(stock, 0.5 * longs[stock])  
    for stock in context.portfolio.positions:  
        if stock not in longlist+shortlist:  
            order_target(stock, 0)

I didn't want to begin making a lot of assumptions, so stopped with these couple of issues. The algo runs but not sure if the logic is what was intended.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Nadeem Ahmed

Hi Dan - Thank You for your help as always. The intention was to simply use the weights from get_reduced_correlation_weights in each longlist and shortlist instead of equal weights. Do you think there is an issue with the logic here?

PS: sorry for double post. I am not sure how to delete it. Please delete the other post if it is possible.

You've successfully submitted a support ticket.

Our support team will be in touch soon.