Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Are there bugs in calculation beta_spy_pt and passedBetaFilter for contest 38?
Vladimir
posted Jan 10, 2018
Contest Show Q Q Updates

Are there bugs in calculation beta_spy_pt and passedBetaFilter for contest 38?

beta_spy_pt     rankBetaSPY_pt      beta_spy_bt     rankBetaSPY_bt      passedBetaFilter  
0.198744854     1       0.003635506     1       TRUE  
4.543898426     4       0.004586902     2       TRUE  
8.515017849     6       0.007775122     3       TRUE  
8.98557035      7       0.016567874     4       TRUE  
2.62052721      3       0.025327434     5       TRUE  
7.75292586      5       0.035917536     6       TRUE  
27.93513798     14      0.037978297     7       TRUE  
10.61741498     8       0.229284324     8       TRUE  
11.90386069     9       0.253551848     9       TRUE  
17.60380623     12      0.270274533     10      FALSE  
0.699672182     2       0.287531362     11      FALSE  
15.13979993     10      0.327356814     12      FALSE  
15.64570006     11      0.333594343     13      FALSE  
21.82388019     13      0.992334777     14      FALSE
6 responses
Robert Petteruti
Jan 10, 2018

Hi Vladimir,

Many of these contest entries have only been live trading for 2 days, so the beta-to-SPY metric is essentially meaningless with such little data (to understand how we calculate this and other criteria, check out the contest Rules page). I am not surprised that these numbers are therefore high. They will remain volatile until there are more days in the sample to evaluate strategy performance.

I hope this helps and provides clarity.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Vladimir
Jan 11, 2018

Hi Robert,

Many of these contest entries have only been live trading for 2 days, so the beta-to-SPY metric is essentially meaningless with such little data.

I agree with that.

But betas 27.93513798 (#14), 21.82388019 (#13) and so on may be generated only if there is a bug.
You probably already fixed it because in today's leaderboard those numbers are more realistic.

Now about a bug in calculation passedBetaFilter.

According rules:

First, the Participant's algorithm will be filtered based on its correlation to the market. The correlation is measured as the beta-to-SPY ratio, and the algorithm must have a beta value between 0.3 and -0.3 to be in the top rank.

Why these two did not pass the filter?

beta_spy_bt     rankBetaSPY_bt      passedBetaFilter   

 0.270274533     10                  FALSE  
 0.287531362     11                  FALSE

One more question.

There is a rule:

Quantopian will run a two (2)-year backtest with Participant's algorithm and start paper trading Participant's algorithm using live data.

(2)-year backtest, as I understand, is 24 month.

Why Quantopian backtest for contest 38 starts 12/01/2017(25+ month) and not 01/01/2018(24+ month) as it should be?

Robert Petteruti
Jan 11, 2018

Hi Vladimir,

The beta-to-SPY values will become less volatile and more reasonable as the number of live trading days increases, which is why they look more realistic now.

To determine beta_spy_bt, we calculate the correlation between an algo’s return stream and the benchmark SPY for every 12-month rolling window within the backtest period. We then average these 12 values. A strategy passes passedBetaFilter if the absolute value of each of the 12 correlation values is within 0.3. Otherwise, it fails the filter.
This explains why a few of these strategies have failed the passedBetaFilter even though beta_spy_bt is within 0.3. We do not list these 12 correlation values for every strategy in the leaderboards, but you can view these values in the ’12 MONTH’ column in the Beta section of a full backtest.

We begin the 2-year backtest a month earlier to ensure we have 12 distinct rolling time windows with which to calculate 12 correlation values. Therefore, the length of the backtest is 25 months instead of 24.

I hope this helps.

Vladimir
Jan 12, 2018

Hi Robert,

If Quantopian need 25 months backtest, why not to put that number in the rules?

If Quantopian do not use beta_spy_bt, why it exist in the leaderboard?

Why not to replace beta_spy_bt by max_abs_roll_beta_spy_bt?

Leo M
Jan 12, 2018

Vladimir, are you using fixed basis points slippage in your backtests? Contest38 will have that as default. Fixed basis points slippage can cause that magnitude of difference.

Vladimir
Jan 12, 2018

Leo M,

Thank you so much for pointing the right direction.

I was using :

    set_slippage(slippage.FixedBasisPointsSlippage(0))

but should:

    set_slippage(slippage.FixedBasisPointsSlippage())
About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.