I do a two year backtest of my algorithm, the same algorithm as the contest. The results are radically different. Why? For example, algorithm backtest return is 124.8% over two years, but the contest says: ANNUAL RETURNS 40.10%. Max drawndown -12.7% backtest, -17.45% contest backtest.
My backtest dates are From 2013-06-15 to 2015-05-15. What dates should I use to replicate the contest backtest? Or what else is different.
Early last week we noticed a bug in our calculations that was affecting some algorithms in the contest, and it sounds like yours is one of them. We're working to fix the calculation and get it right. To note, before we formally award each monthly winner, we vet the algo to confirm the results.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Alisa,
That's a little disturbing to hear. If you vet the winner to confirm the results you're ensuring that the winning algorithm is not showing higher results than it actually should have received. You are not checking to ensure it actually had the best results for the contest period, unless you check all the other algorithms to ensure they weren't unfairly subject to a bug giving them lower returns than they deserved. Do you check that as well? Note to everyone in the top 20 or so of previous contests, you might want to run a backtest to ensure your results match the results you were awarded.
Took my own advice and surprising result, unfortunately the opposite of what I had hoped! I had an algorithm that placed 3rd in the April contest. It was submitted on Jan 29th after market close, so I ran a backtest using an exact copy of that algorithm from Jan 30th to April 30th, with $100,000 and minute data. This showed substantially worse results than my contest paper trading results, i.e. a negative return versus the paper trading contest positive return. Why would this happen, shouldn't a backtest on a given period yield the same result as the paper trading results over the same period?
and it happens the other way around. my algo did 20% + in 1 month and the contest says I did -200%.... I wouldn't rely on the current ranking. Alisa told me last week they are fixing it but I just hope its fast as we are nearing the end of the month
Crickets on this which is surprising since it calls into question the integrity of the entire contest. Not saying that I'm not completely missing something crucial here, and note that I'm advocating for something that would appear to lower my rankings if I'm correct, but it would be nice to hear some official word one way or the other.
We have a couple engineers working on this (in fact they're pointing at their screens next to me right now :) ). We know the symptoms but are still working on a reliable, reproducible case. In the contest, we test the top contenders to confirm the results and check the validity. Once we have a more concrete update, I'll post back here.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
