Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
May/June leaderboard inconsistencies complicate stop/start decisions
Michael Van Kleeck
posted

Even with the latest June leaderboard update (Sunday evening -- about an hour ago), some paper trading statistics are still inconsistent with the May leaderboard. Since there's been no June trading yet, paper trading on the two boards should match, shouldn't it (except for the redefined beta)?

For example, this algo shows a max drawdown of about -0.57% on the May board:

https://www.quantopian.com/leaderboard/54cae33758432ab1b0000173?month=5&year=2015

But the same algo on the June board has a max drawdown of about -5.45%:

https://www.quantopian.com/leaderboard/54cae33758432ab1b0000173?month=6&year=2015

That's quite a big difference. The uncertainty about whether the June leaderboard is correct makes it harder to decide whether to stop an existing algorithm and start a new one.

7 responses
Lucas Silva

The back testing start and end date are not consistent, I think this was a suggestion I made a while back.
Regardless of the submission date the backtesting should be the same for all algorithm. Currently it is 2 years from submission date then paper trading. So the weight of back-testing and paper trading of all algorithms in the competition may vary a lot.

Deleted User

I submitted an algo on Friday last week and it has some positions and P/L. However I don't figure on the list at all? Why is that? Should I have to wait for another day before it gets a rank?

Brad Snider

Same here. Submitted algo Friday morning, it made some trades and seems to be running fine as contest entry, but thinking I should shut it down, gove up Friday's p&l and resubmit as it still has not shown up on list. Contacted [email protected] Friday night with no response yet.

Deleted User

On Friday we fixed a bug affecting the calculation of performance metrics for a small subset of contest entries. We've been in touch with Kevin about the miscalculation of his returns. Most entries should see little to no difference in the metrics between the two leaderboards. We vetted the returns of May's top performers to ensure this bug did not affect the contest outcome.

Leaderboard metrics should match what you see in the full backtest and live algo screens. When in doubt, trust the numbers that match what you see in those dashboards.

Beginner and Sniderweb,
Friday's development work on the June leaderboard prevented a few algorithms submitted Friday morning from appearing at the end of the day. You should see your entry on the leaderboard today.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Deleted User

@Andrew, I still don't see my entry on leadership board but it shows as active under my algorithms. Do you mean it will appear on board after today's close?

Best regards,
Beginner

Lucas Silva

I have 3 algos only one showing up in the rank, can you guys checked that you have not deleted some or moved to the wrong month?

Cheers

Deleted User

@beginner @lucas The June leaderboard will be update after trading ends today. If we don't see your algos pop up then, we'll take a closer look. Thanks for your patience!

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.