Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Curious to know the formula Q uses to evaluate contest algorithms and the weight given to each metric???

Hey, I'm curious to know the formula Q uses to evaluate contest algorithms. I just took a look and the original I entered is still running and has made over 10 million on 10 million in less than a year. All the contest winners post less than 6%, but the only thing lower I see are draw down and beta in most cases. Yet, I'm sort of scratching my head simply because I can't reason as to why someone would take 500k as opposed to 10 million. Can someone help me understand how this is possible?

My algorithm:

113 PAPER TRADING SCORE
64.95
9 ANNUAL RETURNS
118.9%
532 ANNUAL VOLATILITY
19.72%
9 SHARPE
4.099
524 MAX DRAWDOWN
-10.60%
36 STABILITY
0.8829
9.0 SORTINO RATIO
7.663
419 BETA
0.2510
CORRELATION
-6.019%

Contest 36 Winner:

1 PAPER TRADING SCORE
PAPER TRADING SCORE
Score of algorithm based solely on paper trading track record.
90.80
233 ANNUAL RETURNS
3.911%
31 ANNUAL VOLATILITY
0.4173%
1 SHARPE
9.271
21 MAX DRAWDOWN
-0.09234%
1 STABILITY
0.9839
1.0 SORTINO RATIO
17.61
59 BETA
0.006711
CORRELATION
8.889%

Contest 35 Winner:

2 PAPER TRADING SCORE
90.43
144 ANNUAL RETURNS
5.077%
48 ANNUAL VOLATILITY
1.850%
10 SHARPE
2.709
22 MAX DRAWDOWN
-0.3054%
29 STABILITY
0.8596
4.0 SORTINO RATIO
9.377
30 BETA
0.002493
CORRELATION
0.07669%

7 responses

Very impressive!!

Does it stay within the bounds of the new contest criteria? If not, you may find your answer there?

From everything I read originally, I believe so. If it didn't, I assumed it would have been ejected. It's currently being ranked consistently on a daily basis. I'm interested in the formula though. How are the weights calculated and used to score the contest? Does anyone have any idea? I'm trying to understand so I can refactor if needed.

You can find the formula how the score is calculated in the detailed rules (I’m on a phone now so difficult to link to it). There are no other weights for the criteria, they are just minimum requirements that the algo needs to meet.

Don't want to hijack the thread, just out of the curiosity - having such results, why do you care about Quantopian contest after all? If this algorithm really works, you can be financially independent in just a few years with just a modest initial capital. And if you are heading higher and the algo can scale, there are many other hedge funds with different criteria for which this would have been no-brainer. Serious question.

I'm currently utilizing a similar algorithm in my trading geared towards small funds. The yield is 15 times the original investment in a given year. So, no I don't need a hedge fund. I'll do quite well on my own. However, I don't have ten million dollars. That was the point of entering the contest, to essentially prove that over the course of time that I could do this. After running this for a year, I don't think there is a single algorithm on this platform that comes close. So, I'm asking the question because I'm shocked at this point, given these results that I have not heard anything from Q regarding the strategy I crafted. Nevertheless, I've copywritten my algorithms. However, my curiosity is at an all time high given what I've observed over the last two contests, and it seems almost comical to me at this point, but there could be valid reasons as to why Q fails to embrace this reality, as Joakim alluded to earlier.

https://www.quantopian.com/leaderboard/35

Beta weighs heavily in the score.

@Aniken, What you're looking at is the old contest, which was discontinued a while ago. The new contest has new rules and new ranking criteria. It doesn't really make sense to complain about the old contest rules anymore, since they've already changed and improved it.

If you want to participate in the current contest, you need to re-enroll your algorithm in the new contest. I'm pretty sure the scoring for the new contest is as simple as returns / volatility.

The reason you didn't rank higher in the old contest is because your beta and volatility metrics are terrible. Your beta is so high it's not likely to fall within +-0.3 on a rolling basis, which is a requirement for the new contest. Your volatility is also so high, that it will severely affect your score.

As far as allocations go, they are not giving allocations to algorithms with 20% annualized volatility.

Why don't you just modify your algorithm so that the beta and volatility metrics are improved? These things are actually very important when dealing with large amounts of leveraged capital.

If you post a tearsheet we can maybe help point out things you can improve.