I have a few concerns with how the Quantopian Contest evaluates algorithms. I developed a VIX Daytrader algo which has produced excellent returns annually in the backtest since VXX/XIV was created, real life since I started running it in my IB account late September, and the Quantopian November Contest. Despite both versions of the algo I submitted being top 8 in terms of returns with 704% and 1299% annualized respectively as of 1/29/2017 (Screenshot), both algos are ranked very poorly. I also have an earnings algorithm that I just submitted for the Feb contest which has similar rates of return but similar risk profile. These are algorithms which have great potential for return, but high risk (i.e. 20-35% max drawdown). So this brings up a few factors with how the contest is ranked that I disagree with:
1. Not identifying other sources of long/short diversification. For example, VXX and XIV are ETFs which are inverse of one another. It makes no sense to short VXX when you can avoid short fees and lack of shares to borrow by longing XIV in place.
2. Low beta requirement. While low beta in general is good, there are exceptions. There are plenty of successful algorithms which scalp certain futures contracts or trade only one or a few types of assets. These algorithms, especially if they trade something correlated with the S&P will always have a high beta. In fact, many of the most successful production algos in sites such as Collective2 (e.g. ES ST IT) do exactly this. With futures coming to Quantopian, it may be worthwhile to penalize algos only if they rely on specific movements in the S&P rather than just trading assets correlated to the S&P.
3. Most Importantly Significant points appear to be dinged for having high volatility despite great returns. Currently, most of the top algorithms have returns of ~10% or less, but minimal drawdown. While this would be ideal if you could trade only one algorithm, a diversified set of volatile algorithms with potential high returns will produce higher returns overall. Most winning volatile algos would probably do better than this, but imagine an algorithm with 150% return, 40% drawdown, and 50% chance of hitting the drawdown then quitting. With hypothetical probability being reality, running 10 of those algos with 100K each would result in a portfolio value of 100000 * 2.5 * 5 + 100000 * .4 * 5 = 1450000. Meanwhile 10 "stable" algos of 10% annual return and 0% chance of drawing down would only result in 1100000. And this is once again with the fact that high quality, winning volatile algos will have even better rates of return and lower rates of failure.
Thoughts on changing the criteria or creating a second type of contest? Could Quantopian create contests that are perhaps more focused on return such as World Cup Trading Championship? This could be very worthwhile especially as Quantopian enters the futures arena and has more capital to allocate.
