Hi! Let's say, hypothetically, that someone wants to assure that he/she will be the one with the best backtest in the contest and makes an algo that does a lot of overfitting... How can the Quantopian team guarantee to other Q Open contestants that this algo will be disqualified?
If someone makes a good, not gamed algo, that really makes a beautiful profit, that has a low beta, low DD, but the annual reuturns, sharpe and calmar ratio are not good enought to assure the 1st place and decides to implement some cheating lines of code in the algo that will run only in the backtest. Is Q able to see that? How does Quantopian see that, without looking at the code?
Let's take my algo. I worked night and day and I came up with a good strategy. The result of the 2 years of backtest:
Annual Returns 22.28% // Sharpe 2.472 // Max Drawdown -3.652% // Calmar Ratio 6.101 // Beta 0.03256
I could do better, a lot better, but I liked the strategy and I think it has great potential in live trading. Whatever.
The point is that a CALMAR of 1700+ seems to be out of this world. And I work with statistics. When something looks highly improbable, almost impossible, you maybe want to check it twice. It's medium to low frequency trading, not HFT. I don't think Renaissance Technologies has algos with that calmar for MFT.
320% 2-year return with -0.095% max DD?! Come on!*
* I don't accuse anyone yet, but my rational part of the brain is very suspicious.
So, if I would want to game the backtest of an algo, it would be extremely easy**:
1. Search for securities or ETFs with very high return in the past 2 years. Ex: ADXS, BLUE, CBMG, ESPR. Or even NASDAQ-100 and DJIA components with small periods of very high positive percentual change.
2. Cut out the periods of DD by buying and selling at exactly the perfect moment by using lines of code like:
def handle_data(context, data):
pos = context.portfolio.positions
if not pos:
if get_datetime().year == 2015:
if get_datetime().month == 3:
if get_datetime().day == 5:
order_target_percent(sid(44989),0.2) # ESPR
if pos:
if get_datetime().year == 2015:
if get_datetime().month == 3:
if get_datetime().day == 23:
order_target_percent(sid(44989),0) # ESPR - Lower the beta by finding more opportunities for short selling some stocks.
- Be clever and use simillar stocks with high liquidity in the NAS-100 / DJIA with simillar periods of high return, low volatility. It needs to look like the algo is trading the same strategy on every stock it uses.
- Work for a couple of weeks with a stock screener to find the best stocks.
- Use low market exposure in order to get a very low DD.
- Use a simillar strategy for the live trading algo, even if the results are not so consistent with the backtest.
- Don't get greedy.
- Voilà! Evrika! You are in top 10.
Excuse my english, I'm not a native speaker.
**P.S.: Please don't use this exemple to create a contest algo, because I would be extremely disappointed.
