First, my background is in modeling electronic circuits (differential equation solvers) so I was excited on finding Quantopian, since it looks like what I do for a living. We can model circuits very accurately, so I figured this stuff should work.
Note at this point I am interesting in trading my own money as opposed to the Quantopian contest.
I have not tried minute buy minute stuff since I can't trade at that rate.
I developed some algorithms, mostly by improving algorithms from this forum which gave good back test results.
I was surprised how quickly they failed with real money. Failed means giving a drawdown larger than
the backtest over a period of several years.
First I tried the "Gold King algs". I really did't expect this to work (due to liquidity issues and incorrect accounting of delisted stocks) but figured it would be a good test for Robinhood. It didn't work and robin hood had problems I was not expecting,
orders being canceled etc.
Next I tried a real simple trend line basically the 80day one from Anthony. Simplicity is very important to avoid overfitting. There are papers proving that overfitting is real and issues with it are encountered in other areas of modeling. This one failed due to the drop in bond prices. My code assumed TLT was safe.
From my backtesting the only thing I have observed to work reliability is momentum, and only when there is a strong upward or downward trend. In flat markets it whip-saws and underperforms. In strong markets many stocks work, but momentum does let you pick better ones (although I don't know they are better than a 3X bull ETF)
Grant gives a good explanation. The noise to signal ratio ratio is large, and becomes larger in a flat market. It is impossible to predict noise so the algorithm underperforms in flat markets like we have had recently. In backtesting we try and run through as long a time range as possible which includes the strong markets so we conclude the alg is better than it is. On the other hand the backtests don't go back far enough to test things like rising interest rates.
I doubt shorter term algs will work better, noise is larger over shorter time periods. Machine learning is likely to take overfitting to the extreme, although it may uncover bizarre hidden trends like stocks that go up on the third Thursday when there is a full moon.
Fundamental data seems to lag too much, and I have been burned during hand trading by companies that lied about their numbers.
Finally the efficient market theory is working against us.
I hope someone can explain why the above is incorrect...