Hi Dan: I'm developing a new algorithm from scratch with the hope of building something worthy of receiving an allocation. When developing my competition algorithm I found a few small things came out of the wash during paper trading that never showed up during backtesting. So I'm paper trading a draft version of my latest algorithm with a similar hope.
Some of the issues I found while paper trading my competition algorithm, like intra-day slippage, I couldn't see without the fast resolution. Other issues, like subtle dynamics when building up the initial positions, I could have found during back testing, but I noticed because I was checking the behaviour of my competition algorithms every day, and happened to notice a transient that didn't occur during any of my backtests. I was standardizing the start/stop dates for my backtests, so I had only checked the initialization routines for a few time periods, which didn't occur to me until paper trading.
I realize there's a risk of survivorship bias if I do too much paper trading, so I try to be careful of this and generally don't stop/start paper trading unless I have a good reason, like a tweak or bugfix. I also keep two out of sample sets, Jan. 2015-Jan. 2017 and Jan. 2017-present. With the latest algorithm I've only tested the first out of sample set once, and the second set zero times (besides the paper trading starting today).