if you run year by year you have to set the starting capital properly or it won't compound

Thanks for the comments, but why wouldn't it compound?

Yes, I think there is some sort of bug which gets worse in longer backtests. I have been seeing anomalous cash flows too:

https://www.quantopian.com/posts/debugging-anomalous-dividends
https://www.quantopian.com/posts/anomalous-appearance-of-cash-during-backtest

I recall someone else mentioning it was worse in longer backtests, which leads me to believe that it's a bug with their split-adjustment of dividends or something, but I haven't heard anything concrete. I emailed Dan about this a few days ago with my algo and some analytics in a research notebook which highlighted the error.

Thanks for the confirmation. I am pretty sure the cash is not dividend since I carefully compared the total balance between two scenarios:

Scenario A:
Don't start buying into any stock until January, 2013. But run the test starting from 01/04/2012. So it will keep 100% cash for a year.

Scenario B:
Don't start buying into any stock until January, 2013. But run the test starting from 2013 directly. (No cash balance for a year).

In these two scenarios, you will find that Scenario A has extra cash flows (about 3% - 4% a year) than Scenario B. I also observed dividend payment in B, so the cash flow in A is not dividend. It is not interest on cash either, because if it is, you would see some gains in year 2012 as well.

That's a huge issue and i've seen a similar anomaly on some of my backtests where cash is randomly ejected. FED bailout perhaps? lol

I've spent a lot of time on this algo, and I haven't found any bugs in the backtest yet. There are no cash additions or subtractions that I can't explain.

Mingran's original post includes this note:

Also, when I run it from 2003-01-04 to 2009-12-31, it is 515% gain. When I run it from 2009-12-31 to 06-24-2015, it is 93% gain. So the total gain is (5.15 + 1) * (.93 + 1) - 1 = 1087%. But when you run from 2003-01-04 to 06-24-2015, the gain is 1398%.

I think I understand why this is true. In my testing, I have found that the algorithm behaves differently depending on what day the algorithm starts. This difference takes two different forms.

1. The algorithm rebalances annually, So, if you start on December 29, your algo rebalances every January; on Jan 29, then you rebalance every February. You get different results depending on which month you start.
   
2. Even if you run multiple backtests that start on the same "cycle," it still isn't consistent. For instance, I kicked off one backtest that starts on Jan-03-2003, and a second backtest on Jan-29-2010, which is the February cycle, closest to Mingran's original post. Then, I check the holdings of the two backtests on Feb-16-2010. On that date, the longer backtest has 23 positions. The shorter backtest only has 20 (the same are in both tests, but the longer test has 3 more).

I didn't dive deep enough into the algo code to understand why the number of positions varies. But the positional differences very much can explain the differences in returns across multiple backtest runs. When the dates change, the behavior of the algorithm changes.

FYI, I did my study by running the following lines in the research environment. The backtest_id's are my own, and won't work for you, but you get the idea.

# 'full' refers to the backtest starting in 2003, 'second_six' refers to the one that starts in 2010 (bad naming convention, sorry)  
full = get_backtest('558dc24a175d18124f1f7252')  
second_six = get_backtest('558dd517f47f39123f134eef')  
january_2010_positions_full = full.positions.loc['2010-02-16']  
january_2010_positions_second = second_six.positions.loc['2010-02-16']  
january_2010_positions_full['sid'].isin(january_2010_positions_second['sid'])  
find_the_extras = january_2010_positions_full[~january_2010_positions_full['sid'].isin(january_2010_positions_second['sid'])]  

Dan, if you only enter positions starting Feb, 2010 (filter year to be 2010 only), then run two experiments, one start backtest from 2009-01-01, the other start from 2010-01-31, you might see the bug I talked about more clearly.