You can use the get_backtest() function in research to load a backtest. If you call the function multiple times, you can load multiple backtests and run a comparison.

Here are the API docs for the function: https://www.quantopian.com/help#quantopian_research_get_backtest

Is there a way to generate a rolling backtest? Basically, I want to be able to run a series of backtests where I simulate starting the algorithm on different dates and holding for a specified period of time. For example, it would be good to know how it performs if I start on Monday of each week in every year from the beginning of the available data until now while holding the portfolio for X [days, weeks, months, years]. Then, I would like a plotted summary of all the various annual returns for each week bucket and statistical breakdowns of all those simulations to know the average of all the average annualized returns, the median value, the stddev / 95% confidence interval, minimum, maximum, etc. I would like such stats and graphs for other important metrics like the Alpha, winning trade percentages, Sharpe values, etc.

Basically this is for testing the robustness of the trading algorithm so I can ensure I am not curve-fitting historical data and so I can catch potential holes in how my algorithm trades when starting / stopping in various different market conditions.

It probably would be good if you think of a way to cache pipeline calculations so they don't have to be re-computed every time. For example, a one-year run of my algorithm will end up computing all the pipelines for that period. If you offset by one week and go another year on the next backtest, then you should be able to drop the first week worth of cached results and simply generate the new final week of cached results and run the algorithm again using the cached results from the first run. This would save an incredible amount of computational overhead.

In the absence of such a feature, I am likely to be running hundreds of backtests manually to ensure I am not curve fitting, and in that case it's going to still be computing all those numbers over again every time I manually run a backtest. While it will take me a decent amount of time to do this manually, it's still going to happen, which means that load is still going to be present on your servers. Providing the rolling backtest functionality would simply make me more productive, make me happier, and save your system from a lot of extra computation.

Hi Eliot,

The ability to run rolling backtests as you described doesn't exist as a builtin tool on Quantopian, but it sounds like what you are asking could potentially be done in research. Take a look at this post that discusses parameter optimization. I realize that it's not exactly what you're asking for but it might be a could starting point in research that could be altered to what you're trying to do.

Regarding the performance comments, your caching suggestion is definitely an interesting one and it's something that will be considered. At this point, the manual route appears to be the only solution but the platform is always evolving and your comments here are valuable to us.

Regarding the caching, it would actually make a lot of sense if you computed a checksum of the factors and then built a cache system (maybe on top of a fast key-value store) where you cached the results. You could build a key based on the checksum, factor name, factor parameters, and date for which the values are calculated, then dump the computed results as a JSON struct (or whatever is most convenient) into a key-value store. Each time you need to look up the key, just touch a timestamp associated with the key and expire anything which hasn't been touched in 30 days. There are tons of benefits you would get from this:

1. It would save expensive CPU cycles and trade them in favor of much cheaper storage space
   
2. It would dramatically speed up running backtests as you would not need to re-compute every factor every time. You would only re-compute the factors which changed, because you wouldn't find a key (since the key would contain a checksum of the factor's code)
   
3. Anyone using publicly available built-in factors would help build a shared pre-computed cache of those factors being computed for stocks across the date ranges. This would expire within 30 (or fewer) days of someone changing the source of the factor, but that wouldn't happen often.
   
4. When running algorithms live, again, the first algorithm to compute a factor on new data would create a cached copy of that computation for others to use
   
5. People could freely use a much larger number of factors in their algorithms because of the drop in computation resources
   
6. It would encourage people to submit their factors for public use because then more people would help build up a readily available cache of pre-computed data which would allow them to continue to use a larger number of factors in their algorithms

If you are worried about issues with the cache in case you change the way you handle data, you could include a "serial number" in the key name, as well, which you could bump any time you make a change to the overall system which would invalidate the previously cached results.

The API just says about the 'create_full_tear_sheet' function. Isnt there a way to get say returns from a particular security?