Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Day vs Minute Backtest Results
Hayden McMaster
posted
Seeking Help Getting Started

Hi Everyone,

Apologies for asking a question that's been asked 1000 times here before, but it's still got me stumped.

I've recently started experimenting here and been tinkering with my first algorithm. It's a pretty simple algorithm; once per month it calls a scheduled function to generate a new list of stocks based on some fundamentals and rebalance the portfolio. Given that it's just a recurring monthly trade I would have thought that there would be minimal difference in results from daily and minute backtesting so I've been using daily while developing. Now that I've run it in minute mode I'm seeing a massive difference; in daily mode it gets 500% return over a 10 year period but it's down to 150% return over a 10 year period in minute mode.

Does anyone have any thoughts as other reasons why there might be such a large discrepancy? If I was doing something in handle_data I could understand, but I don't use handle_data at all. After hunting through the forums here the only difference that I'm aware of is that the daily backtest will buy / sell stocks at the close price of the following day while the minute level backtest will buy / sell at the price of the next minute. This would obviously cause some variations in fill prices between the two but I would have thought the overall impact would be minimal given the low trade frequency. There must be something else at work?

When looking at the transaction lists for both backtests most of the stocks selected are different which surprised me. I would have thought that building a portfolio at market open on day 1 would generate the same portfolio irrespective of if it was running in daily or minute mode, but that doesn't seem to be the case. Maybe the timing of the two is a bit different? Maybe given the different timing the daily backtest is just lucky and picking some big winners that the minute backtest isn't? It seems strange because the minute results are consistently below the daily results over the entire 10 year period; there's no one or two noticeable events that account for the difference.

Finally, this whole thing has got me thinking about what is reasonable in terms of returns. In minute mode (which I imagine is more close to reality) I'm getting 150% return over 10 years. From one point of view this nicely beets the market (SPY = 97%) over the same period so maybe that's a good result. From another angle the same algorithm was telling me 500% an hour ago so maybe it's a poor result. What's a reasonable target to shoot for? Does anyone care to share what sort of returns they are targeting with their algorithms?

Thanks in advance!
Hayden

2 responses
Simon Thornington

Another other major difference is that the slippage model will be applied to minute bars, not daily bars. If you are using the default quadratic slippage model on illiquid stocks (where your order is bigger than what the stock trades in any given minute), this means you're going to hit maximum slippage for one or more minute bars until the order fills, whereas if it's all rolled up into a single day, you might be getting less overall slippage.

Try without slippage at all, see if the results are closer.

Hayden McMaster

Hi Simon,

Thanks for your reply, that's a big help. I tried disabling slippage and the results are certainly closer.

Results from the minute test are still lower and now it seems to be due to leverage / fill rate. The daily backtest seems to fill all my orders at once like clockwork; I have a nice constant leverage of 1 for the whole backtest. The minute backtest seems to often take days, weeks or even months to fill orders and my leverage seems to fluctuate between .8 and 1.2. I don't think the stocks are particularly illiquid but 'll keep having a play with it.

Re slippage, I imagine it's not a major issue for small traders and could be safely ignored for all but the most illiquid stocks?

Thanks for your help!

Cheers,
Hayden

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.