Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Resource limits in algorithm backtest?
David Byrd
posted Jul 31, 2016
API Machine Learning

My academic lab plans to use the Quantopian platform for an academic research project using machine learning techniques and simulated backtest performance.

Would it be possible to outline the resource constraints placed on (or recommended for) algo backtests? We would like to be good stewards of shared resources (and also know where any hard limits are).

For example, some of our approaches would use quite a bit of state. What memory limits are imposed on the context object?

Some approaches would require quite a bit of computation or runtime. This could mostly happen on a single day, followed by low consumption thereafter, or it could be spread out more evenly across the backtest. How is runtime limited, and are there per-simulated-day limits, or is the limit only for the whole backtest?

It it possible to construct a backtest in such a way that days (or the whole period) are replayed more than one time for learner training purposes, or must the backtest always be a single forward pass through time?

Any other advice you could offer along these lines? Thank you very much!

3 responses
Delaney Mackenzie
Aug 1, 2016

Hello David,

I run Quantopian's academic outreach.
https://www.quantopian.com/academia

We appreciate your desire to be careful with resource usage. For research, please do not exceed 4GB or else your kernel may become unstable. For backtests, please do not exceed 5 minutes in before_trading_start, 50 seconds per minute bar, or 2 hours total for the backtest. I don't have any info for backtesting memory usage currently, if at any point you notice your backtests becoming unstable or have any further questions please email me at [email protected]. I can provide some limited assistance to those using the platform for academic purposes, or at least point you in the right direction for more answers.

The backtest must be a single pass through time, however you can access historical data using the history function.
https://www.quantopian.com/help

Much of what you are trying to do is probably best approached first from the research environment, as you can rapidly iterate on prototypes before moving to the backtesting phase. For template research code and algorithms please see the lecture series:
https://www.quantopian.com/lectures

You can also see what libraries we whitelist here:
https://www.quantopian.com/faq

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

David Byrd
Aug 1, 2016

Thank you for the information, and we will definitely be in touch as our work progresses!

Grant Kiehne
Aug 2, 2016

Hi David,

Note that the backtester will support launching multiple backtests, that will run in parallel (by "backtester" I mean the browser-based IDE, with integrated performance charts, etc., not the IPython notebook-based research platform). Each backtest has to be launched manually (e.g. tweak a parameter by typing, and then click a button--I think the terms of use preclude automating the process with a script). Maybe Delaney knows, but I suspect there is no limit to the number of backtests that can be running in parallel, nor on the overall memory consumed, on a per user basis.

Backtest results can be pulled into the research platform using get_backtest (https://www.quantopian.com/help#quantopian_research_get_backtest). The documentation I found is kinda sketchy, but if I run x = get_backtest('579b7db0d9906d1005797b65') and then dir(x), I get:

['algo_id', 'attrs',
'benchmark_security',
'capital_base',
'create_bayesian_tear_sheet',
'create_full_tear_sheet',
'create_interesting_times_tear_sheet',
'create_position_tear_sheet',
'create_returns_tear_sheet',
'create_txn_tear_sheet',
'cumulative_performance',
'daily_performance',
'end_date',
'frames',
'launch_date',
'orders',
'positions',
'preview',
'recorded_vars',
'risk',
'scalars',
'start_date',
'stoppages',
'transactions']

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.