I've run into a few situations where I've built an algorithm that has some free parameters. Trying to optimize them using repeated backtests is very tedious. Does anybody have advice for how to do this more efficiently? I've thought about setting up a series of backtests with zipline locally, but I don't have access to fundamentals data on my local machine.
Note: I am aware that I should avoid data snooping bias, which is why I'd use different backtesting timeframes for tuning and for testing.
This is part of ongoing research and we're currently writing a blog post about our approach and results. I think it's quite cool. That work however will not result in having optimization available on Quantopian (at least not at first) so I'm afraid this feature is the undetermined future but stay tuned for the blog post!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
One thing I've done is to write a parameter out to the log at the outset of the backtest. I'll manually vary the parameter over a series of backtests that I run in parallel in multiple browser tabs (I've had as many as about 10 backtests running at once). Upon completion, there is a fair amount of copying and pasting into Excel for analysis.
My sense is that much of this tedious manual data input and output could be automated with a script that effectively controls the mouse and keyboard. However, it might run afoul of the Quantopian terms of use (https://www.quantopian.com/policies/terms). For example, you cannot:
--use any engine, software, tool, agent or other device or mechanism to navigate or search the Site, other than the search engines and agents available through the Service and other than generally available third party web browsers;
--use the Site or the Services in any manner that could damage, disable, overburden or impair Quantopian’s servers or networks, or interfere with any other user's use and enjoyment of the Site and/or the Services;
Even a simple 10x10 or 10x10x10 grid search would increase the load for an individual user by 2-3 orders of magnitude. Aside from the security concern of wholesale dowloading the restricted bar data, my hunch is that Quantopian has made batch processing awkward so that they can keep the load down, without having to deal with throttling individual users (which is reasonable, since it's a free service).
Thomas, have you sorted out how to run parallel code in the research environment yet? Also, what do you mean by "That work however will not result in having optimization available on Quantopian"? Are you still thinking along the lines of asking users to port their algos over to the research environment for study/optimization and then port them back to the online backtester? Why not just put some hooks into the online backtester (or a separate specialized instance of it, running as part of the research environment)? What am I missing? It seems like a "no-brainer."
Grant
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
