Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Optimisation In Backtesting
Peter Cawthron
edited Oct 25, 2013
Q Meta

Hello Dan et al.,

The topic of parameter optimisation comes up regularly and I sometimes suggest zipline as a potential solution. But could something be done relatively easily with just a few modifictions to the existing GUI?

Imagine a simple dual MA crossover algo that goes long when MA(50) crosses MA(200) from below and goes short when MA(50) crosses MA(200) from above. Two optimisation parameters are Fast_MA period and Slow_MA period. Add in a Stop Loss that is a % of the filled price above/below a short/long order i.e. a third Stop_Loss percentage parameter.

The backtest options become 'Run Full Backtest' or 'Run Optimisation'. Selecting 'Run Optimisation' allows a finite number optimisation parameters to be given a Start, Finish and Step parameter i.e.

                Start     Finish      Step  
Fast_MA         40        60          10  
Slow_MA         100       300         20  
Stop_Loss       3         15          3

This is what Metatrader does now, more or less.

The algo is then run 165 times and the results are tabulated.

I think this would be easy to implement but, obviously, computationally expensive. But this is where you can start to charge users per unit of 'compute time' per month after a (generous) free allocation per month. Unused monthly units in the free tier to be carried forward. That last one would REALLY differentiate you. And it's a very quick way to get a revenue stream even if it only pays for the coffee machine initially!

P.

6 responses
Grant Kiehne
Oct 25, 2013

Yep, it's a no-brainer. Combined with the parameter scans, there needs to be a way to download the results in a file (e.g. csv, Excel, etc.). Or push them out to the log in a way that they can be copied-and-pasted into a local application.

Regarding charging, it seems that there must be lots of cheap, unused CPU cycles out there. Rather than rushing to get the results back to the user, just run the backtests at a low priority, and notify the user when the batch is complete.

And bummer...no free coffee at Quantopian Headquarters until they turn a profit! We should take up a collection...

--Grant

Peter Cawthron
Oct 25, 2013

Hello Grant,

I actually think charging for 'something' now is the way for Quantopian to go as it sets the expectation that there is a chargeable 'premium' tier. Perhaps more importantly a billing system could be implemented and debugged with something relatively trivial like 'excessive' compute time.

P.

Thomas Wiecki
Oct 28, 2013

Hi Grant and Peter,

Thanks for the suggestions. Certainly parameter optimization has been on our radar for a long time and what you propose makes sense (although there are some ways to be smarter than doing an exhaustive grid-search). We'd love to offer it -- it's mainly a question of resource allocation (engineering resources, not computing resources) at this point.

I do plan on writing a blog post that should give you instructions on how to do parameter optimization in zipline. It's quite easy to convert an algorithm to/from zipline to Quantopian (could be even easier, but still), so you can optimize in zipline and then plug those parameters into Quantopian to validate.

Thomas

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Marcus Williamson
Aug 31, 2015

Hi,

Is there a way of testing an algorithm that runs on minutely level data in the research environment? Has anyone got an example notebook of this working?

Thanks,

Marcus

Dan Dunn
Aug 31, 2015

Marcus, I suggest you open a new question - much better response than responding to one that is dead for a couple years!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Marcus Williamson
Sep 1, 2015

Hi Dan,

I wanted to ask here before I posted a new question, thats a point though! I believe I have an answer to my question, if I have further difficulties I will open a post and share what im stuck with etc.

Thanks,

Marcus

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.