Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Robustness of strategy with respect to parameters
Konsta Tiihonen
edited Apr 22, 2013

Yesterday's implementation of an algorithm made me wonder how to achieve robustness of a trading strategy. Having used something like moving averages with window sizes 10 days and 60 days, I dropped them to 30 and 6 afterwards. In addition I changed two other parameters and randomly the strategy improved from -10% total return to +150%. Although I accidentally arrived at that number and not by searching for it, I feel like I should use some method to check for robustness with respect to parameter change.

1) I can think of something like testing for "neighboring parameters" (+-1 day/minute in each rolling statistic, or +-10% etc.) and looking at changes in risk metrics etc. Is there a way to automate this process in quantopian?

2) Do you know of other methods to ensure/check robustness of a strategy with respect to parameter changes?

4 responses
Peter Cawthron
Apr 23, 2013

Hello Konsta,

The sort of optimisation you want is being discussed - see: https://www.quantopian.com/posts/feature-request-multiple-runs-for-testing-non-deterministic-algorithms

If you had your own data you could do this now with zipline (https://github.com/quantopian/zipline) perhaps with some 'walk-forward' testing. See: http://blog.quantopian.com/parameter-optimization/

Regards,

Peter

Simon Thornington
Apr 23, 2013

Yeah, if I were looking at optimizing parameters, I'd feel better if they were on a broad shallow plateau of profitable systems, rather than spiky points. If you are doing a brute force optimization of all possibilities of just a couple of parameters, it's easy enough to graph the heatmap to see. I guess if you were wanting to optimize directly to the parameters, you might want to try to maximize the sum of neighbouring systems? I would be interested to read any material about this.

Grant Kiehne
Apr 23, 2013

Hi Konsta,

I suggest considering if you could run the algorithm using zipline with a representative data set that you obtain. Then, once you understand what's going on, you can return to Quantopian and give it a whirl.

Grant

Konsta Tiihonen
Apr 24, 2013

Thank you. I will try zipline, if I can find appropriate data set. Another simple test: try the strategy on a different universe/sid.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.