Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Catastrophic Failure(~150% drawdown) on mean-reversion algo
Ian Leatherbury
posted May 12, 2016
Seeking Help Getting Started Mean Reversion

Hey guys,

I am working on a basic mean reversion algo, using the PPO indicator as a ranking factor. I'm seeing a huge drawdown that only occurs under certain conditions. I haven't been able to narrow down the cause just yet. I'm hoping the community might have some ideas.

Conditions:

  1. If you run this algo from 7/1/15 through the present, it chugs along just fine, with average returns(~15%).
  2. If you back the algo up, and run from 6/1/15, there is a ~150% drawdown that occurs the week of 12/21/15, and runs through 3/7/16. During that time my leverage jumps from around 1, to around -4.

I've looked at the logs during those times, and my only guess is that one of my short positions is unable to be filled, and is skyrocketing, thus pushing my returns down.

Any ideas? Is it possible to prevent such a high drawdown?

11 responses
Josh Payne
May 12, 2016

Here's a pyfolio run for the algo you posted. It looks like there's something wrong with one of your long positions in the Dec 2015 timeframe. Maybe a missing split in the data (that would be my first guess).

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Deleted User
May 12, 2016

I looked at the logs. I think ticker TROV_U is the culprit.

Ian Leatherbury
May 12, 2016

Thanks guys, running to work now. Will check these out this evening. Thank you!

toan tran
May 12, 2016

brings up an interesting question for me. how do we know that there is not systematic error in adjustments made that are too small to detect but pervasive in backtesting?

Ian Leatherbury
May 13, 2016

Here are my findings:

  1. Yes it does look like TROV_U.
  2. There is no stock named TROV_U, but there is a TROV.
  3. TROV split in 2012, but not in 2015 where the issue is happening.(http://performance.morningstar.com/stock/performance-return.action?p=dividend_split_page&t=TROV)

@Josh, is it possible that a split was applied that should not have been?

Simon Thornington
May 13, 2016

brings up an interesting question for me. how do we know that there is not systematic error in adjustments made that are too small to detect but pervasive in backtesting?

Good question!

Vladimir
May 13, 2016

May be the question: "how do we know that there is not systematic small errors?" is interesting.
But for me more impotent question: "when Q going to fix hundreds of known big data bugs ?", like above, which pipeline pushing into algos.

toan tran
May 13, 2016

Vlad, this made me smile. I'm glad i'm not the only one who is frustrated.

Ian Leatherbury
May 13, 2016

Hey guys,

Still curious what the process is for getting this looked into on the data side. Is there a place to report bugs? Or is it possible for me to dig into this myself somehow?

Best,
Ian

Jamie McCorriston
May 13, 2016

Hi Ian,

This is a data bug. The price of $199,999.00 for TROV_U looks bad but it's not clear to me why. We'll have to do some digging to figure out why.

In general, the best way to report a data issue is by emailing in to [email protected]. Posting in the forums in tandem is a good idea so that others can see that it's been reported if they find the same issue.

Some data bugs are easier to fix than others. There will be a batch of data fixes rolling out soon but this bug won't be included in it simply because we don't yet know what's causing it.

In the meantime, my best suggestion would be to take a look at this notebook posted by Quantopian engineer Scott Sanderson. It's a very good filter for pipeline and I suspect it will take TROV_U out of your universe. It's a good baseline filter to remove a lot of securities that aren't usually good candidates for trading.

Thanks for reporting the issue.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Ian Leatherbury
May 14, 2016

Gotcha. Just getting started on the platform, so I wasn't sure if it was me or not. I've never met a platform without a bug though, so it's good to know what to keep an eye out for, and what to do with them :-)

Thanks for the help!

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.