Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Major difference between contest and paper Sortino for the same algo. Why?
Adrian Boca
posted
Seeking Help Contest

Hi,

I don't quite understand why is there such a big difference between contest and paper Sortino for the same algo. http://postimg.org/image/rdd160vz5/
Please help. Can you please explain?
Isn't the same code valid for both? https://github.com/quantopian/zipline/blob/master/zipline/finance/risk/period.py 

def calculate_sortino(self):  
        """  
        http://en.wikipedia.org/wiki/Sortino_ratio  
        """  
        mar = downside_risk(self.algorithm_returns,  
                            self.mean_algorithm_returns,  
                            self.num_trading_days)  
        # Hold on to downside risk for debugging purposes.  
        self.downside_risk = mar  
        return sortino_ratio(self.algorithm_period_returns,  
                             self.treasury_period_return,  
                             mar)

There is not such a difference for the Sharpe ratio.

5 responses
Jamie McCorriston

Hi Adrian,

I wanted to let you know that I'm in the process of investigating this discrepancy. I will post here when I can get to the bottom of it!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Simon Thornington

My money is on the bug related to using 10 year risk-free rate for one thing, but using some shorter rate based on backtest-length etc for the other thing.

Adrian Boca

Hi Jamie,

Thank you for doing this. Please also keep in mind that yesterday was the last day of the October prize contest and, even if I'm currently #3 (before the last update), my algo had a really great last day of trading. So, any change in the calculation of the Sortino should come before the last leaderboard update.
Sorry for the pressure:) !

Thanks,
Adrian

Adrian Boca

After waiting 3 days for the last leaderboard update for the October Prize Contest, my algo just dissapeared from the leaderboard page.
I spent my last 9 months developing algos in the Quantopian IDE for your contest and I deserve a detailed explanation.
Also, the April prize leaderboard is a complete mess. I have a 5056 beta, for crying out loud. Everything is falling apart.

Adrian Boca

P.S. nobody from the Q team cared to resolve the Sortino problem before the last update

Dan Dunn

Hello Adrian - Part of the delay in announcing the winner was caused by tracking down the Sortino calculation bug. We fixed it before we generated the new leaderboard.

The April Prize leaderboard is still being generated - we have to recalculate everyone's rolling beta, and that burns a lot of electrons. It takes a couple days.

I'm sorry for the bug you found, and thank you for point it out to us. As for the April, please bear with us while we get that new contest started.

I just now published the October Prize process.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.