Quantopian calculated Sharpe ratios seem much too high and don't match the zipline risk metrics

Back to Community

posted

In most of my algorithms the calculated Sharpe ratio in the results overview seems much too high. If I run the same algorithms in QuantShare or ETFreplay, then the Sharpe ratios are most of the time less than half of the Quantopian Sharpe ratio. How is this Sharpe ratio calculated? It seems not to be the annualized return / standard deviation.

If I do a full backtest and look at the risk metrics Sharpe table, then the numbers seem correct. See the copy below. If you calculate the mean of all 12 month sharpe ratios, then you get about the same as I get with other software programs.

In this example Quantopian shows a Sharpe ratio of 3.76 and the mean 12 month Sharpe ratio is 1.33. 3.76 is more than 2x too high.

Best regards Frank

Risk metrics sharpe table (last number is 12 month Sharpe)
November 2004 1.869 2.619 1.964 0.832
December 2004 0.926 1.642 1.98 1.319
January 2005 -0.179 1.2649 2.283 1.157
February 2005 3.122 1.49 2.696 1.579
March 2005 -1.016 0.057 1.327 1.048
April 2005 -0.142 0.046 1.0009 1.596
May 2005 0.34 -0.666 0.64 1.579
June 2005 0.638 0.171 0.144 1.573
July 2005 0.357 0.735 0.354 2.04
August 2005 0.442 0.795 -0.294 1.821
September 2005 2.633 2.146 1.413 2.066
October 2005 -0.4061 0.6 0.863 1.406
November 2005 1.665 1.0209 1.284 1.3759
December 2005 0.6939 0.558 1.427 1.221
January 2006 1.461 1.838 1.787 1.711
February 2006 -0.593 0.82 1.3779 0.912
March 2006 -0.056 0.292 0.637 1.3799
April 2006 0.783 -0.12 1.3959 1.741
May 2006 -0.48 -0.103 0.585 1.3779
June 2006 0.0509 -0.026 0.15 1.22
July 2006 0.135 -0.198 -0.241 1.178
August 2006 1.305 0.369 0.191 1.268
September 2006 0.469 0.719 0.311 0.741
October 2006 0.934 1.603 0.261 1.301
November 2006 1.218 1.633 1.008 1.1399
December 2006 -0.551 0.77 1.075 0.68
January 2007 -0.226 0.009 0.677 0.137
February 2007 0.924 -0.124 0.664 0.558
March 2007 -0.735 -0.055 0.395 0.516
April 2007 0.916 0.769 0.3469 0.449
May 2007 0.104 0.184 -0.048 0.794
June 2007 -0.3411 -0.047 -0.072 0.633
July 2007 -0.071 -0.232 0.005 0.422
August 2007 -0.088 -0.222 -0.184 0.158
September 2007 1.477 0.182 0.161 0.365
October 2007 1.447 0.641 0.333 0.502
November 2007 0.168 1.935 0.365 0.371
December 2007 -1.003 0.249 0.265 0.249
January 2008 0.074 -0.494 0.415 0.411
February 2008 0.279 -0.374 0.999 0.335
March 2008 -0.177 0.04 0.231 0.329
April 2008 0.703 0.339 -0.011 0.3439
May 2008 0.96 0.744 0.371 0.548
June 2008 -1.009 0.792 0.525 0.543
July 2008 -0.714 -0.395 -0.1401 0.188
August 2008 -0.013 -0.852 -0.226 0.295
September 2008 0.132 -0.461 -0.062 0.064
October 2008 -0.352 -0.086 -0.346 -0.299
November 2008 1.812 1.1339 0.262 0.452
December 2008 1.949 2.309 1.197 1.409
January 2009 -2.532 0.91 0.675 0.46
February 2009 -0.526 -0.486 0.441 0.221
March 2009 0.947 -0.997 0.808 0.603
April 2009 1.034 1.113 1.54 1.048
May 2009 1.5189 2.2559 1.621 1.546
June 2009 -0.152 1.635 0.832 1.499
July 2009 1.158 1.647 2.186 2.394
August 2009 0.751 1.0129 2.682 2.64
September 2009 0.99 1.715 2.557 2.844
October 2009 -1.069 0.586 1.809 2.753
November 2009 1.418 0.889 1.34 2.376
December 2009 -0.783 -0.074 1.458 1.58
January 2010 -1.854 -1.061 -0.419 1.669
February 2010 1.221 -0.819 -0.247 2.228
March 2010 1.827 0.084 0.026 2.2919
April 2010 1.93 2.716 0.755 2.066
May 2010 0.217 2.463 0.298 1.295
June 2010 0.752 1.727 0.732 1.679
July 2010 1.319 1.393 3.169 1.389
August 2010 1.29 2.051 3.38 1.421
September 2010 1.585 2.559 3.214 1.545
October 2010 0.119 1.578 2.172 1.89
November 2010 -0.484 0.296 1.318 1.086
December 2010 1.824 0.327 1.686 1.728
January 2011 0.461 0.505 1.419 3.453
February 2011 1.097 1.894 1.439 3.365
March 2011 0.293 1.157 1.0149 2.7519
April 2011 1.194 1.48 1.346 2.558
May 2011 1.038 1.246 2.35 2.699
June 2011 -0.661 0.719 1.387 2.318
July 2011 0.575 0.434 1.445 2.129
August 2011 0.889 0.875 1.301 2.068
September 2011 0.628 1.214 1.403 1.84
October 2011 1.358 1.352 1.439 2.107
November 2011 -0.026 0.902 1.227 2.442
December 2011 0.585 0.871 1.526 2.152
January 2012 1.201 0.772 1.604 2.269
February 2012 0.153 1.1299 1.427 1.948
March 2012 -0.565 0.3439 0.916 1.743
April 2012 0.489 0.0429 0.633 1.612
May 2012 0.709 0.315 1.014 1.604
June 2012 -0.984 -0.121 0.112 1.422
July 2012 1.408 0.243 0.206 1.539
August 2012 0.054 -0.042 0.1739 1.175
September 2012 1.046 1.434 0.83 1.2709
October 2012 0.802 1.104 0.926 1.116
November 2012 0.423 1.231 0.85 1.342
December 2012 0.921 1.137 1.854 1.409
January 2013 -0.613 0.434 1.061 0.912
February 2013 0.74 0.513 1.314 1.041
March 2013 2.313 0.749 1.3839 1.592
April 2013 1.092 2.092 1.5149 1.776
May 2013 -1.349 0.364 0.627 1.076
June 2013 -0.75 -0.758 -0.286 1.091
July 2013 2.181 -0.132 0.881 1.416
August 2013 -0.837 -0.161 0.033 0.841
September 2013 1.798 1.1359 0.331 1.084
October 2013 1.36 0.913 0.603 1.3839
November 2013 0.303 2.053 1.149 1.338
December 2013 0.558 1.3819 1.845 1.288
January 2014 -0.622 -0.042 0.617 1.037
February 2014 0.308 0.008 1.2609 0.977
March 2014 0.025 -0.261 0.6969 0.746
April 2014 0.744 0.547 0.295 0.649
May 2014 1.623 1.256 0.546 1.258
June 2014 -0.618 0.865 0.139 1.491
July 2014 0.6879 0.874 0.984 1.089
August 2014 1.299 0.903 1.522 1.955
September 2014 -1.126 0.667 1.076 1.181
October 2014 0.622 0.641 1.074 0.875
November 2014 0.011 -0.091 0.595 0.801

4 responses

Dan Dunn

We can certainly look into it further. It would be helpful if you can share the actual backtest so we can run the test ourselves and get the daily returns. Also, one of the most common reasons things like Sharpe can vary is things like whether it's looking at minutely, daily, or monthly returns (Quantopian looks at daily returns). It's not clear to me what those other sites are using.

We had a bug in our calculation of Sharpe, at least the one in the "top line results," that we fixed in April. That post includes two really helpful elements, the formulas, and a link to the "answer key" that we run our tests against. It might help in understanding the calculations that you are seeing.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Frank Grossmann

It seems to have something to do with the use of leveraged ETFs. Here is a simple cloned backtest which rebalances TMF-UPRO.
In the results overview it shows a Sharpe of 3.7 for the period 2009-06-26 to 2014-09-17. This seems much to high.
If I calculate the mean sharpe of the Zipline risk metrics table for 12 month sharpe, then I get a Sharpe of 1.7 which seems right.
If I simulate the same in ETFreplay for the same period, then I also get a Sharpe of 1.7. I get the same also with Quantshare.

So, there seems something to be wrong in the results overview sharpe ratio. If I use normal non leveraged ETFs, then the results are ok.

Seong Lee

Frank,

In that last backtest you've attached can you clarify a few things for me?

In this question "If I calculate the mean sharpe of the Zipline risk metrics table for 12 month sharpe, then I get a Sharpe of 1.7 which seems right." Are you taking the aggregated values of all portfolio values and calculating an annualized sharpe ratio over that? E.g. (Portfiolio_returns*252)/(portfolio_std*sqrt(252))

I ask because in the first post you posted this:

September 2013 1.798 1.1359 0.331 1.084 October 2013 1.36 0.913 0.603
1.3839 November 2013 0.303 2.053 1.149 1.338 December 2013 0.558 1.3819 1.845 1.288 January 2014 -0.622 -0.042 0.617 1.037 February 2014 0.308 0.008 1.2609 0.977 March 2014 0.025 -0.261 0.6969 0.746
April 2014 0.744 0.547 0.295 0.649 May 2014 1.623 1.256 0.546 1.258
June 2014 -0.618 0.865 0.139 1.491 July 2014 0.6879 0.874 0.984 1.089
August 2014 1.299 0.903 1.522 1.955 September 2014 -1.126 0.667 1.076
1.181 October 2014 0.622 0.641 1.074 0.875 November 2014 0.011 -0.091 0.595 0.801

But when I look at the results of your backtest, the sharpe ratios that are given are very diferrent e.g. :

June 2014 0.521 1.963 2.7879 2.407
July 2014 -0.256 1.091 2.149 2.043
August 2014 3.34 1.734 2.816 3.865
September 2014 -1.869 0.367 1.61 2.585

Could you provide what exact time periods you used for the first post?

Sorry if I'm taking too much of your time, but we're looking to make our risk calculations as accurate as possible and we need to gather enough details to determine whether or not there is actually something off.

Thank you,
Seong

Disclaimer

Frank Grossmann

The first numbers are of a different strategy. I could not share the algorithm of the first post, because it is private, so I just took another one (TMV-UPRO). This second one has also the advantage that it is very simple and can so be backtested very easily in ETFreplay and Quantshare. I thought this way it is easy for you to replicate the error. This seems to be a general problem because I see this discrepancy between zipline risk metrics Sharpe to the overall Sharpe in most of the algorithms I have tested.

You've successfully submitted a support ticket.

Our support team will be in touch soon.