ok, so I did a test, ordered 100% of capital into SPY on day 1. I don't at all understand the alpha/beta values given by the results.

i would expect the alpha to be slightly negative, and the beta to be almost 1.0

Dan posted a good explanation of why your algo gives slight differences to benchmark returns here.

If you want to dig a little deeper, jump into the alpha and beta source code!

i don't mind getting different returns, what I am asking about is how the quantopian risk metrics are not documented and don't seem to follow industry conventions.

for example, beta, is a measurment of risk related to some industry "standard index". a value of 1.0 would imply "as risky". a value less than that implies "less risky". so I would assume that an investment into SPY would provide a Beta value similar to that of SPY (which is 1.0) but instead quantopian provides a beta of 0.75,

for alpha, a value of 0.0 means the same return as the standard index, with a value of less than zero meaning the returns are lower than the standard index. So I'm wondering why the Alpha for my example is greater than zero? that implies my returns are greater than the standard index.

thanks for posting the source code links, though skimming it doesn't really help me understand the difference between quantopians values and that of other finance websites.

ok it looks like if I click on the details for Alpha/Beta it looks more realistic. Does that mean that the "overall Metrics" values are being miscalculated?

For example, Beta only shows 0.705 for Feb 2008. Every other month is above 0.92, so the overvall Beta of 0.75 seems in error.

Beta will only be equal to 1 if the algo returns are equivalent to the benchmark returns.

Remove the if statement around order_target_percent to enable portfolio rebalancing for dividends and beta will be closer to 1.

ok, so in that case, my last statement about the overall Metrics being miscalculated looks even more evident.

In the following, every month except the first (which is only a couple days) has a Beta of 0.997 or above. Yet the final Beta displayed in overall metrics is 0.76

That is a bit strange, you may well have found a bug! Alpha doesn't look right either.

Hi Jason,

Thanks for pointing this out to us! I'm going to investigate this issue and you (or anyone else) are welcome to dig in as well. Take a look at this thread here: https://www.quantopian.com/posts/sharpe-ratio-other-risk-metrics-change-slippage-in-live-trading

In that post, we shared the excel file answer key against which we test our Zipline code logic. If our calculations match the answer key, we consider the code to be correct in the backtester. I'm going to run some local calculations and check the results against the Zipline code base and answer key. I'll report back my findings.

You may be right that there's a bug, and I'm curious to find out. I agree with you that we can do a better job documenting the risk metrics; it's something I've been thinking about but haven't gotten my hands on. Thanks for the encouragement to make it a priority! We want to be as transparent as possible about our documentation, calculations, and logic.

Best,
Alisa