Let me restate your problem so I can be sure I understand it correctly. You're tracking the price of an ETF, and suddenly that price drops. It looks like the ETF isn't that good. But when you dig in, you find that your cash went up when the price went down - a dividend was paid into your account. Did I get that essentially correct?

When we built dividends into our system, we did part of it really right, and part of it we need to get better. The part that we got really right is that we're correctly replicating how dividends work in the real world - you get cash, and you need to manage that cash in order to reap the full benefits of the dividends in your long-term performance. Your overall portfolio performance is displayed very well in this scheme.

The part we need to improve is what you're running into now. We need to provide the dividend event information into the algorithm so that you can track the performance of individual securities. We are not providing you with enough information right now to correctly compute the overall returns of a specific security when it's paying a dividend.

The fix for this is pretty high on my priority list for future development. We haven't started it yet but it's something I know we need to get done.

Hello Frank,
Until this info is provided by quantopian (really, really high on my "must have" list), you may use Yahoo dividend feed in a separate fetcher to get dividends for specific instruments.

As well, I'm using this opportunity to thank you Frank for detailing your algo on SA and on your website. I've implemented your strategy in Quantopian (and zipline) with success. (to be quite honest, currently live trading UIS)
Best,

In my quest to build basket/spread market-making algorithms, I ran into the same problem - I basically had to dividend-adjust the spreads in back-testing to make sure that:

a) hedge ratios fitted on historical data make sense ex-dividend
b) meaningless drops in the asset prices ex-dividend aren't misleadingly interpreted as spread innovations

I am not sure where quantopian is with providing dividend-adjusted data?

I was also hit by this problem: https://www.quantopian.com/posts/tactical-bond-strategy

Hi Florent,

Do you happen to know the way to fetch dividend data for more than one ticker from yahoo? I'm asking because fetcher can be invoked only 5 times per algo, which makes it impossible to use for algorithm with amount of securities more than 5.

Here is a thread about it: https://www.quantopian.com/posts/methodcalledtoomanytimes-0033-fetch-csv-may-only-be-invoked-5-times

Regards,
Ed

Hi,
I've shared some code to compare Yahoo and Q 12-month rolling returns. The objective is to get something clean to investigate and derive conclusion on that matter.

Frank, I've shared an example of the paired_switching strategy where I've modified the code to easily change for signals to be generated from either Yahoo or Q.

Ed, I was not aware of that limitation. Eventually you could process your universe at night, upload that results in a single CSV and fetch it the next day considering FETCH is applied a hour before NY market open if I remember well. And if I may, I would use the R package to make sure the data is consistent as presented thereafter.

In the below R code, I basically show that for get 'split' only adjusted prices, I recalculate things as the 'adjust=c("split")' does not deliver as expected (or I don't use it correctly)

library(quantmod)  
symbols <- c("MSFT")  
from <- "2000-01-01"  
to <- "2013-12-31"

par (mfrow=c(2,3))  
for (i in 1:length(symbols)) {  
  getSymbols(symbols[i], from=from, auto.assign=TRUE)  
  plot(Cl(get(symbols[i])), main=symbols[i], type="p", pch=20)  
}

# use.Adjusted=FALSE (the default), the function pulls the split and dividend data from Yahoo and calculates the ratios manually  
getSplits("MSFT")  
getDividends("MSFT")

splitdiv = symbols  
split = symbols

# adjust for splits and dividends  
for (i in 1:length(symbols)) {  
  assign (splitdiv[i], adjustOHLC(get(symbols[i]),  
                                 adjust=c("split", "dividend"),  
                                 use.Adjusted=FALSE,  
                                 symbol.name=symbols[i]))  
}

# plot adjusted series  
for (i in 1:length(symbols)) {  
  plot(Cl(get(splitdiv[i])), main=paste0(splitdiv[i],".split-div"), type="p", pch=20)  
}


# adjust for splits  
for (i in 1:length(symbols)) {  
  assign (split[i], adjustOHLC(get(symbols[i]),  
                                 adjust=c("split"),  
                                 use.Adjusted=FALSE,  
                                 symbol.name=symbols[i]))  
}

# plot adjusted series for slipts only  
for (i in 1:length(symbols)) {  
  plot(Cl(get(split[i])), main=paste0(split[i],".split"), type="p", pch=20)  
}

getSymbols('MSFT', to=to, from=from)  
div <- getDividends('MSFT')  
spl <- getSplits('MSFT')

adj <- na.omit(adjRatios(spl, div, Cl(MSFT)))  
price_split <- MSFT  
price_split[,1] <- price_split[,1] * adj[, "Split"] # * adj[, "Div"]  
plot(price_split)

price_splitdiv <- MSFT  
price_splitdiv[,1] <- price_splitdiv[,1] * adj[, "Split"] * adj[, "Div"]  
plot(price_splitdiv)  

Hi Florent,

Ed, I was not aware of that limitation. Eventually you could process your universe at night, upload that results in a single CSV and fetch it the next day considering FETCH is applied a hour before NY market open if I remember well.

I implemented tiny web app to aggregate dividend data from http://ichart.finance.yahoo.com/table.csv?s=&g=v into one csv to work around this limitation. This way it should work automatically and requires just one call of fetch_csv.

Regards,
Ed