Hello Alessio,

When we find signals that differ from other tools, we generally try to track down the root cause. If it's a bug, then we fix it. Most of the time it ends up being a difference with a logical explanation.

In this case, I don't think it's a bug, it's a question of data parameters. MACD uses a window of data. Google and Yahoo use windows that grow in length over time. By default, Quantopian uses a fixed window. One can argue that one approach is better than the other. I'd argue that our approach is better because you don't want your signal to vary on the basis of what day you started the test.

Eddie went into it in some detail in a post this summer.

If you have other places where you see signal issues, please let us know. Of course, we do have a different pricing data source than Yahoo and Google, so some minor price differences are expected. But in general there should be a match.

Thanks,

Dan

Hi Dan,

thanks for the quick reply and explanation. While I do understand the underlaying motivation, I can't agree with you on the "better approach".

Every day thousands of investors invest actual money relaying on the signals provided by TD Ameritrade, Yahoo! and Google. If yours was a "better approach", I think either these providers would have followed it or investors would rely on the ones who did.

Personally, I wanted to move all my strategies to Quantopian and connect them with my IB account, but I can't because they do not work here.

Simple things like MACD crossovers happen at different times respect to the other trading platforms I use. So I can't do it. I will probably leave the platform and get stuck again with implementing my own code for fetching data and testing strategies, which sucks.

It depends on what you're optimizing for. By the Google/Yahoo method, the MACD values are a function not only of the price, but also a function of how many days have gone past in the year.

Imagine if in 2014, the price for Apple is exactly repeated every 34 days. I think the 35th trading day of the year will be Feb 19. So the price for Jan 2 will be repeated on Feb 19, Jan 3 on Feb 20, and so forth. That cycle would be repeated 5 times over the year. Now, imagine you evaluate the MACD on the 5th day of any one of those 34 day cycles. In the fixed-window version, you'd always have the same value - the previous 34 days are always the same. In Google and Yahoo, you'd have a different value every time - they'd be calculating using an ever-increasing window that starts on Jan 1.

Personally, I think that the determinism of the results using the fixed windows are preferable. Google and Yahoo have made a different choice. They've chosen to fix their window start on January 1. It's an easier calculation to start from the first of the year, and so they do it.

All that said, there's no technical reason that one couldn't reproduce the values seen on Google and Yahoo within Quantopian. Rather than using our helper functions I think you could use imported ta-lib and create a variable window length calculation. I'll see if I can find someone to help me code that up.

I am willing to bet TD Ameritrade, Yahoo! and Google all have different values for each respective indicator. Whether due to differences in how they are calculated or data source. Yahoo uses CSI (EOD only) for historical data vs Nanex (23ms) at Quantopian (aggregated to 1min), just look at pricing if you aren't convinced as to who has the better feed. Look at a Bloomberg vs a Thompson Reuters stream and you are still going to see discrepancies, by the way these feeds are 3k a month base no frills. If you really aren't happy with the methodology Quantopian has built in, then simply roll out your own MACD that has an expanding window, pandas has expanding window functions that would make this stupid simple. As Dan explained above moving window operations are going to have the same value after the initial burn in, while expanding windows are dependent on start, end dates. Since you have access to python you are only limited by your imagination as to what you can implement. If you still feel limited fork zipline and modify the library. I have my doubts that switching to quantopian is the root of your broken strategy, you may just be under more scrutinized assumptions now even without leverage restrictions or you had some bias that was exaggerating its effectiveness.

This book essentially shows Technical Analysis provides no predictive power in forecasting equity returns
http://www.amazon.com/Evidence-Based-Technical-Analysis-Scientific-Statistical/dp/0470008741

Hi Dan,

are you sure they restart on January 1st of that year? If that's the case, wouldn't every signal that requires some warmup not be available for the first days of the year? I personally think they just start from the first day they have data for that symbol. That is what I personally would recommend to anybody implementing these formulas.

Google/Yahoo are just examples. ScotTrade and TD Ameritrade, two of the biggest financial platforms, have also the same values for MACD. Just Quantopian is different.

@Brandon, you would lose your bet! :) They all have the same value. Check MACD for GOOG this year for example. Here are a links to YCharts (http://bit.ly/1ao57AC), Google Finance (http://bit.ly/1iYk7tk) and Yahoo! Finance (http://yhoo.it/19j4BVV). If you look at any random day, for example September 3rd 2013, the values are the same (MACD -9.97, signal -8.81, divergence -1.15) on all the platforms.

I am not so sure, these seem to be using a similar fixed window operation which gives approximately the same result however the float precision used is not sufficient to say they are identical. If you start changing the aggregation periods on any of those charts you will end up with wildly different values despite the frequency still being daily, whereas the moving window would have the same value across all aggregations. There was already another thread that addressed this, and even using the current implementation the resulting differences are pretty negligible in my opinion. It truly boils down to differences in how the data is chunked into bars which results in minor differences, whereas the windowing is where the real variation lies. Again if you really need it, you can certainly implement it

Hi @Brandon, the MACD values for September 3rd 2013 on Quantopian are MACD -12.64, Signal -12.23, divergence -0.41. The difference seems significant to me.

Also, crossovers (for same signal with same parameters, e.g., MACD[12,26,9] ) happen on the same day on TD Ameritrade, Google Finance, Yahoo Finance, YCharts and Scottrade, but not on Quantopian. Which could be even a bigger issue, in my opinion.

Here http://ta-lib.org/d_api/ta_setunstableperiod.html the author(s) of TA-Lib discuss this issue.

P.

By default, Quantopian uses a fixed window to compute the MACD. TAlib requires a window of at least 33 bars, and we are using 34 bars to compute MACD. Google and Yahoo are using a much larger set of data to compute the MACD. When you use the larger window like that, the MACD numbers begin to "converge" on a value. You can use a larger window in Quantopian. To obtain a MACD result nearly identical to that of Yahoo Finance or Google Finance, use the code below:

# Python TA-Lib wrapper  
# https://github.com/mrjbq7/ta-lib

import talib

def initialize(context):  
    context.my_sid = sid(26578)

def handle_data(context, data):  
    price_history = history(bar_count=255, frequency='1d', field='price')  
    my_sid_series = price_history[context.my_sid]  
    # Compute 'fast' exponential moving average  
    ema_fast_result = talib.EMA(my_sid_series, timeperiod=12)  
    # Compute 'slow' exponential moving average  
    ema_slow_result = talib.EMA(my_sid_series, timeperiod=26)  
    # Compute the difference between the 'fast' and 'slow' EMA  
    ema_diff = ema_fast_result - ema_slow_result  
    # Compute the exponential moving average of the difference time series  
    our_macd = talib.EMA(ema_diff, timeperiod=9)  
    # Compute MACD directly using TAlib  
    ta_macd = talib.MACD(my_sid_series)  
    # Record relevent time series  
    record(ema_diff=ema_diff[-1])  
    record(our_macd=our_macd[-1])  
    record(talib_s0=ta_macd[0][-1])  
    record(talib_s1=ta_macd[1][-1])  

As one increases bar_count, the function coverges to the value observed on Yahoo, Google, etc., so simply extend the window to obtain the desired result.

For more granular insight, the function decomposes MACD into the 26-day and 12-day EMA, and then takes the 9-day EMA of the difference between the two. The result matches the result from when MACD is called directly. You can see the decomposed function in my custom data area.

That's great Ryan, thank you for telling me about history() and how to use talib.MACD directly!

Unfortunately it seems that history() is only available for minute data. Any way around that?

Hey Alessio, currently history() is only supported in minute mode. See https://www.quantopian.com/help.

Alessio, is there something you are trying to do in minute mode that you can't do in daily mode? I believe that pretty much everything that works in daily will also work in minutely.