Hi Andrew,

I think the problem in this case is that you're calling 'data[sid]' which calls all the data available for that sid, not just the price.

One way to fix that is to call 'data[sid].price' instead

    for sid in data:  
        now_close = data[sid].price  
        now_sma30 = talib.SMA(np.array([now_close]), timeperiod=30)  

Hope that helps,
Seong

Hi Seong,

Thanks for your response.

Why do I need the [] around now_close?

I tried this on the following code:

def handle_data(context, data):

    for sid in data:  
        now_close = data[sid].price  
        now_sma30 = talib.SMA(np.array([now_close]), timeperiod=30)

        now_macd, _, _ = talib.MACD(np.array([now_close]), fastperiod=12, slowperiod=26, signalperiod=9)  
        now_macd_sma9 = talib.SMA(np.array(now_macd), timeperiod=9)  

and receive this error:

Error   Runtime exception: Exception: inputs are all NaN  

Why isn't there enough data in the series to calculate the macd signal?

Or is it the case that in the early part of the day, there isn't enough minute data so I need to include a check like:

if now_macd_sma9 == None  
    return  

Thanks again,
Andrew

Hi Andrew,

Great questions.

You need the '[]' around now_close because np.array() takes in a Python list as a parameter and will throw an error if you try to pass in a single float.

And what's happening in the exception is that when you call data[sid].price you only get back the close price for that bar. So 'now_sma30' only contains one value in it's array, and the result of that list is '[NaN]'. How talib.SMA works is that it takes in the number of values in your np.array and returns back a similar array filled with a rolling average: '[NaN, 50, 49, 30]' and etc. And so when you're passing in 'now_sma30', a list with just 'NaN's in it, talib.MACD is throwing an error because all the inputs are NaN. You can solve this problem by passing in an np.array of more values to talib.SMA.

Just out of curiosity, is there a reason why you aren't using history() to obtain historical values? That might solve your problem here, but feel free to share your thoughts.

Seong

HI Andrew,

Here's an example that does what you do above:

for sid in data:  
        #: Get the daily close for each sid instead of using data[sid].price  
        daily_prices = history(bar_count=1, frequency='1d', field='price')  
        daily_close = daily_prices[sid]  
        daily_sma30 = ta.SMA(np.array(daily_close), timeperiod=30)  
        daily_macd, _, _ = ta.MACD(daily_close, fastperiod=12, slowperiod=26, signalperiod=9)  

Edit:

Looking over this now, you don't need the np.array(daily_close) as long as you're using history. But I think you need more values than the time period that you're using (e.g. if you're using timeperiod = 30, have > 30 values in your daily_close parameter)

I haven't used talib very much so if anyone would like to chime in with their expertise, feel free to do so!

Seong

Hi Seong,

Thanks for this again.

There is a reason why I'm using just minute data. When I use historical data the MACD values that I'm calculating using the historical data doesn't correspond with my technical analysis software and I'm trying to understand why. Hence, was going to try using minute data to see if I get anywhere closer. I read another post where this seemed to be the case but I can't find it more.

Are there any reasons that you can think of that could be causing this?

In effest, the cloned and adapted algo that I'm using doesn't fire an order when I expect it to and so, I'm getting strange results.

Any feedback will be helpful.

Thanks again.
Andrew

Hi,

For instance:

Alcatel Lucent (ALU) on the NYSE between periods 05/02/2013 and 02/28/2014.

My technical analysis software records the MACD and Signal to be the following:

05/06/2013 Close 1.31 MACD -0.021 Signal -0;018  
05/07/2013 Close 1.44 MACD -0.013 Signal -0.017  

Hence there is a cross over (a Buy signal)

But my algo is getting this values where MACD0/Signal0 = [-1] in the array and MACD1/Signal1 = [-2] in the array:
2013-05-06handle_data:116DEBUG 2013-05-06 02:00:00+02:00: close=1.33, MACD0=-0.0201, Signal0=-0.0186, MACD1=-0.0195, Signal1=-0.0188 2013-05-07handle_data:116DEBUG 2013-05-07 02:00:00+02:00: close=1.42, MACD0=-0.0131, Signal0=-0.0179, MACD1=-0.0201, Signal1=-0.0186

Not the same values at all.

Loading the values into an Excel spreadsheet using data from finance.yahoo.com I get:
06/05/2013 1.31 MACD -0.022342091 Signal -0.020220082 07/05/2013 1.44 MACD -0.013488888 Signal -0.018873843

In theory I'm using the same signal periods etc. Can you explain the difference?

My code:
``` import math
import talib as ta
import numpy as np
import pandas as pd

def initialize(context):
context.price={}
set_symbol_lookup_date('2014-10-09')
context.positions = [symbol('ALU')]
context.max_notional = 100000.1
context.min_notional = 0
context.day = None

def handle_data(context, data):
if get_datetime().day == context.day:
return
context.day = get_datetime().day
for sid in data:
close = data[sid].price

    daily_prices = history(bar_count=365, frequency='1d', field='price')  
    daily_close = daily_prices[sid]  
    daily_macd, _, _ = ta.MACD(daily_close, fastperiod=12, slowperiod=26, signalperiod=9)  
    daily_macd_sma9 = ta.SMA(np.array(daily_macd), timeperiod=9)  
    daily_macd_1 = round(daily_macd[-1],4)  
    daily_macd_2 = round(daily_macd[-2],4)  
    daily_macd_sma9_1 = round(daily_macd_sma9[-1],4)  
    daily_macd_sma9_2 = round(daily_macd_sma9[-2],4)  
    budget = round(context.portfolio.starting_cash * 0.25)  
    qty = round((budget - 9.99)/close)  
    cost = (close * qty) + 9.99

    if (( daily_macd_1 > daily_macd_sma9_1 ) and  
        ( daily_macd_2 <= daily_macd_sma9_2 ) and  
        ( context.portfolio.cash >= cost ) and  
        ( not ( sid in context.positions ) ) ):  
        order(sid, qty)  
        log.info("Buy {0} shares of {1} at {2}, MACD0  = {3}, Signal0 = {4}, MACD1  = {5}, Signal1 = {6},".format(qty, sid, daily_close[-1], daily_macd_1, daily_macd_sma9_1, daily_macd_2, daily_macd_sma9_2))  
        context.positions.append(sid)

    if (( daily_macd_1 < daily_macd_sma9_1 ) and  
        ( daily_macd_2 >= daily_macd_sma9_2 ) and  
        ( ( sid in context.positions ) ) ):  
        order(sid, -(context.portfolio.positions[sid].amount))  
        log.info("Sell {sid} at {c},  MACD0  = {m0}, Signal0 = {s0},  MACD1  = {m1}, Signal1 = {s1}, cost basis = {cb}, current price = {cp}".format(sid=sid, c=daily_close[-1], m0=daily_macd_1, s0=daily_macd_sma9_1, m1=daily_macd_2, s1=daily_macd_sma9_2, cb=context.portfolio.positions[sid].cost_basis, cp=context.portfolio.positions[sid].last_sale_price))  
        context.positions.remove(sid)  

Let me know what you think!  
Andrew  

Hi Andrew,

Thanks for looking into it. We looked into what's going on and Alisa pointed out that the differences in the signals might come from the fact that our data and Yahoo's data is slightly different. I.e. We don't have adjusted close prices the way Yahoo does and we only have data from 9:30am to 4:00pm without any data from after-market or pre-market prices.

So our close price will be, in some cases, slightly different than the data you get from Yahoo's close prices.

Dan elaborates on the differences here: https://www.quantopian.com/posts/quantopian-price-data-actual-close-or-adjusted-close so it might be a good read :)

Thanks and let me know if you have further questions

Seong

Hi,

One comment, I've not knowingly used the adjusted close prices from Yahoo in my calculations and my technical analysis software uses a real time data feed from alternate market sources (www.waldata.fr) so this doesn't explain why your prices are different.

If I'm going to get unpredictable results from your data source how can we trust the algo's we write? Surely, consistency is required?

Incidentally, Google says the closing prices for NYSE:ALU for 6 and 7 May 2013 are:

May 6, 2013 1.31  
May 7, 2013 1.44  

which corresponds to Yahoo and Waldata.

I don't get why your data source aren't using the same data. Why are you using different rules to create the daily data when, I assume, everyone is using pretty consistent ones as far as I can tell!

Sorry, to be a pain!
Andrew

HI Andrew,

I see where you're coming from. It does get a little confusing when you're comparing different data sources. Yahoo/Google take into consideration the open and close auctions in their OHLCV data while we provide the trade prices from between 9:30 AM and 4:00 PM, aggregated across all exchanges. Usually the discrepancy is too small to notice, but in this case, it seems like it makes a big enough different to result in different numbers for your ta-lib signals.

It also depends on how dividends are handled. Our close prices are not adjusted for the price, whereas other data sources, including Yahoo's are backwards adjusted. These differences in the data handling result in the discrepancies you're seeing.

Seong

That's good to know. Pre-Q a year ago comparing yahoo and google the occasional big difference (OHLC in whole dollars here to focus just on big diffs using CompareIt, plus Volume, which I was ignoring), and then saw fewer differences comparing google and nasdaq.com data, I went with nasdaq. Then I found Quantopian.

(On my wish list for Christmas from Wall Street to make volume more meaningful, added columns, number of buyers and sellers).