Hi Andrew,

The error you're seeing is telling you that, when you do

context.daily_sma30[sid] = ta.SMA(np.array(daily_close), timeperiod=30)  

You're trying to set a value you in your daily_sma30 list with a Security object as a key rather than an integer.
A simpler example of what you're doing would be something like:

In [1]: mylist = [1,2,3]

In [2]: mylist['foo'] = 4  
---------------------------------------------------------------------------  
TypeError Traceback (most recent call last)  
 in ()  
----> 1 mylist['foo'] = 4

TypeError: list indices must be integers, not str  

In python, lists indices have to have integers, and you can only set into a list at a given index if it's already of the right length. For example, with the same list, if you try to add a key at index 0 (the first index) it will work.

In [3]: mylist[0] = 5  
In [4]: mylist  
Out[4]: [5, 2, 3]  

But if you try to add a key at index 6, it will fail, because the list isn't that long

In [6]: mylist[6] = 5  
---------------------------------------------------------------------------  
IndexError Traceback (most recent call last)  
 in ()  
----> 1 mylist[6] = 5

IndexError: list assignment index out of range  

Since, in your case, you want to be storing a map from arbitrary keys to values, what you want to use is a dicitonary. The way to initialize an empty dictionary is with curly braces instead of square brackets.

So in summary, the fix for your issue is to change

context.daily_sma30 = []  

to

context.daily_sma30 = {}  

Also, if you're new to python, I'd wholeheartedly recommend taking a look at the standard docs on the Core Data Structures. Understanding how and when to use list vs dict vs tuple vs set is a big part of using Python effectively. If you're feeling ambitious you could also take a look at Pandas, the library we use for tabular data structures. Among other things, the history() API for loading trailing windows of data is built on top of pandas, and lots of our internal systems use Pandas for storing and manipulating tabular and time series data.

Hope that helps,
-Scott

Thanks for your help Scott!

Having changed the [] to {} to make context.daily_sma30 an associated array (dictionary item) I'm now getting this error:

Error   Runtime exception: KeyError: Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 10, 1, 0, 0, tzinfo=), first_traded=None)  

It seems to be happening on line in the handle_data function (not in the original code snippet)

daily_sma30 = context.daily_sma30[sid]  

This is the first time that I try to use context.daily_sma30 dictionary item. In theory, sid should equal 24 (for Apple Inc) but I don't understand why this error is being produced.

Within the intialise_context funciton I've added a line:

context.daily_sma30.insert(sid,ta.SMA(np.array(daily_close), timeperiod=30))  

to make and index of 24 (Apple Inc) to be a list producted from the ta-lib SMA call.

Any clues what"s going on now?

Sorry to be a pain!

Thanks in advance.
Andrew

Andrew,

I think I understand where the issue might be but it'd be great if you could post what your code looks like now? Or if you'd like to keep it private you can send me an invitation to collaborate and I could help you out there.

Seong

And I realized that I didn't include my email: [email protected]. Feel free to reply here or send me a email!

Seong

Thanks Seong, I've invited you.

Andrew,

Check out the algorithm you've invited me to now. I've changed and commented a few things and let's talk in the chat there when you see this

Seong

I'm also new to Python and am attempting to do something very similar for my first algo on Quantopian. Would you consider sharing your solution or at least the initialise portion?

I'd like to initialize each stock differently based on whether or not it is a new day or not. If its not a new day, i only want to pull 1 day of history and update the last entry of my array. My goal is to set up a few indicators in the context of each stock, for example: context.ADX, context.MACD & context.RSI. Based on whether or not the stock is trending and the respective indicators, I'll individually determine if each stock is a buy, sell, hold or none of the above.

Any help would be greatly appreciated and i'd love to see where your ideas led and share mine.

[email protected]
-Austin

Austin,

I'm not able to share Andrew's solution unless he chooses to, but if you'd like, you could post what you have here and myself and others will be able to help you out (and we can help out anyone else who might have the same questions). In the meanwhile I can post a few code snippets that might be useful:

Pulling 1 day of history

price_of_stock = history(1, "1d", "price")["sid name here"][-1]  

Updating the last element of an array (list)

#: Initialize a new list  
new_list = [1,2,3,4]  
#: Update the last element  
new_list[-1] = 5  
#: new_list is now [1,2,3,5]  

Feel free to ask any more questions on this!

Seong

Thanks for your help Seong, it aided me greatly.

Austin, I've no problem sharing my code with Seongs corrections. Hope it helps:

import math  
import talib as ta  
import pandas as pd  
import numpy as np

def initalise_context(sid, context):  
    historical_close = history(bar_count=365, frequency='1d', field='close_price')  

    daily_close = historical_close[sid]  
    #: Instead of insert use this notation which essentially says:  
    #: In this dictionary, update this key (sid) with this value (ta.SMA(np.array(daily_close), timeperiod=30)) and if it doesn't exist  
    #: Create a new key (sid) with this value (ta.SMA(np.array(daily_close), timeperiod=30))  
    context.daily_sma30[sid] = ta.SMA(np.array(daily_close), timeperiod=30)  


def process_order(sid):  
    return  


def initialize(context):  
    context.stocks = [symbol('AAPL'),symbol('MSFT')]  
    context.max_notional = 100000.0  
    context.min_notional = 0

    context.day = None  
    context.daily_sma30 = {}  

def handle_data(context, data):  
    #: Instead of asking context.daily_sma30 == None, ask if the length of it is 0  
    #: Because None only applies if the object is actually None, where in this case it isn't because 'context.daily_sma30 = {}' assigns  
    #: context.daily_sma30 to a dictionary object  
    if len(context.daily_sma30) == 0:  
        for sid in context.stocks:  
            initalise_context(sid, context)  
    for sid in context.stocks:  
        close = data[sid].close_price  
        daily_sma30 = context.daily_sma30[sid]  
        if get_datetime().day == context.day:  
            return  
        context.day = get_datetime().day  
        initalise_context(sid, context)  

Good Luck Austin and thanks again Seong!
Andrew

Hi everyone,

I want to do something similar to Andrew but my question is not as python specific. Let's say I want to build an algorithm that buys when the all time historic closing price of a security has been exceeded. For example, security XYZ's highest closing price ever is $10. I need to be able to know that and every minute check/ update that value so I know when to buy. It's similar in that the original question where Andrew wants to pre-calculate SMA for a security for the previous year before running his algorithm. I want the highest closing price of all time --- even including years prior to the quantopian dataset. It's a similar class of problems... Can I programatically generate aggregated historical data before my algorithm runs?

I know I can use fetcher to get this data from a csv file.... But can I use quandl or any other service to generate a csv file that quantopian can read in that will give me the all-time highest closing price of a security? Or do I have to do this offline on my own? I just got the research product beta but I haven't played with it enough to know if that's what can do this for me. Maybe that's where I should look?

Thanks!

Hello Brett,

I've attached some code that may solve your problem. The trick here is that 'history' returns trailing daily highs and the high for the current day, up to the current backtest minute.

Grant

(Note that pandas does not need to be imported. I thought I'd end up using it, but didn't need it.)

import pandas as pd

def initialize(context):  
    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund  
                       sid(19656),  # XLF Financial SPDR Fund  
                       sid(19658),  # XLK Technology SPDR Fund  
                       sid(19655),  # XLE Energy SPDR Fund  
                       sid(19661),  # XLV Health Care SPRD Fund  
                       sid(19657),  # XLI Industrial SPDR Fund  
                       sid(19659),  # XLP Consumer Staples SPDR Fund  
                       sid(19654),  # XLB Materials SPDR Fund  
                       sid(19660) ] # XLU Utilities SPRD Fund  
def handle_data(context, data):  
    high_daily = history(2520,'1d','high')  
    peak = high_daily.max(axis=0)  
    peak_XLY = peak[context.stocks[0]]  
    record(peak_XLY = peak_XLY)  

Thanks Grant.

I'm looking to find a way to do almost exactly what Austin mentioned above. Has anyone created or found an example that shows how to precalculate the indicators for multiple stocks for comparison?