Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Question on using history() to calculate percentage change

Hi! Fairly new to Quantopian. I looked here but I couldn't find a similar error, so hopefully this is not a repeated thread. On my initialize function, I wrote this to calculate a percentage change. Just like in the API docs:

 price_history=history(bar_count=30,frequency='1d', field='close_price')  
  context.pct_change = price_history.iloc[[0, -1]].pct_change()  

Then, on my before_trading_start function I'm trying to use it to calculate the percentage change on the stocks of a list of securities I've defined in initialize(). Like this:

 for stock in context.stock_list:  
        percent=context.pct_change[stock]

And when I run the algo, I get the following exception:

49 Error Runtime exception: KeyError: Security(21651, symbol='IYM', security_name='ISHARES U.S. BASIC MATERIALS E', exchange='NYSE ARCA EXCHANGE', start_date=Timestamp('2000-06-16 00:00:00+0000', tz='UTC'), end_date=Timestamp('2015-06-17 00:00:00+0000', tz='UTC'), first_traded=None)

Which really doesn't help me a lot in trying to figure out what I did wrong.

Any ideas? :)

7 responses

You said you're calling history() in your initialize() function, but that would usually generate a TypeError rather than a KeyError. If you post a stripped-down backtest with the code that generates the error, it would be easier to diagnose the problem.

In Python, and in object oriented programming in general, we view objects as having state and behavior. In this case the object is a Pandas Series and the behaviour is the ability to calculate pct_change on itself. Here are the Docs for .pct_change() What you were trying to do was pass the object as a parameter for a method designed for itself if that makes any sense. For example, to get the pct_change of a single security from history you can use...

percent_changes = histories[security].pct_change()  

where histories is a data frame object containing the historical information and security is a Quantopian Security object as the key.

Does that help a little?

Also as @Michael said, using the history() function in initialize() will give you a TypeError.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

You were both correct. Calling it on initialize () was a mistake. I changed the code the way @James suggested and now it works. The explanation was spot on. Thanks guys!

I am getting the same error, but I am not running history in initialize, and I am calculating percent change manually.

KeyError: Security(48073, symbol='CRC', security_name='CALIFORNIA RESOURCES CORP', exchange='NEW YORK STOCK EXCHANGE',
start_date=Timestamp('2014-11-13 00:00:00+0000', tz='UTC'),
end_date=Timestamp('2015-06-23 00:00:00+0000', tz='UTC'),
first_traded=None) There was a runtime error on line 230.

def initialize(context):

    # Select a group of stocks based on the (price * average_volume).  
    set_universe(universe.DollarVolumeUniverse(  
            floor_percentile=90.0, ceiling_percentile=94.9)

    # Submit new trades at market close  
    schedule_function(HandleEntry, date_rules.every_day(),  
                          time_rules.market_close(minutes=10))

def HandleEntry(context, data):

    # Make a list of all available stocks with data  
    eligible_stocks = [stock for stock in data if 'price' and 'volume' in data[stock]]

    # Get the last X days of prices for every stock in our universe.  
    prices = history(50, '1d', 'price').dropna(axis=1)

    # Calculate the 1-day return for all stocks remove any NaNs if there are missing prices  
    returns = ((prices.iloc[-1] - prices.iloc[-2]) / prices.iloc[-2]) * 100  
    returns = returns.dropna()

    MinPctChangeLong = 3.0  
    MinPctChangeShort = -3.0

    # Remove stocks below minimum change requirements  
    for stock in eligible_stocks:  
        if MinPctChangeShort < returns[stock] < MinPctChangeLong:  # <---- This is the line with the error  
            eligible_stocks.remove(stock)  

Any ideas what is happening?

Well if the stock is in eligible_stocks and got dropped from returns because it had a NaN value that would be one possible reason.

James - Good idea! Do you think this will solve that condition? (Updated with new code)

    # Calculate the 1-day return for all stocks remove any NaNs if there are missing prices  
    returns = ((prices.iloc[-1] - prices.iloc[-2]) / prices.iloc[-2]) * 100  
    returns = returns.dropna()

    # Only include stocks that have returns  
    eligible_stocks = [stock for stock in eligible_stocks if stock in returns]  

Honestly Tristan I would get rid of the eligible_stocks variable altogether. Basically you spend a line of code at the top iterating through all the securities and filtering them, and then you iterate over most of them again later to execute your order logic. I would do it with just one iteration like so:

for security in data:  
    if security in returns:  
        # execute logic  

There maybe a more Pythonic way to check the two collections with set / intersect... Hope that works!