Hello Troy,

Your understanding of expected behavior sounds right.

The backtester loads up the data for every sid it thinks it needs before it runs the backtest (that's an oversimplification, but good enough for this explanation). Generally that error means that you're trying to access a sid that wasn't in the dataload. The most common way to trigger that error is to do something like order(sid(rand.next()), 100). It would seem that you're not ordering the sid that you think you are, or there is a bug in the backtester - both seem possible.

Check out this simple algo I wrote. It buys the universe once per quarter and then exits everything on 9/22/08. Perhaps you can modify this to repro your case?

If you give me permission, I can log into your account and look at your algo. But we only do that when expressly permitted. You can also email [email protected] if you'd rather debug over email.

I'm also getting an OrderSecurityOutsideUniverse error, and wanted to share the offending code to get some clarification on how set_universe, fetcher, and the data dict interact.

Please see source code at the bottom of post. My understanding of what is going on is as follows - let me know if any of the following is incorrect:
1) If the fetched csv has tickers that represent securities which are not in the Quantopian database, these are never added to data
2) If the fetched csv has tickers that are not in the set_universe, these are still added to data, and thus passed to handle_data
3) Following from (2) above, if you iterate through data in handle_data, you will come across securities that are not in the set_universe. If you order these securities, you will get the OrderSecurityOutsideUniverse error

If all this is true, my follow-up question is how do you iterate through only the set_universe securities? It doesn't appear that universe.DollarVolumeUniverse(a,b) is iterable. Thanks so much for your help!

import pandas as pd  
import statsmodels.api  
from pandas import Series,DataFrame

# Idea source: Dellavigna and Pollet  
# "Investor Inattention and Friday Earnings Announcements."  
# The Journal of Finance. April 2009.

# Code sources: Jessica Stauth, "Trading Earnings Surprises with Estimize Data"  
# and John Fawcett, "Ranking and Trading on Days to Cover"

# TODO  
# 

def clean_spaces(df):  
    df['symbol'] = df['symbol'].map(lambda x: x.strip())  
    return df  
def time_lag(df):  
    df = df.tshift(1,freq='b')  
    return df

def initialize(context):  
    # This file contains Friday earnings announcements and prior consensus estimates from Yahoo! (Reuters data)  
    # from 1-1-2008 to 6-1-2013  
    fetch_csv('https://dl.dropboxusercontent.com/u/99351570/Friday%20Earnings%206-24-13.csv',  
        date_column='date',  
        date_format='%m/%d/%Y',  
        pre_func=clean_spaces,  
        post_func=time_lag)  
    set_universe(universe.DollarVolumeUniverse(80,90))  
    context.other_sids = [sid(8554)]  
    # NOTE: this block doesn't work at populating there universe  
    # There is also a 100 security limit when inidvidualy defining sids?  
    #context.stocks = []  
    #for i in range(10000):  
    #    try:  
    #        context.stocks.append(sid(int(i)))  
    #    except:  
    #        continue  
    #set_commission(commission.PerShare(cost=0.005))  
    #set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.1))  
    # trade_dict keeps track of various attributes of stocks  
    # prior_surprise: prior surprise, as % earnings  
    # surprise_dir: surprise direction, 1=positive, -1=negative  
    # consecutive: # prior surprises in same direction  
    # holding: 1 = currently held in portfolio, 0 = not held  
    # days held: days stock held in portfolio  
    # hedge_shares: number of SPY shares to offset position, tracked for closing out position  
    context.trade_dict = {}  
    context.max_notional = 1000000  
    context.min_notional = -1000000  
    context.max_pos = 0.10  
    context.unit_size = 0.10  
    context.pos_multiplier = 100000  
    context.surprise_threshold = 0.05  
    context.holding_period = 20  
def handle_data(context, data):  
    # update trade_dict with any new securities in universe  
    for stock in data:  
        if stock not in context.trade_dict:  
            context.trade_dict[stock] = {'prior_surprise':0,  
            'surprise_dir':0,'consecutive':0,'holding':0,'days_held':0,'hedge_shares':0}  
        if 'price' in data[stock]:  
            #TO DO  
            hedge_price = data[sid(8554)].price  
            share_unit = (context.max_notional * context.unit_size) / data[stock].price  
            hedge_unit = (share_unit * data[stock].price) / hedge_price  
        else:  
            share_unit = 0  
            hedge_unit = 0  
        if context.trade_dict[stock]['holding'] == 1:  
            context.trade_dict[stock]['days_held'] += 1  
        if context.trade_dict[stock]['days_held'] >= context.holding_period:  
            order(stock, -1 * context.portfolio.positions[stock].amount)  
            order(sid(8554), -1 * context.trade_dict[stock]['hedge_shares'])  
            context.trade_dict[stock]['holding'] = 0  
            context.trade_dict[stock]['days_held'] = 0  
            context.trade_dict[stock]['hedge_shares'] = 0  
        earnings = 0  
        estimate = 0  
        surprise = 0  
        # eps and eps estimates exist for the date  
        if 'eps' and 'estimate' in data[stock]:  
            earnings = float(data[stock]['eps'])  
            estimate = float(data[stock]['estimate'])  
        # avoid dividing by zero  
        if earnings != 0:  
            surprise = (earnings - estimate) / earnings  
        # only update trade_dict if the has been a change in earnings or estimates  
        if context.trade_dict[stock]['prior_surprise'] != surprise:  
            # consecutive positive surprises  
            if context.trade_dict[stock]['prior_surprise'] > 0 and surprise > 0:  
                context.trade_dict[stock]['surprise_dir'] = 1  
                context.trade_dict[stock]['consecutive'] = context.trade_dict[stock]['consecutive'] + 1  
            # consecutive negative surprises  
            elif context.trade_dict[stock]['prior_surprise'] < 0 and surprise < 0:  
                context.trade_dict[stock]['surprise_dir'] = -1  
                context.trade_dict[stock]['consecutive'] = context.trade_dict[stock]['consecutive'] + 1  
            # non-consecutive surprises  
            else:  
            # last surprise was negative, current surprise positive  
                if context.trade_dict[stock]['prior_surprise'] < 0:  
                    context.trade_dict[stock]['surprise_dir'] = 1  
                    context.trade_dict[stock]['consecutive'] = 0  
            # last surprise was positive, current surprise negative  
                elif context.trade_dict[stock]['prior_surprise'] > 0:  
                    context.trade_dict[stock]['surprise_dir'] = -1  
                    context.trade_dict[stock]['consecutive'] = 0  
            # last surprise updated with current surprise  
            context.trade_dict[stock]['prior_surprise'] = surprise  
            log.debug(str(stock))  
            order_management(stock,surprise,context,share_unit,hedge_unit)  
def order_management(stock,surprise,context,share_unit,hedge_unit):  
    if surprise > context.surprise_threshold:  
        order(stock,share_unit)  
        order(sid(8554), -1 * hedge_unit)  
        context.trade_dict[stock]['holding'] = 1  
        context.trade_dict[stock]['hedge_shares'] = -1 * hedge_unit  
    elif surprise < -context.surprise_threshold and context.trade_dict[stock]['holding'] != 1:  
        order(stock,-1 * share_unit)  
        order(sid(8554), hedge_unit)  
        context.trade_dict[stock]['holding'] = 1  
        context.trade_dict[stock]['hedge_shares'] = hedge_unit  

Andrew:

Your conclusion about securities appearing to being added to what is iterated through data is correct.
Though your statement 1) is incorrect, if you have a 'symbol' when fetch_csv has mask=False, which is the default, symbols that are not Quantopian database securities are added to data, e.g. this is how arbitrary signals can be added via fetcher.

A subtle distinction is that symbols may be added to data here where the integer that would be set as stock in something like for stock in data would look like a key into the pricing data that is normally included in data, however the data associated with that key would just be values provided by the CSV found by fetcher.

To fix your issue, I believe the correct approach is to add mask=True to the parameters to your fetch_csv call, so that entries won't be added to data that are not in the current universe.
mask filters both on a) whether the symbol is in our database, b) the symbol is in the current universe. With mask=True the pricing data for members of the universe will have the fetch_csv data added to it.

As far the need for an easy way to access what the current universe is, I agree.

Thanks Eddie, adding mask=True to the fetch_csv parameters did the trick.

If you hit OrderSecurityOutsideUniverse without fetcher, this might help.