You 'll have to read the detail of the corporate action. In some case shares of the acquired company may continue to trade, in some you may get new shares.....

I see. There is a similar problem with delisted stocks or anytime a stock stops trading. I saw a code snippet that excluded stocks that end before the end date of the backtest but that doesn't work if you are using set_universe. I suppose I can call set_universe in initialize, then exclude all the stocks that end before the backtest ends. I'll only be missing out on new additions. That will probably still work for me.

Brett,

You're right. Usually when a stock gets delisted it remains in your portfolio (E.g. in the case of Gilette) even though you can no longer buy or sell that stock after it's end date. Unfortunately, as Lionel pointed out, you'll have to look up the acquisition details in order to find out what actually happened but if you wanted to go the route of simulating a cash acquisition one thing you could do is to sell all the shares a few days before it's end date:

def initialize(context):  
    #: Look up Gillette which expired in 2005  
    set_symbol_lookup_date('2003-01-01')  
    context.stock = symbol('G')

def handle_data(context, data):  
    #: Only order shares if it's greater than 2 days before expiration date  
    if (context.stock.end_date - get_datetime()).days > 2:  
        log.info("ordering")  
        order_target_percent(context.stock, 1)  
    #: Sell all shares just before expiration date to simulate a cash buyout  
    else:  
        log.info("selling")  
        order_target_percent(context.stock, 0)  

And I wasn't sure if you needed help with removing all the stocks that end before the backtest ends but here's a code snippet just in case you needed it:

from datetime import datetime  
from pytz import timezone

def initialize(context):  
    set_universe(universe.DollarVolumeUniverse(99.0, 100.0))  
    #: Set the timezone here and identify the custom end date you'd want to be using  
    #: datetime(year, month, day)  
    tz = timezone("US/Eastern")  
    context.custom_end_date = tz.localize(datetime(2007, 10, 26, 0, 0, 0))  
def handle_data(context, data):  
    #: Remove all stocks that aren't valid  
    to_remove = []  
    #: Add all stocks to be removed into a list  
    for stock in data:  
        if stock.end_date < context.custom_end_date:  
            log.info("Removing %s with end_date %s" % (stock, stock.end_date))  
            to_remove.append(stock)  
    #: Go through list and delete stocks  
    for i in to_remove:  
        del data[i]  
    """  
        Perform all valid operations here after the stocks have been removed  
    """  
    #: Do order operations here  

Seong

Thanks Seong. I came up with something similar for keeping stocks that stop trading out of my positions but I do plan to try the "sell before it stops trading" approach also.

Along similar lines, I just run into the case of CEPH which got acquired in October 2011. A generic way to deal with this, that avoids having to deal individually would be to sell the disappeared stock for the last known price. So money will be conserved and there will be no stale positions. I run into these problem with a larger universe (volumetraded), but the algorithm I attach demonstrates the problem. Can anyone modify this so that when it finds that there are open orders for CEPH, it cancels them and furthermore eliminates any accumulated position in favor of cash?

Thank you

Hi Georgios,

One way to do this is to check every day at the start of your trading, remove any securities that have a end date that is more than 30 days from today like so:

from datetime import timedelta

def handle_data(context, data):  
    for stock in data:  
        if stock.end_date <= (get_datetime() + timedelta(days=30)):  
            order_target_percent(stock, 0)  

And if you're checking to remove any stocks that you want to not order any more, you can build off the code snippet above by adding the stocks to a variable like context.to_not_order = [].

Then you can have a code snippet like:

for sec in context.to_not_order:  
        orders = get_open_orders(sec)  
            if orders:  
                for oo in orders:  
                    cancel(oo)  

Try implementing those in your algorithm and let me know how it goes.

A note on managing expired securities; if you use this code:

if stock.end_date <= (get_datetime() + timedelta(days=30)):  

You will effectively remove all of your positions 30 days from the end of your test as all securities have at least an end_date of the end of the test.

In addition use a reference security (SPY) that you know doesn't expire in concert with the rolling get_datetime() to managed your expired list (Thanks to Grant K. for this):

    context.Expired = []  
    schedule_function(ExpiredStop)

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
def ExpiredStop(context, data):  
    for stock in data:  
        if (stock in context.Expired):  
            continue  
        if (stock.end_date >= context.REF.end_date):  
            continue  
        if (stock.end_date > (get_datetime() + datetime.timedelta(days=5))):  
            continue  
        order_target_percent(stock, 0)  
        print("Expired {0}".format(stock.symbol))  
        context.Expired.append(stock)  

And of course when you go to your entries, check to see if the target security is in the Expired list first.

    # Rebalance all stocks to target portions  
    for stock in data:  
        if (stock in context.Expired):  
            continue  
        order_target_percent(stock, context.Portions[stock])  

Is this being treated like a bug by Quantopian? Adding custom python to everyone's algorithms to handle the corporate actions that Quantopian doesn't seems like a gross hack.

I have to agree with Simon here. Have there been any developments on this front or do we still need to manage this individually?

All of the discussion appears to be in regards to back testing. What do you do for your real money algo? What happens if you buy a cusip say tomorrow and the cusip gets delisted in one week, but you did not sell prior to the delisting. What will happen during the contest? Will the value drop to zero?

Ujae, to answer your questions:

What do you do for your real money algo?

This will depend how the exchange and broker handle the delisting. In live trading with the broker, we query your IB account every minute to check the latest cash and portfolio values. If there are any differences (for example if you bought a stock manually in TWS/WebTrader outside of your algo), we accept IB's values as truth. So if the broker says you own 0 shares of the stock ABC, then this will be reflected in your algo. Alternatively, if the broker says you own 10 shares of ABC at $0.00, then you will have the position in your portfolio.

What will happen during the contest?

The behavior will be identical to the backtest - the position will stay in your portfolio.

To complement Market Tech post, I post some more filterings that I am doing. Please note: I did not invent those but found those in several community posts and I thought it would be good to put it all together:

import zipline

def initialize(context):  
         context.formation_days = 200  
         context.spy = symbol('SPY')

class DataStock(zipline.protocol.SIDData):  
    def __init__(this, sid, dataSid):  
        this.Update(dataSid)

    def Update(this, dataSid):  
        this.Prices = dataSid.price  
        this.Open   = dataSid.open_price  
        this.High   = dataSid.high  
        this.Low    = dataSid.low  
        this.Close  = dataSid.close_price  
        this.Volume = dataSid.volume

def expired_stocks(context, data):  
    close_prices = history(context.formation_days, '1d', 'close_price').dropna(axis=1)  
    prices = history(context.formation_days, '1d', 'price').dropna(axis=1)  
    open_prices = history(context.formation_days, '1d', 'open_price').dropna(axis=1)

    context.S = {}  
    for stock in data:

        if (stock in context.expired):  
            continue  
        elif (stock.end_date < context.spy.end_date):  
            context.expired.append(stock)  
            log.info("Filtered because date less then spy date: %s" % (stock.symbol))  
        elif (stock.end_date <= (get_datetime() + datetime.timedelta(days=5))):  
            context.expired.append(stock)  
            log.info("Filtered because date will expire: %s" % (stock.symbol))  
        elif stock in security_lists.leveraged_etf_list:  
            context.expired.append(stock)  
            log.info("Filtered because in ETF: %s" % (stock.symbol))  
        elif (stock not in prices or stock not in open_prices or stock not in close_prices):  
            context.expired.append(stock)  
            log.info("Filtered because in no prices: %s" % (stock.symbol))  
        else:  
            context.S[stock] = DataStock(stock, data[stock])

            std = prices[stock].std()  
            if abs(prices[stock][-1] - prices[stock][-2]) <= 10 * std:

                context.S[stock].Prices = prices[stock]  
            else:  
                context.expired.append(stock)

            std = close_prices[stock].std()  
            if abs(close_prices[stock][-1] - close_prices[stock][-2]) <= 10 * std:  
                context.S[stock].Close = close_prices[stock]  
            else:  
                context.expired.append(stock)

            std = open_prices[stock].std()  
            if abs(open_prices[stock][-1] - open_prices[stock][-2]) <= 10 * std:  
                context.S[stock].Open = open_prices[stock]  
            else:  
                context.expired.append(stock)  

This should be called after fundemental data collection and before trading.

And because everything is stored in context.S, one can use it later for different calculations (reducing "history" calles).

Yes I know, you could also "append" more intelligently.

FWIW, we've been discussing this issue in Zipline for a little while here (thanks to Thomas for pointing out this conversation!). This really needs to be handled on the back end because of how Zipline deals with securities that no longer have prices.

We've made an update to how the backtester handles delisted securities and removes them from your portfolio. You can learn more about it here.