Karen, good to hear. I want clarification on how this relates to live trading. Does it mean that we can remove special handling of stocks soon to be delisted from our algorithms, and use that code without any further change in paper/live trading?

Thanks,

Sunil

Hi Sunil,
There is no change to how securities are handled when they are delisted for IB paper and real money trading.

However for Quantopian (or Zipline) paper trading, these new rules will be followed for delisted securities without any additional changes to your code.

Thanks! Good news.

What happens to open orders after the end date? And what if an order is placed after the end date? Will it be automatically rejected?

[EDIT:] If a stock is held after the end date, it'll still be in data, right? So, what happens if an order is placed for the stock within the three days after the end date, when the stock will still be in data, waiting to be sold automatically? Will the order end up open forever?

Grant,

You are correct that a stock will still be in data after the end date; it is not removed from data until the cash out date. If an order is placed within the three days between the end date and the cash out date, an open order will be created but never filled. The order will not end up open forever though; it will persist only until the cash out date, at which point it will be canceled.

Thanks David,

So, I gather that not only have you changed the way delisted securities are handled, but also their open orders. Excellent!

Grant

Was this de-listing problem an open issue in zipline that was closed by this fix?

Yes, here's the zipline issue: https://github.com/quantopian/zipline/issues/188.

Great news. A lot of algorithms become possible now in the contest because no longer are we overleveraging from dead stocks.

Q, this is amazing! I have always wanted to start using the platform to build a distressed asset strategy. We are starting to see a pick up in CH11 filings,so it seems like the time is ripe. I think the road map for a strategy buying/selling distressed assets would be the following:

1. Buy / Sell asset prior to de-listing within the Q backtester.
   
2. Manually adjust impact of price changes occurring after de-listing and outside of Q exchanges via fetcher (i.e. throw pink sheet prices on a spreadsheet and use it to calculate adjustments to the portfolio)
   

3. Loop any Q exchange listed stock received post CH11 re-organization back into the algo & calculate any exotic securities received on the csv file.

   Taking this one step at a time, I would love some help with the first issue I encountered. I initialized an algo using my favorite distressed asset play of all time (Dynegy, Inc). As you can see from the error code below, I think I am limited to either pre CH11 Dynegy or post CH11 Dynegy. I am wondering if there is a way I can get the backtester to buy/sell both securities? From there I can worry about adding in the performance of the stock while it was trading in the pink sheets, and the performance of the warrants that were received by shareholders after reorganization. Any suggestions would be much appreciated?

   (one last thing...if PACER every develops an API for their data, that would be an awesome feature add)

13 Error There are multiple assets that have used the symbol DYN. You must use set_symbol_lookup_date() before using symbol(). The table below shows when each company traded using the symbol DYN. Learn More
ASSET FIRST TRADED LAST TRADED
DYNEGY INC/NEW 2012-10-02 2016-04-24
DYNEGY INC(HOLDING CO)CL A 1993-11-09 2012-07-08

2 questions please:

1) Is there any additional guard I can use in back tests for long (3+months) trading halts (without lookahead bias) ? My algo is picking up a ticker (FU) that NYSE halted on Nov 22/13 and that was delisted Aug 11/12 2014 essentially causing me to carry it for 8.5 months in portfolio using up my port weight %

http://ir.theice.com/press/press-releases/nyse-regulation/2014/nyse-mkt-to-suspend-trading-immediately

stock
Equity: Equity(31271, symbol=u'FU', asset_name=u'FAB UNIVERSAL CORP', exchange=u'AMERICAN STOCK EXCHANGE', start_date=Timestamp('2008-02-07 00:00:00+0000', tz='UTC'), end_date=Timestamp('2014-08-12 00:00:00+0000', tz='UTC'), first_traded=None, auto_close_date=Timestamp('2014-08-15 00:00:00+0000', tz='UTC')

2) with v2 closing unfulfilled order at end of day, I added logic to re-order unfulflled amount of previous orders - is there supposed to be some sort of guard to prevent allowing placing an order of a delisted stock after its end date?

Log output
2013-12-19 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.
2013-12-20 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.
2013-12-23 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.
2013-12-24 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.
. . . 2015-06-16 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.
2015-06-17 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.
2015-06-18 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.
2015-06-19 WARN Your order for -1958 shares of FU failed to fill by the end of day and was canceled.

Umar,

Regarding you #2 question, on https://www.quantopian.com/posts/when-to-use-data-dot-can-trade, Karen stated:

Since many of our order functions rely on having a last known price and volume (which now return NaN and 0 respectively) if those prices don't exist, then the algorithm would blow up with an error. To prevent this exception, orders that are placed for delisted securities during this 3-day window will return None, and the order will not be placed. The position will be automatically closed after the 3 days have passed.

I haven't tested it yet, but if it works, then orders would not be placed during the 3-day period. I'm not sure what happens after the 3-day period, when the security would obviously still be delisted. Maybe orders continue to be ignored?

Thanks Grant - yeah after the 3 day period appears as though the order request passes through (not fulfilled) and then is cancelled

O.K. Then I guess the rule is that after the security end date (delisting date), new orders are cancelled automatically. I wonder if this is true for order_target_percent, as well?

I have an issue with this that perhaps someone can help with. My algo shorted PVAC prior to it filing for CH11. The company was delisted from the NYSE (last price 0.15), went OTC, did CH11 and re-emerged under the ownership of the debtors, continued OTC then on Dec 30 2016 it pops up at $49 on NASDAQ and destroys my portfolio. Shouldn't the backtesting have taken me out when it was dropped from the NYSE? What am I missing?

Based on some test in research mode, delisted stocks are not yet handled. Am I right or missing something ?

@Ali: Delisted stocks should be handled, but I'm not sure how you would be able to detect that in research. You can always look up pricing data for stocks, even if they've been delisted. The backtester, however, should close out any positions held in a stock that goes delisted.

@Scott: You have found an exception to the rule I described to Ali. Right now, there's a limitation where a stock can only have one start_date and one end_date in our system. This means that if a stock goes OTC, and then comes back, the end_date of that stock gets updated to be the current date (assuming PVAC is still trading). The rule we have in place for handling delisted stocks depend on the end_date. Using the delisting logic, the code is technically still trading, and doesn't know about the time when it was trading OTC, since the end_date was updated to today. This is an issue that we are aware of. We don't yet have a good solution for it but it's on our list of issues to fix. The best I can suggest now is to blacklist PVAC from your portfolio. I know this means introducing lookahead bias, but I don't have a better workaround for you at this time. Sorry for the inconvenience.

Jamie, the issue with PVAC and other stocks that are shorted show up in an algorithm that I have been working on since I found Quantopian. If they go OTC instead of NASDAQ or other listed exchanges couldn't there be a filter for no OTC traded securities?

Hi Todd,

Yes, you certainly could. I don't know how quickly that field would be updated with Morningstar data for this specific security but it would certainly be worth investigating.

Message edited, code removed. The idea was to avoid opening positions that are about to be delisted since auto close can sometimes result in phantom profits (because the auto close 3 days later presumes all shares would have traded). Per JM message below though, in live trading, end_date is always today. The end_date field is only there for the engine behind the scenes. It has nothing to do with market advance notices of known impending delistings like https://listingcenter.nasdaq.com/IssuersPendingSuspensionDelisting.aspx or https://www.nyse.com/regulation/delistings

@Blue, it's important to note that the delisting function you defined above uses forward lookahead bias to close positions. It uses information that it would not know in live trading to close positions. In fact, in live trading, the end_date of a stock is always the current date if it's still actively trading. The delisting function above will automatically return true every day that you hold a position in a stock that's still trading. I strongly discourage you from using it as it will add unrealistic behavior to the algo that does not translate to live trading.

Instead, the backtester automatically closes out delisted positions for cash when they are delisted. The cash-out happens 3 trading days after the delisting.

Thanks for pointing that out. The idea was to match reality better, I had run a backtest with fantastic results, they were not faux due to margin. I didn't trust it, tried adding delisting() and the results were then a lot lower, not higher. I was glad about it, as the unrealistic behavior in that case seems to have been the reverse, when auto close happened, profits appeared to be reaped that would not have happened in the real world. So it is a tough bind that can cut both ways.

We can benefit by being aware of delistings and the code below is a decent option, it merely logs when auto close occurs. In initialize(), set_auto_close_notice() may be worth considering. Seeing what's happening when, individuals might notice patterns they can use to diminish their likelihood of investing in one or more stocks that will wind up delisted.

Logs when an auto close has happened with amount, a ballpark price and value ...

def initialize(context):  
    context.auto_close = {}

def before_trading_start(context, data):  
    c = context  
    for s in c.auto_close:           # Log auto close value etc when it happened  
        if s not in c.portfolio.positions:  
            value = c.auto_close[s]['price'] * c.auto_close[s]['amount']  
            log.info('{}  prc {}  amt {}  value {}  end {}  auto close {}'.format(  
            s.symbol, c.auto_close[s]['price'], c.auto_close[s]['amount'],  
            value, s.end_date.date(), s.auto_close_date.date()))  
    c.auto_close = {}  
    for s in c.portfolio.positions:  # Store info when auto close approaching  
        if (s.auto_close_date - get_datetime()).days > 5: continue  
        price = 0   # somersaults to try to capture a recent valid price  
        prc = data.current(s, 'price')  
        if prc and prc == prc:  # avoid 0 and trick to avoid nan  
            price = prc  
        else:  
            prc = c.portfolio.positions[s].last_sale_price  
            if prc and prc == prc: price = prc  
            else:  
                prc = data.history(s, 'price', 1, '1d')[-1]  
                if prc and prc == prc:  
                    price = prc  
        # keep previously stored price if 0 this time  
        if price == 0 and s in c.auto_close: price = c.auto_close[s]['price']  
        c.auto_close[s] = {  
          'price' : price,  
          'amount': c.portfolio.positions[s].amount,  
        }