When to use data.can

Back to Community

When to use data.can_trade

edited

TLDR
You don't need to use data.can_trade when you are working with today's output of pipeline or get_fundamentals.

The two most common cases where data.can_trade is necessary:

When ordering a security that is manually referenced using sid, symbol, or fetcher.
When ordering a security listed in context.portfolio.positions rather than the output of pipeline or get_fundamentals.

Long Answer
The question of when data.can_trade should be used is a topic of much discussion here at Quantopian. I am going to do my best to explain all of the the details so you can make your own decision with full information.

The first thing to know is that when a security doesn't exist (either because it hasn't IPO'd yet or because it was delisted), Quantopian 2 returns NaN for price, open, high, low and close and 0 for the volume. This is different from Quantopian 1, where prices were forward-filled even after delisting. Quantopian 2 is more precise, and this behavior change drives much of what is below.

I'm going to explain how data.can_trade is determined on IPO dates and delisting dates. Then I will cover what securities are returned from pipeline based on their IPOs and delistings. Finally, I'll cover some of the places where we have seen issues arise.

IPO Date
When a security IPOs, its first trade is rarely at at market open. The exchange gets everything together, prices the security, and sets the first trade price, generally a few hours after the market opens. Before the first trade, the price of the security is NaN and the volume is 0, so data.can_trade returns false. Once the first trade has happened, data.can_trade will return true.

Delisting Date
With security delistings, data.can_trade will return True before and through the delisting date. The delisting occurs at market close on the delisting date. Any day after the delisting, data.can_trade will return false.

The tricky thing here is that your portfolio can hold securities on the delisting date. In order to protect algorithms from accidental delistings (which happens occasional due to bad data from our data providers) we have a 3-day buffer before automatically liquidating delisted securities from your holdings. If a security is held when it delists, it will remain in your portfolio for those 3 days, and then the position will be closed at the last known price. If you try to get the price or volume of the security after delisting, you will get NaN and 0, respectively.

Pipeline
On any given day, pipeline will only give you securities that can be traded on that day. Since your pipeline is generated based on yesterday's data, it will not include any securities that IPOed today, or have delisted. Pipeline will return securities on the delisting date itself, as they can be traded on that date. This means that if pipeline generated your security list today, you can buy and sell those securities without issue. You don't need can_trade.

Other notes
During our internal Quantopian 2 testing, we noticed a problem when users used pipeline to define their portfolio. After a security is delisted, the security is no longer included in the pipeline results. However, it is still possible for the security to be present in the portfolio during the 3-day window between the security delisting and the security being automatically removed from the portfolio. If the algorithm tried to close the position it would run into problems because it couldn't sell the delisted security.

Since many of our order functions rely on having a last known price and volume (which now return NaN and 0 respectively) if those prices don't exist, then the algorithm would blow up with an error. To prevent this exception, orders that are placed for delisted securities during this 3-day window will return None, and the order will not be placed. The position will be automatically closed after the 3 days have passed.

Another way to run into a problem is to use a static list of securities, as it's very easy to order a security before its IPO date or after its delisted date. In these cases, order methods that rely on price or volume will result in an algorithm error. For this reason, you should use data.can_trade whenever working with static lists.

I hope this explanation helps make the various use cases here more clear. Please let us know if you have any other questions.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

12 responses

Grant Kiehne

I don't understand why data.can_trade is necessary when ordering a security listed in context.portfolio.positions (rather than the output of pipeline or get_fundamentals). If a position is held in XYZ, then it must be beyond its first-trade date. And "orders that are placed for delisted securities during this 3-day window will return None, and the order will not be placed." So, it would seem that data.can_trade would be unnecessary for XYZ.

You seem to assume that the IPO date and the first-trade date will be the same, however it is not necessarily true. A stock could be listed but not trade on its IPO date. Presumably, pipeline needs to see the first trade before it will admit a stock, correct? Potentially there could be some confusion over your language, since if I understand correctly, the IPO date (initial listing date) is irrelevant. What you are really talking about is the first-trade date.

Is there any way using pipeline (or otherwise) to trade IPOs? As you say, "order methods that rely on price or volume will result in an algorithm error" but as long as these order methods are avoided, then it should be feasible to place orders prior to the first trade. So, is it possible to write an algo that provides a daily list of stocks that have just gone on the market?

Grant Kiehne

Another way to run into a problem is to use a static list of securities, as it's very easy to order a security before its IPO date or after its delisted date. In these cases, order methods that rely on price or volume will result in an algorithm error.

orders that are placed for delisted securities during this 3-day window will return None, and the order will not be placed.

Aren't these statements inconsistent. In the first case, you say that an algorithm error will result by placing an order after a security's de-listing date. But then in the second case, you say that the order will return None and the order will not be placed (at least for the 3-day window...what happens after the 3-day window?).

I don't understand how order_target_percent will work, after de-listing.

Say I do this:

for stock in stock_list:  
    order_target_percent(stock,weight[stock])

So what happens for stocks that are de-listed? Presumably, order_target_percent will work as normal, using the last trade prices for the de-listed stocks, but not place orders for de-listed stocks. Stocks that are not de-listed will have their positions adjusted, but the leverage could be incorrect (since order_target_percent computes the portfolio adjustments based on the assumption that all stocks can trade).

Grant Kiehne

Hi Karen,

This algo crashes with:


Cannot order LILA_V on 2015-06-30 14:30:00+00:00 as there is no last price for the security. Try using data.can_trade to check whether an asset is delisted.  
There was a runtime error on line 21.

However, it is simply getting a daily list of stocks from get_fundamentals, so if I understand your guidance correctly, it should work. Am I doing something wrong?

def initialize(context):  
    schedule_function(place_orders, date_rules.week_start(days_offset=1), time_rules.market_open(minutes=60))  
def before_trading_start(context,data):  
    fundamental_df = get_fundamentals(  
        query(  
            fundamentals.valuation.market_cap,  
        )  
        .filter(fundamentals.company_reference.primary_exchange_id == 'NAS')  
        .filter(fundamentals.valuation.market_cap != None)  
        .order_by(fundamentals.valuation.market_cap.desc()).limit(50))  
    context.stocks = [stock for stock in fundamental_df]

def place_orders(context,data):  
    pct = 1.0/len(context.stocks)  
    for stock in context.stocks:  
        order_target_percent(stock,pct)

Grant Kiehne

The attached code suggests that get_fundamentals might admit securities on their IPO dates (prior to their first-trade dates).

Grant Kiehne

This code crashes with:

Something went wrong. Sorry for the inconvenience. Try using the built-in debugger to analyze your code. If you would like help, send us an email.  
Cannot order COVR on 2011-01-05 15:30:00+00:00 as there is no last price for the security. Try using data.can_trade to check whether an asset is delisted.  
There was a runtime error on line 33.

It seems like it should work, since I am obtaining the list of stocks from pipeline. Is there a mistake in my code?

from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline import Pipeline  
from quantopian.pipeline.data.builtin import USEquityPricing  
from quantopian.pipeline.factors import CustomFactor  
import numpy as np

class AllStocks(CustomFactor):  
    inputs = [USEquityPricing.close]  
    window_length = 1  
    def compute(self, today, assets, out, close):  
        out[:] = None  
def make_pipeline():  
    pipe = Pipeline()  
    everything = AllStocks()  
    pipe.add(everything, "everything")  
    return pipe

def initialize(context):  
    attach_pipeline(make_pipeline(), 'all')  
    schedule_function(place_orders,  
                      date_rules.every_day(),  
                      time_rules.market_open(hours=1))  
def before_trading_start(context,data):  
    context.output = pipeline_output('all')  
    context.security_list = context.output.index  
    context.security_list = [context.security_list[i] for i in np.random.random_integers(0,len(context.security_list)-1,50)]

def place_orders(context,data):  
    pct = 1.0/len(context.security_list)  
    for stock in context.security_list:  
        order_target_percent(stock,pct)

Grant Kiehne

Here's the same code, except with a fixed random number generator seed, so that the error is repeatable. In this case, I consistently get:

Something went wrong. Sorry for the inconvenience. Try using the built-in debugger to analyze your code. If you would like help, send us an email.  
Cannot order GFNC_Z on 2011-01-05 15:30:00+00:00 as there is no last price for the security. Try using data.can_trade to check whether an asset is delisted.  
There was a runtime error on line 35.

Lyell Griswold

Hi Karen and Quantopian Devs,
For the instances of data.can_trade behavior, are we only talking about Backtesting. Or will the 3 day rule for a delisted stock and automatically closing positions also happen during a Live Trading situation like in Robinhood? Or will we still need logic to capture and close a delisted position?
These situations above do not make clear the use case of either Backtesting or reality/realtime trading.

Thanks,

Jamie McCorriston

Hi Lyell,

In live trading, positions are read in from the broker. Similarlly, held positions of delisted securities are handled by the broker. The 3-day rule for delisted stocks being liquidated for cash is only for backtesting. Sorry for the confusion!

Disclaimer

Jamie McCorriston

Lyell, I need to revise my previous statement. The 3-day rule also applies to Quantopian paper trading (Zipline) algorithms. It's used any time we need to simulate the liquidation (anything not tied to a broker). Sorry for the confusion.

Disclaimer

Suminda Dharmasena

In practice since you cannot trade after delisting maybe a delist_pending function might be also good to have so you can close out positions and stop trading.

Dan Dunn

Suminda, that would be a huge source of lookahead bias. If we could all know who would be delisted in advance, we'd all make a lot of money!

Disclaimer

Grant Kiehne

Hi Dan,

I got the impression that there are lists of pending suspensions, but Q just doesn't have the data. For example, see:

https://listingcenter.nasdaq.com/IssuersPendingSuspensionDelisting.aspx
https://www.nyse.com/regulation/delistings

Perhaps this is what Suminda is talking about?

You've successfully submitted a support ticket.

Our support team will be in touch soon.