Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
First trading date

When running backtests for my and other people's algorithms, runtime exception is often raised when trading data is not available. vwap() function is likely to cause an error before say 2003 for BHP ADR. Is there a way to check if the data is available before using the function? Something like

if data[x].first_trade_date < today:  
    continue  
8 responses

That seems like a bug. I would expect it to return None is there is insufficient data - not throw an exception.

In general, Quantopian will auto-adjust your algo start and end times if there is no data for a stock, like from an IPO or a bankruptcy. That auto-adjustment can be overridden, and there are times where the auto-adjust doesn't solve the problem. Stocks stop trading for all sorts of weird reasons, and when you use set_universe, it's a magnet for finding the oddball stock situations!

One tool for your toolbox:

if sid(x) in data:  
    mavg = data[sid(x)].mavg(3)  

That doesn't solve all problems, either, but it keeps things running when you hit a problem.

Sometimes you hit specific stocks that are crazy.

if stock == sid(5149):  
    continue  

That particular code gets you past MXIM "From October 2007 to October 2008, Maxim's common stock was delisted from the Nasdaq Stock Exchange due to the company's inability to file financial statements related to stock option backdating. Maxim's stock was traded over-the-counter and quoted on the Pink Sheets until the company completed its restatement in 2008."

Papering over the problem like I just did seems like a bad idea. It's an outlier that represents real risk - that one of your stocks suddenly stops trading. We shouldn't ignore it, but it's not clear how we should handle it, either. I'm interested in what people think should happen in this case. A problem we're trying to solve is, "what should this asset be priced at?" We don't know the price. I think the next-best idea is to keep the price info from the last day traded and essentially fill-forward from that data. It's not the RIGHT price but it's one that can be used.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Give us the option to the fill forward or replace with None or 0 (For bankruptcies). Fill forward is the best as many strategies use returns in which case there is no abnormal returns when delisted.

Only get prices from venues you plan to support trading in.

IMHO, if the stock is not traded at time t, then it should not be in the data for that time frame. In the example you give. Maxim was traded on Nasdaq for a while, and for those trades there should be data available, I believe there is. For the times the stock was not traded on the list of supported exchanges, presumably Nasdaq and NYSE there should be no data. Yes, the stock was traded elsewhere, but does that mean we should include Enron laminated stock certificates from Ebay? The issue is what does traded mean. I believe it should mean traded on exchanges covered.

I agree with Vlatko.

@Suminda, your version sounds too manipulated. I want my data to be more accurate than that even if it's a bit messy. You can always use Dan's suggestion above to skip missing values.

Dennis, Vlatko - I feel like I'm missing something. "accurate even if messy" makes a lot of sense to me. But that's not compatible with "not be in the data" as I understand it. If I'm long on MXIM, and then MXIM is delisted, I can't just make MXIM disappear from the data.

MXIM was traded, but not on an exchange that we track (the Pink Sheets). So, what should we do? If you have a stock with a non-zero but unknown value, how we handle price and mavg() and other price-derived values?

Dan, I guess it depends on what we're talking about.

Vlatko's original compaint was about .vwap() throwing an error because data was missing (presumably in the period that vwap was looking at). So checking to see if the security is trading today doesn't really help.

I think what I'd like to avoid is just making up numbers by forward filling as Suminda suggests. That's what I mean by "accurate even if messy".

But the original point stands: vwap() shouldn't throw an error if the security had trading halted during the look-back period. It should return None because the value of the function is mathematically undefined.

OK, I can follow that. It's still likely to throw an error, tho, because most algos (so far) don't handle a vwap of None very well. But the distinction is good.