Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Issues with fetcher, data.can_trade. and failed order fills
Steven Denison
posted Dec 13, 2017
Seeking Help Getting Started Fetcher Using Data

So I'm working on a simple annual rebalance strategy and have been running into odd issues with fetch_csv, data.can_trade and orders failing to fill.

As it relates to fetch_csv, I've encountered an issue where for a few random names, instead of fetching the correct value, it simply returns nan. I'll include a link to my CSV below, but as you will see, as of 4/30/14, the algorithm should be able to see all the values in the CSV, and for A, should return 0.03303348 when called through data.current(sid(24757),'ROA'). However, instead it simply returns nan. Has anyone encountered an issue like this before, or have any idea what might be causing this?

https://docs.google.com/spreadsheets/d/14OirozvJ33x2ygCvPaJ6KnsGjF7JL3CWJic75fChbxs/edit?usp=sharing

As it relates to data.can_trade, I was under the impression that the method should return True as long as the security is alive and has traded at least once before. However, I've also seen instances where one day it will return True, the next day False, and the following day True again. Any idea what would cause this?

The final issue I have encountered is with orders failing to fill by the end of the day. Maybe I'm missing something obvious here, but if a security passes the can_trade test, and I place a market order in a zero slippage environment, is there any reason why the order would fail to fill?

Apologies for the slew of questions, I've just been running into these random odd issues which are making what I feel like should be a very simple algorithm into a continuing headache. I really appreciate any input and help! Thanks!

8 responses
Dan Whitnable
Dec 13, 2017

Can't help with the CSV issues but can add one piece of insight to orders not filling.

If one uses the 'VolumeShareSlippage' model then only a percentage of the minute volume is filled each minute. This of course could lead to unfilled orders especially if trades are made late in the day (not enough minutes left to fill). If one uses the 'FixedSlippage' model then an order will fill entirely in the first non-zero-volume bar. So typically orders will always be filled using this model. However, note that it won't fill until a non-zero-volume bar. A lightly traded security ordered late in the day could very well have a lot of zero volume bars and therefore the may not fill even with this model.

Dan Dunn
Dec 13, 2017

On the other issues, can you post a shared backtest or notebook with the problem? Or contact support with the relevant algorithms? I don't recognize those problems, and examples will help us see the problem.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Steven Denison
Dec 13, 2017

So this backtest should highlight the issues I'm seeing with nan's.

Ernesto Perez
Dec 14, 2017

Hello @Steven,

The csv file imported by your backtest above is missing ROA data for AAPL, that's why you get NaN values when querying that field.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Steven Denison
Dec 15, 2017

Hi Ernesto,

I've just double-checked the link I posted and I still see 10 in the ROA column for AAPL. Does that not appear when you look at the CSV file through that link?

Ernesto Perez
Dec 20, 2017

The file linked in your original post does have a value fo 10 for AAPL, but I am checking the link in the algorithm (the one fetch_csv is getting the data from). That file is missing data for AAPL on 4/30/14.

Steven Denison
Dec 20, 2017

I don't mean to sound like an idiot here so I do apologize if I'm just missing something obvious but if I copy and paste the URL referenced in fetch_csv in the source code of the above backtest (the URL being https://docs.google.com/spreadsheets/d/14OirozvJ33x2ygCvPaJ6KnsGjF7JL3CWJic75fChbxs/), I still see a value of 10 for AAPL on 4/30/14. That URL also matches the file linked in my original post.

Where are you getting the "link in the algorithm" from?

Ernesto Perez
Dec 20, 2017

Hi Steven,

I am referring to the algorithm/backtest you shared above. If you use the full link including the output type set to csv,

https://docs.google.com/spreadsheets/d/14OirozvJ33x2ygCvPaJ6KnsGjF7JL3CWJic75fChbxs/gviz/tq?tqx=out:csv

... the resulting file does not have data for AAPL. This is the file fetch_csv is ingesting.

The shared version of your Google Sheet file (linked both in your original post and directly above) does contain data for AAPL, so something must have gone wrong when you generated the url using "Publish to the web...". Have you tried generating the url again?

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.