Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Stock prices missing using Pipeline in Research environment?
Taylor Jensen
edited Jul 21, 2017
Seeking Help Research Pipeline

Hi All,

I feel like I've found a bug but please let me know if I'm doing something incorrectly to see these issues. I noticed that for more than a few stocks, their closing prices are mysteriously missing for large amounts of time using Pipeline in the Research environment. One such stock I found was GM. Using the Q1500US universe, I only observe a few months in 2016 where GM actually has data. Is this a bug? It's possible for a stock to be in and out of the Q1500US I realize, but a stock as highly traded as GM seems unlikely. I also compared it to Pipeline data using the StaticAssets for GM and I got all stock prices as normal. Also, get_pricing() within Research is functioning correctly. Please see the attached notebook, it's really messing with some of the research I'm currently doing. If there is something obviously wrong with my code here please tell me! Thanks in advance.

7 responses
Taylor Jensen
Jul 20, 2017

Other examples I've found looking between 2014 - 2016:
EROS (nothing from 2014 to Dec. 2015)
LQDT (nothing in 2015)
ARCO (nothing post July 2015)
WTW (Missing April 2015 - Nov 2015)

The list goes on... am I missing something fundamental here? Are other people seeing these results?

Jamie McCorriston
Jul 21, 2017

Hi Taylor,

The Q1500US is a dynamic universe. At the beginning of each month, the universe is updated with some names moving in and some moving out. There are some names that will move in and out of the list, which is what it looks like is happening with these names. If you'd like to learn more, I'd recommend checking out the universe selection lecture which walks through the construction of the Q1500.

I hope this helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Taylor Jensen
Jul 21, 2017

Hi Jamie,

Thanks for the response. I just made an edit to the original post but will also put it here. Is it possible that a large stock like GM moves in and out so frequently? It's one of the highest volume traded stocks on a daily basis so it seems weird that it would be out of the Q1500 for so long. Thanks for the response!

Taylor Jensen
Jul 21, 2017

Reading the article you provided, perhaps even companies like GM are running into this rule:

"At the start of each month, throw out "hard-to-trade" equities, then choose the top 1500 equities by 200-day average dollar volume (ADV), choosing no more than 30% of the requested equities from
any single sector."

Perhaps the sector is hitting a limit and even stocks like GM are being omitted. Thanks again!

Taylor Jensen
Jul 21, 2017

For anyone else reading this thread, just received this message in the form of a support ticket:

"I just checked your notebook, and the issue you initially pointed out happens because GM is missing from the Q1500.

You are correct, GM should be in this universe based on the selection criteria for Q1500 stocks. There is actually an issue with GM's fundamentals data, which we use to construct these universes. Our data team is aware of the issue, and it is currently being investigated as a part of a larger project we are working on to improve our fundamentals data query performance.

Sorry for the inconvenience. Please let me know if you have any more questions, or if there is anything else I can assist you with. "

Jamie McCorriston
Jul 21, 2017

Taylor, that's definitely a good path to follow. In this case, I think there is a data issue with GM. It looks like GM doesn't have a sector assigned to it until mid December 2015. We'll have to investigate and get that fixed.

Sorry for the confusion.

P.S. If you want to filter the pipeline output dataframe down to a specific stock in research, I'd recommend using this: 

result[result.index.get_level_values(1) == symbols('GM')]

That will filter by the second column in the index looking for instances of GM.

Taylor Jensen
Jul 21, 2017

Makes perfect sense, and thanks a lot for the tip!

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.