Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
DEPRECATION WARNING: Morningstar is removing the 'operating_income' field on March 31, 2018.
Costantino
posted

Today my algo got this warning:

DEPRECATION WARNING Line 323:  Morningstar is removing the 'operating_income' field on March 31, 2018.

Morningstar recommends using a similar field, 'pretax_income' in its place.  
We are investigating the similarity of these two fields to determine the impact of the change.

In the meantime, we recommend that you research these two fields yourself based on your specific needs.

After March 31, the 'operating_income' field will stop receiving updates.

I tested 'pretax_income' as suggested in the warning but the data are quite different.
Then I tried Fundamentals.gross_profit - Fundamentals.operating_expense: better but still a lot of differences.
My last and best way to replace 'operating_income' was Fundamentals.operation_margin * Fundamentals.total_revenue.

The question is: Will be operation_margin also deprecated?
Actually there is no warning, but I cannot see the rationale to remove 'operating_income' and then leave 'operation_margin'.

@Quantopian: Any result about the impact of the change from your "investigation"? I cannot understand, why an important field as 'operating_income' will be removed.

13 responses
Alisa Deychman

Hi Costantino,

This is part of a larger update that Morningstar is pushing, take a look at this thread for more details. There were a few different category of changes that they are enacting. We have been busy working to support this, to aim for minimum impact on our community, especially to the entries in competing in the 6-month contest and the new daily contest. Unfortunately, these decisions were made by the vendor and are outside of our control -- we did not choose to remove this set of fields.

Rather, we are exploring ways to minimize the effects. Yesterday we shipped deprecation warnings in the IDE so authors can know the changes are coming. Only the fields affected have the associated deprecation warning and suggestions from Morningstar about where to find similar data. From what I've seen -- I agree with you, the fields from their recommendations are not always a good fit. The fields sometimes align and sometimes are different by orders of magnitude. Instead, we are passing all the information we have over to you to make the best decision for your particular strategy.

I'm sorry for the troubles, we're in the same boat as you and are not excited by the update.

Best,
Alisa

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Doug Baldwin

@Alisa

"operating_income" is NOT on your list of factors to be removed. What this means is that either: A) the list at this thread is inaccurate, or B) this Depreciation Warning for "operating_income" is invalid, and needs to be corrected.

Doug Baldwin

Will van Es

Constantino,

It might be worth trying EBIT instead of pre-tax profit as in theory EBIT should be closer to Operating Profit.

Will

Costantino

@Will Thanks for the suggestion. The problem with EBIT is that "non-recurring" earnings are also included

@Doug That's right! operating_income (and also opersting_margin) isn't on the list. How could it be that the "absolute" measure is deprecated but the margin not? Is the warning really correct?

Costantino

I've just seen that operating income a margin are in the second list (calculation adjustment)... I'm confused, Quantopian please elaborate ;-)

Alisa Deychman

@Doug/Costantino - sorry for the confusion and you're right, there was a mistake in the NB. I've updated the list in the main thread. There were a few other fields also in the wrong category, it's now been fixed. Operating_income is marked to be recalculated by Morningstar.

And the IDE messages will be more descriptive this evening, an update will go out tonight. In the warning you'll see the expected impact per category of change.

Thanks for flagging!

kamran sokhanvari

What is a best way to detect Partnerships in the company info? Looks like the Partnership flag (morningstar.balance_sheet.limited_partnership) is being removed? !!

Alisa Deychman

Morningstar will not update the limited_partnership field after March 31. Looking at the values, many of them are NaN with approximately ~15 LPs identified daily. Since many algos rely on this field, we are keeping the field and will forward-fill with NaN so that it doesn't break existing strategies.

If you are looking for a similar field unfortunately, I don't think they offer a good option. They suggested the parent field "Other_Invested_Assets", but from our analysis, that field was quite different.

Alisa Deychman

One thought is you can screen for the Limited Parternship name yourself, by doing something like:

LP = Fundamentals.standard_name.latest.matches(r'.* L[.]?P[.]?$')

kamran sokhanvari

Thanks Alisa. I already filter for LLPs but many partnerships don't have LP or LLP in their names. So that method can leak quite a few Partnerships into the Universe.

The Partnership flag field is essential to really filter those out. I am glad Q is maintaining it. But right now the backtester gives a warning that it wont be available past April 1st. So will that go away?

Alisa Deychman

The limited_partnership field will remain in our Quantopian database but it won't be receiving updating values from Morningstar. Effectively, the data going forward in that field will be NaN.

For your universe, I'd suggest to use the QuantopianTradableUniverse which is a convenient built-in to select liquid, tradable securities and filters out generally unwanted categories of stocks(ADRs, OTC, and uses the LP filter I mentioned above).

kamran sokhanvari

Regarding the rest of the data that will be updated with new methodology from Morningstar, is the old data staying the same and just new data as of April 1st is recalculated? or is the entire series going back is being recalculated with the new methods?

Please clarify. Thanks

Alisa Deychman

The current plan is for the old data to remain the same and ingest new data starting on April 1st in the new calculations. We want to see what does the new data look like. And evaluate how different is it from the old calculation. As of right now, we don't know how different the calculation will be from Morningstar. Our hope is the recalculations are indeed minor, and we can move forward with no other modifications.

If the recalculations look quite different, we will have to think about updating the historical series. Because strategies using a percent change or trailing windows would be significantly impacted in backtesting any time over the April 2018 period. Once the files become available from Morningstar in mid-April (full timeline of events is here) we will know more and can make a more informed decision.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.