Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Historical Fundamental Data

Hello,

I'm totally new with Quantopian/any sort of programming, and I'm working on my first algorithm. I've watched the Getting Started Tutorial and read the Pipeline Tutorial, and both were very helpful but I still am not sure if it's possible to get past fundamental data through the Pipeline API.

For example, can you look up when the FCF margin was for the full year of 2015?

Thanks,
Ian

7 responses

Hi Ian,

Technically yes, it's possible. However, you will need to reduce the trailing year-long window to a single value to output it from your pipeline. For example, you could take the mean, the max, or use a more complex computation that reduces the year to a single output value. The concept is very similar to the SimpleMovingAverage factor in the pipeline tutorial.

That being said, I should caution you that loading a year of fundamental data in pipeline will be quite slow. Big fundamental data loads in pipeline currently take significantly longer than pricing data lookbacks, but we are actively working on speeding this up.

I hope this helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thank you for the response! FYI everything I am mentioning here I am doing in Research, if that makes a difference (like I said, very new at this)

I was browsing the forum and found the following code to retrieve past data:

class gross_margin_year(CustomFactor):

inputs = [morningstar.operation_ratios.gross_margin]

def compute(self, today, assets, out, inputs):  
    out[:] = inputs[-365]

def make_pipeline():
gm_1yr = gross_margin_year(window_length = 365)

I tried this with the current ratio just to see if it would work, and I got accurate results. However, when I used the following, the results were way off:

current_ratio = morningstar.operation_ratios.current_ratio.latest

Can you tell me why these would give me such different figures?

Thanks,
Ian

Pipeline definitions between research and the IDE are the same so there shouldn't be any difference there, but thanks for mentioning it.

In your custom factor, you are returning the gross margin value from 365 trading days ago. If you want the value from a year ago, I'd recommend using 252 (~252 trading days per year).

Your .latest factor looks correct. Would you be willing to share your notebook so that I can take a look at the numbers you're getting? As well, what source are you comparing the numbers? I'm curious where the deviation is coming from.

If you want to know the latest reported Free Cash Flow margin, based upon the latest edgar filings, then one can use the Morningstar dataset and retrieve the data via a pipeline (as Jamie alluded above). You can use something like the following. Don't think summing them is what you want.

# First thing is to import the morningstar dataset  
import quantopian.pipeline.data.morningstar as mstar

# Then get any data  you need. Real numbered data are called 'factors' in Quantopian.  
# The format is 'dataset_name.category_name.field_name.latest'  
free_cash_flow = mstar.cash_flow_statement.free_cash_flow.latest  
sales = mstar.income_statement.total_revenue.latest

# Calculated fields (or columns or factors) can be made using simple math operators  
fcf_margin = free_cash_flow / sales

I've attached a notebook with a sample. Doing this in an algorithm is almost identical except you can only retrieve data for a single date (ie the backtest date).

Fundamental data is typically very static from day to day with discontinuous jumps at the release of a new filing. One can see this in the notebook cell with the AAPL data. Also note that data isn't updated exactly as of the filing date but rather a few days after (typically). There are also times that the filing date is BEFORE the 'as_of' date (don't know how that could be?) From what I can tell the values (in this case 'Sales' for example) are the trailing 4 quarters of sales with the year end data coming from the annual filing. Maybe verify this for a particular piece of data you need though.

There is A LOT of fundamental data so comb through it and make sure you are getting what you think you are getting. I'd suggest ALWAYS viewing the data in a notebook before using in an algorithm to ensure you have what you expect. See https://www.quantopian.com/help/fundamentals#operation-ratios.

I've attached the notebook I have so far. For example, it gives a FCF margin of 0.0355, or 3.55%, for DD. For work I use s&p capital IQ for data, and I'm seeing that they have a levered FCF margin of 8.0% and an unlevered of 8.9%. From what I understand, the API gives the levered free cash flow.

Dan, thanks for the reply as well. It looks like I did mostly what you showed there, or at least had the same sort of idea. I think you're right about it being the past year of data, probably TTM. I was taking a look at that list in "Help" earlier today, which is where I got everything I used in the notebook. Definitely a very helpful resource.

Ian

You may want to consider using an index of 0 rather than a fixed negative index in your custom factors. Instead of..

class current_ratio_year(CustomFactor):  
    inputs = [morningstar.operation_ratios.current_ratio]  
    def compute(self, today, assets, out, inputs):  
        out[:] = inputs[-365]

class return_on_equity_1y(CustomFactor):  
    inputs = [morningstar.operation_ratios.roe]  
    def compute(self, today, assets, out, inputs):  
        out[:] = inputs[-365]

Do something like this..

class current_ratio_year(CustomFactor):  
    inputs = [morningstar.operation_ratios.current_ratio]  
    def compute(self, today, assets, out, inputs):  
        out[:] = inputs[0]

class return_on_equity_1y(CustomFactor):  
    inputs = [morningstar.operation_ratios.roe]  
    def compute(self, today, assets, out, inputs):  
        out[:] = inputs[0]  

That will make it relative to your window length and won't cause an error if inadvertently putting in a shorter window length. Also, regarding window length. If you want a 1 year lookback then use window_length=252 as Jamie mentioned since this is in 'trading days' and not calendar days (maybe see https://en.wikipedia.org/wiki/Trading_day)

Ok, that makes sense. Also I figured out the differences in the numbers. Looks like the API has the data from the most recent quarterly reports, which makes sense. The formulas that cap iq uses are a little different than the API, but nothing too significant. I really appreciate the responses here.

Thanks,
Ian