Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
How to get historical fundamental data with the Pipeline API?

Hello,

I already posted about this subject here:
https://www.quantopian.com/posts/introducing-the-pipeline-api#561bc7b2e2fd0e7b2f0004d3
and here:
https://www.quantopian.com/posts/fundamental-history-based-algo#561e759a0c3c281910000259

but now I decided to start a new post in order to better define the scope of the problem.
The question is, how could one get different time horizons for the fundamentals data, given that the Quantopian fundamentals data always attain to the most recent quarter?

For example, how can I write a factor that takes care of the Total Revenue of the last 12 months?

I implemented the following solution, but it's to slow and a Timeoutexception occurs:

days_in_quarter = 70  
num_of_quarters = 4  
def retrieve_ttm(array):  
    # Differences between the next and the current value, after havind convertet NaN and Inf to numbers  
    diff = np.diff(np.nan_to_num(array))

    # Track the indexes where the difference isn't zero: a change has occurred. Shift the indexes by one.  
    idx = np.where(diff != 0)[0] + 1

    # Add zero as the first index (by definition)  
    idx = np.insert(idx, 0, 0)

    # Look for periods without changes longer than 'days_in_quarter'  
    dup = np.diff(idx) > days_in_quarter

    # Add an index in order to duplicate the previous value  
    idx = np.append(idx, idx[dup] + days_in_quarter)

    # Sort the index for consistence  
    idx = np.sort(idx)

    return array[idx[-num_of_quarters:]]  
3 responses

Hi Costantino,

Have you tried running this again recently? We rolled out some performance upgrades to pipeline not too long ago and I was wondering if it may have fixed your timeout problem. If not, would you be willing to share the backtest so that we can take a look at the entire algo/see if we can improve efficiency elsewhere? (Emailing feedback would work too!)

Regarding getting end-of-quarter dates, we haven't solved this problem yet but it's on our short list to figure out. I totally understand that this is a workaround because of the fact that you don't have the functionality you want in the first place!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Jamie!
I tried the algorithm again, and one of the implementation (RevenueTTM3) worked! A backtest is attached.
It would be great if you or the community could help to improve it, expecially its robustness.

Anyway - as you also said - it remains quite an "hack" and I hope that the Quantopian team will figure soon a solution to offer the end-of-quarter date directly in the Pipeline API... I'm very glad to hear, that this feature is already on the short list!

I've also create a notebook to experiment with the implementation