Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
How to utilize FRED datasets in pipeline?
Thomas Goldenberg
posted Dec 24, 2017

Hi,

New to the platform, but having a hard time figuring out how to use some of the datasets, for example:

from quantopian.interactive.data.quandl import rateinf_inflation_usa # Inflation rate  
from quantopian.interactive.data.quandl import fred_gnp # GNP  
from quantopian.interactive.data.quandl import fred_gdp # GDP  
from quantopian.interactive.data.quandl import fred_icsa # Unemployment Claims  
from quantopian.interactive.data.quandl import adp_empl_sec # Total Jobs

These datasets are in the form of a Blaze expr. object. I figured out how to make them a Pandas.dataFrame, but I have no idea how to access the nearest time values (asof_date) in a CustomFactor or in the Pipeline itself.

What I would like to do is this:

For every day, look at the % change for each of the above values (GDP, GNP, etc) between the current value and the value a month ago).

Would appreciate any help on this, as I've been unable to make much progress. Thank you.

2 responses
Thomas Goldenberg
Dec 24, 2017

Currently handling with an (inefficient) CustomFactor. Here I am iterating over each entry to find the closest date. Would love a more efficient approach, though.

from quantopian.interactive.data.quandl import adp_empl_sec 

jobs = blaze.compute(adp_empl_sec)  
jobs = jobs.sort_values("asof_date")

def find_most_recent_jobs(date):  
    for i in range(len(jobs)):  
        asof_date = jobs.iloc[i].asof_date  
        date = date.replace(tzinfo=None)  
        if asof_date > date:  
            latest = jobs.iloc[i-1].total_private  
            prev = jobs.iloc[i-2].total_private  
            pct_change = latest / prev - 1  
            return pct_change

class TrainingData(CustomFactor):  
    inputs = []  
    outputs = ["jobs_change", "gnp_change", "gdp_change", "inflation_change", "unemployment_change"]  
    window_length = 30  
    def compute(self, today, assets, out):  
        jobs_change = find_most_recent_jobs(today)  
        out.jobs_change[:] = jobs_change
Jamie McCorriston
Dec 27, 2017

Hi Thomas,

Here's a notebook that gets the percent change between the last two Fred GNP values. I also put in some custom factors that get the latest GNP value as well as the latest corresponding asof_date so that you can play around with it. Note that since this is a value that updates quarterly, the percent_change will be forward filled on most days.

Also, Pipeline is limited to outputting dataframes where the result is mapped to US equities. To work with macroeconomic data, I broadcast the result across all of the assets and added a StaticAssets filter to the pipeline to just pull up GNP value and percent change each day.

Let me know if this helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.