Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Loading additional and multiple data for simulation
Algo Trader
posted

Is there anyway to load multiple data sets (say fundamental, price-volume) etc in a strategy?

How can I add more custom datasets and map them to a stock in strategy universe - could anyone point to a custom dataloader?

I'd like to load datasets DataA and DataB (which need to be mapped to stock_1.. stock_2 etc). In a strategy then like to load dataFieldA['stock1'] and dataFieldB['stock1'] and so on to generate target positions for all the stocks in trading universe. Is there any example I can start with?

4 responses
John Fawcett

Yes, all of this is possible using Pipeline. There's an introductory post here: https://www.quantopian.com/posts/introducing-the-pipeline-api
Right now, you can combine all price/volume data with all fundamentals. You can browse the datasets available to Pipeline in the help docs: https://www.quantopian.com/help#importing-datasets

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Algo Trader

Thanks John. Is there any example on how to write a data loader? I'd like to write a data loader for datasets which are stored in YYYYMMDD/MM/DD/data.YYYYMMDD format and contents are "stock|dataAField1|dataAField2|dataAField3". And how do I ensure that multiple data loaders (DataA, DataB and so on) will map the fields to the "stock" so in a strategy I could do target['stock1'] = "dataAField2 * dataBField2"?

(Apologies for such naïve queries as I am a newbie).

Thanks a ton for your help.

John Fawcett

Hi AT,

For integrating your own data, we have a feature called fetcher, which ingests any csv. Fetcher handles symbol mapping and date alignment with other datasets.

We have not yet integrated fetcher with Pipeline, however. Managing the investable universe and blending fetcher with other data is therefore not as simple right now.

We haven't connected fetcher to pipeline because we are thinking through how we might open up the infrastructure we built for third party data (see our 3rd party data here) to user defined data. That would allow us to maintain a point-in-time history for user data, guarantee reproducible backtests with user data, and keep on top of symbol mapping improvements.

What's the input data to create your data fields? The best option would be to reproduce those calculations using Pipeline and our pricing/fundamental data.

thanks,
fawce

Algo Trader

Unfortunately, I cannot recreate the data fields using pricing/fundamental data.

But the infrastructure you mentioned to support user defined data could help massively in this case. An example (for instance Morningstar data loader) could help understand how I can customize it for my usage.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.