I am attempting to load data from a CSV into Pipeline in Research. I am aware how to execute in the IDE, but fetch_csv and local_csv are not identical. I was curious if anyone has successfully loaded data from local_csv into Pipeline? If so, can you share best practice on how to integrate?
I am starting a bit smaller, using local_csv - how can I register a new symbol? Ultimately I would like to add the data into a pipeline column.
I have been able to load the data, but when I print the dataframe, I see the symbol is actually NaN. - I downloaded VIX data from Yahoo, and added in my own symbol column.
Can someone please advise?
I got it working and also fixed the date alignment issue with pandas reindex.
My question now is, how can I add a custom symbol to the data object, so I can reference the custom symbol in pipeline, or simply a column of data to pipeline?
Has anyone solved this yet? I'm interested in getting data from the outside, from a CSV, into a Pipeline as well.
I can load the csv into a DataFrame. But I guess we need to convert the DataFrame into a BoundColumn somehow to use in a CustomFactor in the pipeline. Any ideas?
@Peter
I have not figured out how to do it in research, in the IDE it is easy to do with fetch_csv, but local_csv does not seem to be as flexible as fetch_csv.
Also, I believe the format of the dataframe that pipeline needs is a pandas panel.
Alternatively, you could use pandas concact to add the data to the pipeline output if you are able to get your data in from local_csv, but not able to get the data into the actual pipeline input.
Hope that helps
Currently, there's no way to add custom data to Pipeline, neither in Research nor in Backtesting. The issue is that there is no API for you to register a custom data loader for the pipeline. These only exist internally right now. You can load custom data to research with local_csv as you have been doing, or to an algorithm with fetch_csv, but the data cannot be added to Pipeline in either scenario.
I apologize for the inconvenience and for only seeing this thread now, Adam. This is likely something that we would want to add in the future, but it's not on our short term to-do list. If you have an interesting dataset that you would like to see added to the community, please email [email protected] with the suggestion. We're always looking to add more datasets to the platform.
Disclaimer
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Please sign in or
join Quantopian to
post a reply.
Need help?
Contact support.
Company
Learn & Support
Notices
Follow Us
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.