Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Question re fetch csv for rows that span more than one year

Hi Everyone
When I use fetch csv for data within one year the data pulled in and the date of the backtest match. But when I have data that spans more than one year the first year works fine but the second year does not.
Meaning I do not get the right row for the backtest date in the second year.
Has anyone else had this issue and if yes, can you please let me know how you rectified it or workaround?
Much appreciate
Savio
I am using
fetch_csv('https://docs.google.com/spreadsheets/d/e/...........&single=true&output=csv', date_column = 'date', date_format = '%Y-%m-%d', pre_func = preview)
Header row is
symbol date limit
First row is
F 2008-1-2 5.26

4 responses

Hi Savio,

I recommend switching to Self-Serve Data. You can leverage the pipeline API and we'll capture the point-in-time nature of your data, so it is eligible for use in the contest (fetcher is not).

Based on the sample above, your existing data will fit nicely. Download a csv from Google Sheets and upload load that "historical data" via Add Dataset on your Account/Data tab . You can use the same google sheets URL above to load live data, updates will be processed each trading day.

let me know if you have any additional questions

Chris

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@Chris
Hi Chris
I am getting a key error for the code below - I am trying to access the self-serve data using asof_date.
It is likely that I am overlooking the obvious.
Any thoughts are much appreciated.
Have a nice weekend
Savio

from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.user_ import testdata
from quantopian.algorithm import attach_pipeline, pipeline_output

def initialize(context):
pipe = Pipeline(
columns={
'my_dataset': testdata.limit.latest
},
screen=testdata.limit.latest.notnull())
attach_pipeline(pipe,'my_pipeline')

def before_trading_start(context, data):
results = pipeline_output('my_pipeline')
context.pipeline_results = results.sort('asof_date')

Hi Savio,

If you want to access asof_date you need to add the column to your pipeline..

columns={
'my_dataset': testdata.limit.latest, 'asof_date': testdata.asof_date.latest },

I suggest starting in research and playing around with the dataframe returned from run_pipeline before you migrate it to your algorithm. Lesson 3 and 6 from the getting started tutorial have some detailed examples.

Thank you Chris I see it now - that was very helpful.
Perhaps if I can suggest for the next person using Self-Serve data the code that is generated as a starting point should include all the columns that were imported.
Thank you very much for your help.
Much appreciate and a pleasant weekend
Savio