Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
pipeline: what does it mean 'today'

Hello i am doing some very basic exercises with pipeline Custom Factors and i have encountered this issue:

Let's say i want to print the last close price (of AAPL for example) in three ways to see if all three are the same:
so as you can see in the attachment the three ways are:
1) line 28 --> via data: data[sid(24)].price
2) line 29 --> via history : history(7,'1d','price')
3) line 20 --> via Pipeline

Now, 1) and 2) gives the same result, but 3) does not.
I think this is because of the 'today' parameter in the compute method, so what i am asking is:

What date exactly does it refers to and how can i set it up to retrieve the same date as 1) ?

7 responses

Hi Giuseppe,
The pipeline is only run on daily data, which means the close price you are getting is yesterday's close price.

You should also know that pipeline pricing data is split AND dividend adjusted, where as the data in data and history are just split adjusted. You can read more about that here.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Oh my goodness....so ratio calculations are a no no on pipeline data........

Or perhaps not since it is only done up to the date the pipeline is called....needs thinking about....!

Thank you very much karen, the link you have posted is very helpful to me, however my code returns a huge difference in prices for AAPL, then i have tried to switch to another sec AA and the prices are correct, with the same code.

for example if you run this code

from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline import Pipeline  
from quantopian.pipeline import CustomFactor  
#****************************************************************  
from quantopian.pipeline.data.builtin import USEquityPricing  
from quantopian.pipeline.factors import SimpleMovingAverage  
import numpy as np

class GetPrice(CustomFactor):  
    inputs = [USEquityPricing.close]  
    window_length = 1  
    def compute(self, today, assets, out, prices):  
        #100*((current-previous)/(1.0*previous))  
        out[:] = prices  
def initialize(context):  
    pipe = Pipeline()  
    attach_pipeline(pipe, name='my_pipeline')  
    last_close = GetPrice()  
    pipe.add(last_close, name='close price')  
    context.stock = sid(2)  
    context.returns = {}  
def before_trading_start(context, data):  
    context.results = pipeline_output('my_pipeline')  
def handle_data(context, data):  
    print 'data close: ', data[sid(2)].price  
    h = history(7,'1d','price')  
    print 'history close: ', h[sid(2)].iloc[-1]  
    print context.results  

you will get exactly the close price of yesterday for AA in the Pipeline calculation and it's perfectly fine, BUT
if you change the sec id to 24 wich is AAPL you will get a very different close price like 329.720 in 01-04-2011 although data and history both returns a value of 47.33 for the same datetime.

So since my first attempt was with AAPL i was thinking that something was wrong with my code, but now the question is.. what is this strange discrpancy between AAPL close prices calculated with Pipeline?

Karen i see an issue in your example:
i have limited to 2 the number of stocks to be long and short to be able to clearly read what is going on:

if print for debug the stocks i should be long with and the ones i should be short, and then i print the stocks i actually am in the market with, they don't match for the majority of the cases and other than that in some of them the number of stocks in the market are bigger than the sum of longs with shorts so the result of the backtest is definitely not corresponding to the logic it has in its code..

Can you confirm this issue by watching the print logs in this attachment and please clarify this point to me?

Hi Giuseppe,

For the algo you just posted, you rebalance() method is only called at the start of every month, per the schedule_function() command, but you are printing out your stocks that you want to long/short + currently held positions every day in handle_data(), so they won't match up most of the time!

Regarding your dividend questions, those are the correct prices for AAPL. In June 2014, AAPL underwent a 7:1 stock split. In history() and data[stock].price, prices are fully split adjusted from today's date - for example, today's would be as of Nov 24, 2015. However, in pipeline, prices are adjusted as of the current date in the simulation. For a good description of how pipeline handles splits and dividends, see this post. Right now, there is a difference between how pipeline handles splits+dividends, and how the backtester does. We're working to make the rest of backtesting work like pipeline does, but until then, it's important that you understand this distinction!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

i got it !
Jamie thank you very much for your help