Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Do we have access to data within make_pipeline()

Is it possible to get access to data from within the make_pipeline(context) ?

def initialize(context):

attach_pipeline(make_pipeline(context), 'my_pipeline')

def make_pipeline(context):

def before_trading_start(context, data):
context.pipeline_data = pipeline_output('my_pipelnie')

I want to access data (that is available in before_trading_start) in the make_pipeline method. Is there any way to pass it along.

2 responses

Can't really access anything from 'before_trading_start' in the typical 'make_pipeline' method. The reason is simple. A pipeline is defined and attached to an algorithm (in your case defined in the 'make_pipeline' method) exactly once in the 'initialize' method. This happens before any of the other methods are called. A pipeline can only be defined once and cannot be changed mid-algorithm.

The pipeline definition (ie the 'make_pipeline' method in this case) doesn't do any real calculations or fetching of data. It simply sets up the columns and rows to be returned in the pipeline dataframe. After initialization, the pipeline goes on to fetch data and do it's calculations rather asynchronously to the other algorithm methods (which is another reason why an algorithm cannot interact, or pass data to, a pipeline). Once a pipeline is defined pretty much all the algorithm can do is get the outputed dataframes for specific dates by using the 'pipeline_output' method.

Hi Dan, thanks for your detailed response and insights on how the pipelines work. Very helpful indeed.