Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Running pipeline more than once a day

Dear members,

I'm exploring the usage of pipeline and as per the documentation, the pipeline is designed to capture the following

From Quantopian's help page:
Many trading algorithms are variations on the following structure:
1. For each asset in a known (large) universe, compute N scalar values for the asset based on a trailing window of data.
2. Select a smaller “tradeable universe” of assets based on the values computed in (1).
3. Calculate desired portfolio weights on the trading universe computed in (2).
4. Place orders to move the algorithm’s current portfolio allocations to the desired weights computed in (3).

From step 1, "we compute values on a trading window of data". As per my understanding, the pipeline is designed to run only on 'daily' trailing prices. Is there a setting which I can change so the computation happens every few minutes, and the returned pandas.datframe is updated. I am looking for this functionality to implement intra-day trades. Eg, a strategy like:
Buy when RSI for 10 periods on 15 minutes data < 20
Sell when RSI for 10 periods on 15 minutes data > 80
(This is just an example)

-Dee

4 responses

Hello Dee!

I Can't seem to see a way to run a pipeline every minute, but perhaps the handle_data() function will be of use? This runs every minute. You could compile a pipeline of the Q1500, and then use the returned stocks and to calculate an RSI from these stocks. Data for this can come from the history() function.
Here is a basic algo to print every stock in the Q1500US every minute. You can add calculations from here.

from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline import Pipeline  
from quantopian.pipeline.data.builtin import USEquityPricing  
from quantopian.pipeline.factors import AverageDollarVolume  
from quantopian.pipeline.filters.morningstar import Q1500US  
def initialize(context):  
    # Create our dynamic stock selector.  
    attach_pipeline(make_pipeline(), 'my_pipeline')  
def make_pipeline():  
    # Base universe set to the Q500US  
    base_universe = Q1500US()

    # Factor of yesterday's close price.  
    yesterday_close = USEquityPricing.close.latest  
    pipe = Pipeline(  
        screen = base_universe,  
        columns = {  
            'close': yesterday_close,  
        }  
    )  
    return pipe  


def handle_data(context, data):  
    context.output = pipeline_output('my_pipeline')  
    for stock in context.output.index:  
        print stock  

Thank you Max!

I do see that now the pipeline output is printed every minute. But I am still concerned if we could feed data from the history() function.. From the documentation:
The most important thing to understand about DataSets is that they do not hold actual data. DataSets are simply collections of objects that tell the Pipeline API where and how to find the inputs to computations.

My understanding here is Pipelines expect references as an input, and not the actual data. On the other hand {history} function could only return the desired data.

-Dee

Hi Dee,

Your understanding of Pipeline and the fact that it expects references as input is correct. Currently, Pipeline is designed to be run once every day on daily data. Under the hood, a lot of the engineering work is optimized around this design.

To get intra-day RSI, I would recommend using pipeline to filter down the set of 8000+ equities down to a list that you want to consider trading. A good starting point is the Q1500US or the Q500US. After that, you can use data.history() to get the trailing data that you need at whatever frequency (either via a scheduled function or handle_data). You can then compute the RSI on that data.

Does this help?

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hello Jamie,

Thank you for your recommendation!

I am currently using the scheduling function every few minutes and building a pandas panel to capture the computations. Just thought an easier way would have been attaching a pipeline at fixed intervals. I will continue working in the scheduling function.

Thank you!

-Dee