Help Needed: Daily Indicators In A Minutely Backtest

Back to Community

edited

This gives me daily closing prices:

def initialize(context):  
    pass

def handle_data(context, data):  
   record(Daily_Close =data[sid(24)].close_price)

in a daily backtest. But I want to run a backtest minutely so I can trade at the end of the next minute rather than the end of the next day. And I want to use, say, a 50-Day Moving Average in the algo. What is the best way to use daily indicators in a minutely algo?

In the attached backtest I have stored the daily closing prices in a queue. I want to be able to calculate Moving Average but I don't think a queue is compatible with TA-Lib. And how would I get a VWAP from my daily close data if I needed it?

(The two record statements in the attached backtest produce the same graph in a minutely backtest as the code snippet above does in a daily backtest when run over the same date range.)

8 responses

Amritendu Maji

Peter,
I don't know the answer to your questions but here are couple of suggestions to reduce the data overhead.
1. Define a binary switch called IsLiveTrading. It can be set to true or false. If false, then we do only backtesting and we set it true when live trading or forward testing. Under this condition, we use minute data no matter what.
2. For back testing, we don't need to monitor data every minute. Say we check if the daily low is lower than the sell stop, then we know that stop was hit. So, we then go in to the minute data and then close the trade. This is of course impossible for live trading, but will work for back testing. This will perhaps reduce the time required for backtesting as it will not look into every minutes worth of data.

Thank you for all your hard work.
Maji

Grant Kiehne

Hello Peter,

I'm not clear on what you mean by "use daily indicators in a minutely algo." Are you wanting to use daily closing prices for analysis? If so, I think that you have to accumulate/filter them yourself, since they are not available as a separate data set and there is no flag in the database to indicate a closing price.

I'm not yet familiar with TA-LIB, but presumably the methods work on both daily and minutely data:

Daily vs minute data

Quantopian's TA-Lib methods respect the data frequency in your
backtest or live algorithm. If you're backtesting with daily data,
then all the time periods are calculated in days. If you're
backtesting or live trading with minute data, all the time periods are
in minutes.
So, you just need to scale relevant parameters by 390 to go from daily trading to minutely. For example, I assume that for MA, you would set timeperiod to 390*30 to obtain a 30-day moving average of minute-level data. Of course, if you really need to be working with daily closing prices while running a minutely backtest, then this is not the solution you're looking for.

I'd be glad to try to write an algorithm to help out, but I'm not quite clear what problem(s) you are trying to solve. Perhaps you could clarify?

Best,

Grant

Peter Cawthron

Hello Grant,

Yes, I want to use daily closing prices i.e. MA 50D as an indicator in a minutely backtest. I now believe TA-Lib in Quantopian needs a DataFrame as input which is not the case in a 'standalone' Python environment where TA-Lib functions can be sent an array i.e.

import numpy  
import talib

close = numpy.random.random(100)  
output = talib.SMA(close)

I think I now need to accumulate the prices myself as you suggest but into a DataFrame. So in a minutely backtest I need to check for the last minute of the day, ideally checking for early closes but we can forget that for now, and write the daily close of all SIDs to a DataFrame. Any ideas how?

Dan Dunn

Peter, I'm not much help for the dataframe part of your quesiton. But for the first part, "need to check for the last minute of they day," check out Seong's new algo.

What that algo does is run with minutely data, but only execute once per day, at the end of the day (you configure if it is 1 or 5 or n minutes before the close). You can use Seong's framework to run once per day.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Peter Cawthron

Thanks, Dan, I'll have a look.

Grant Kiehne

Hello Peter,

You actually don't need to worry about early closings to build an accumulator of closing prices. I don't have time now to share the code, but you basically need to hold the most recent prior tic in a context variable. Then, using datetime, test for a new day. If the new day condition is met, then store the tic data in the context variable in your accumulator. This way, you'll always have captured the closing, regardless of scheduled or unscheduled early closes.

Note, however, that if you are looking to submit orders before closing, then you need to know the closing time (and there will be no look-ahead bias, since the closing is scheduled). As you may already know, if you submit the order on the last minute of the day, the order won't be filled until the next day.

Grant

Peter Cawthron

Hello Grant,

Thanks. I realised as much about early closings a little while after I typed the post. I thiught I ws making some progress but it seems not....

Calder Lamb

im having the same issue

You've successfully submitted a support ticket.

Our support team will be in touch soon.