Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Off by 1 minute?

Hi,

Newbie here. Seems like my log and the transaction is off by 1 minute. Why is that? And it seems that I have to run the full back test to get my results. Thanks in advance!

import datetime  
from pytz import timezone  
def initialize(context):

  context.qqq = sid(19920)


def handle_data(context, data):  
  dt = data[context.qqq].datetime.astimezone(timezone('US/Pacific'))  
  if dt.year == 2012 and dt.month == 1 and dt.day == 5:  
    if dt.hour == 10 and dt.minute == 0:  
        log.info("Ordered 50 on %s" % dt.ctime())  
        order(context.qqq, 50)  
        log.info("%s" % data[context.qqq].price)  

Log Output:
2012-01-05handle_data:14INFOOrdered 50 on Thu Jan 5 10:00:00 2012
2012-01-05handle_data:16INFO57.43
End of logs.

Transactions List
1 - 1 of 1 PreviousNext
DATE SECURITY TRANSACTION

SHARES

PRICE
$ AMOUNT 2012-01-05 10:01:00 QQQ BUY 50 $57.44 $2,872.00

10 responses

Hello John,

The Quantopian guys can confirm, but I suspect that you are getting the expected results. The backtester is designed to simulate real-time trading on a daily or minutely basis. In your algorithm, when the backtest loop executes order(context.qqq, 50), you are simulating submitting an order. The order gets fulfilled in the next "tick" of the market for QQQ. Under the quick/daily backtest, the ticks are daily and under the full backtest, they are minutely. For highly liquid securities (I think QQQ is one), there should be a tick for every day/minute the market is open. There are a few more details to order submission/fulfilment, but see if this explains your case above.

For the quick/daily backtest, the datetime stamps are, for example (log.debug output):

2012-01-03handle_data:15DEBUG2012-01-03 00:00:00+00:00

The only non-zero output is the date (times are all set to zero). So, I suggest having a look at your logic to see if it is consistent with the datetime stamps of the daily tick data.

@Grant you have it right, thanks!
@John, thanks for the question - feel free to ask more!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Ah, so the quick backtest is daily ticks. That explains why I have to run the full backtest to see my one trade. A few more generic questions:

-Is there access to options trading?
-Can Quantopian connect to an IB account or IB Test account to test in real-time?

It's been a great platform so far, thanks for the hard work guys.

Hi John,

Orders can be either long or short. Not sure about options...I don't think so.

Quantopian is working on paper & live trading with linkage to a brokerage. Some level of backtesting will continue to be free (hopefully including real-time paper trading, but I wouldn't count on it).

Fawce or another of the Quantopian guys can fill you in further.

@John, glad you like it so far! Thanks for the questions.

We don't have options historical data, but it is a frequent request. Our plan is to build an end to end system that allows research, simulation, and trading for US Equities. Then we'll diversify the instruments to include futures, options, and forex.

Integration with a broker for live trading is our current top priority. Is IB your top choice for a broker?
Historical backtesting with the current data will always be free, and to the extent we can, we will expand the free backtesting data universe.

Please let us know what else you'd like to see!

thanks,
fawce

Yes, IB is the top choice for a broker. Of course, everything free would be ideal but unrealistic. I would like to see free real-time paper trading on IB though. It would help with ironing out bugs associated with actual trading. Basically, as close as we can get please! :)

When I uncomment "orclose = 0", the full backtest crashes and gives me an error. A pop-up window comes up and says "your algorithm can't be backtested because it has some code problems". Am I doing something wrong? Thanks in advance.

import datetime  
from pytz import timezone  


def initialize(context):

    context.qqq = sid(19920)  

def handle_data(context, data):  
    oropen = 0  
    #orclose = 0  
    dt = data[context.qqq].datetime.astimezone(timezone('US/Pacific'))  
    if dt.year == 2012 and dt.month == 7 and dt.day == 5:  
        if dt.hour == 7 and dt.minute >= 0 and dt.minute <=5:  
            if dt.minute == 1:  
                oropen = data[context.qqq].price  
                log.info("%s" % oropen)  
                order(context.qqq, 50)  
            #if dt.minute == 5:  
            #    orclose = data[context.qqq].price  

Hello John,

Please see the attached code. I uncommented the line in question and added log output:

orclose = 0  
log.debug(orclose)  

The log output is required to get the code to run under the full/minutely backtest. It will build & run under the quick/daily backtest without the log output line, with a warning that the variable is not used:

11  Warning Local variable 'orclose' is assigned to but never used  

Note that the logging introduces another problem which I'll try to understand. At one point, the minutely backtest bogged down at ~75% completion. When I reloaded the page, it showed completion, but the time series plot had not updated.

In any case, I suspect that the minutely backtester won't accept variables that are not used. If so, this seems overly restrictive.

I found that instead of using logging, your code will also execute with:

orclose = 0  
orclose = orclose  

A kludge, but it works!

There still appears to be a real snafu here, since your code should execute with all of the commented lines uncommented:

import datetime  
from pytz import timezone  


def initialize(context):

    context.qqq = sid(19920)  

def handle_data(context, data):  
    oropen = 0  
    orclose = 0  
    dt = data[context.qqq].datetime.astimezone(timezone('US/Pacific'))  
    if dt.year == 2012 and dt.month == 7 and dt.day == 5:  
        if dt.hour == 7 and dt.minute >= 0 and dt.minute <=5:  
            if dt.minute == 1:  
                oropen = data[context.qqq].price  
                log.info("%s" % oropen)  
                order(context.qqq, 50)  
            if dt.minute == 5:  
                orclose = data[context.qqq].price  

In this case, I still get the "Local variable 'orclose' is assigned to but never used" warning even though clearly there is a conditional assignment in the last two lines of code! Guess it needs to be an unconditional assignment.

@Grant, awesome debug!
I've opened a new bug on our side to make sure that warnings like this one do not block the full backtest run.

Hi John and Grant, we've fixed that issue - warnings no longer block full backtests. Sorry for the inconvenience, should be all set now!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.