transaction times incorrect in full backtest

posted Nov 27, 2012

Hi all,

First of all let me say how amazing the site is and how much it has improved from last year! Awesome.

I've encountered a weird problem that might be a bug.

Running the program, the log output is:

2011-12-01 handle_data: 17 INFO Ordered 10 on Thu Dec 1 15:00:00 2011  
2012-06-01 handle_data: 12 INFO Ordered 50 on Fri Jun 1 15:00:00 2012  
End of logs.

but looking at the Transactions List in the full backtest report, one of the times is an hour out.

$ AMOUNT
2011-12-01 15:01:00 AAPL    BUY 10  $385.44 $3,854.40  
2012-06-01 16:01:00 AAPL    BUY 50  $570.75 $28,537.50

It only happens certain times in the year, so I guess its a daylight savings thing?

Also, I know this is really simple but its so so annoying.. the dates in the quick backtester are in american format, and if you haven't grown up with the month going at the start, it's rather confusing. Could it go from "08/15/2011" to "2011-08-15" ?

8 responses

Nov 28, 2012

Hi @James,

Thank you for reporting the issue. I took a quick look, and daylight savings changes were in March and November. Also, those happen in the middle of the night over the weekends in the US, so it wouldn't explain the anomaly you're reporting.

I thought perhaps AAPL was held from trading on that day, but I couldn't find any reports of a hold in June of 2012.

We will need to examine the trade data directly for the day in question. Did you notice any other dates that exhibited the same behavior?

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Nov 28, 2012

@James, I stand corrected. You were right that this is a daylight savings effect. I cloned your algorithm and modified it to place a trade the friday before DST began, and then the Monday after. Here are my logs:

2012-03-09 handle_data:12INFO Ordered 50 on Fri Mar 9 15:00:00 2012  
2012-03-12 handle_data:17INFO Ordered 10 on Mon Mar 12 15:00:00 2012

and my transactions:

2012-03-09 10:01:00 AAPL    BUY 50  $545.14 $27,257.00  
2012-03-12 11:01:00 AAPL    BUY 10  $549.16 $5,491.60

The effect is from the change in the offset between UTC (what is used in the events in the algo) and EST (what is reported in the transaction list). As you probably know, UTC does not use daylight savings time, while EST does. So, during EST the time difference is 4 hours, and outside EST it is 5 hours (I think).

The issue here is that your intent in the algorithm was to trade at the same time of day in local exchange time, but the simulator implementation is implemented such that your code trades at the same time of day UTC time.

Disclaimer

Nov 28, 2012

Forgot to mention - we all love the date format suggestion, thanks for that!

Disclaimer

Thanks for all that! (and the t-shirt!)

So, to put it another way, if I want my algorithm to buy 50 shares at 2pm everyday local time, then I need to transform the UTC date to the New York (?) time zone, EST?

import datetime  
from pytz import timezone

def initialize(context):  
    pass

# run full backtest from July 2011 to June 2012, cash $100000  
def handle_data(context, data):  
    # get local EST time for US stocks:  
    dt = data[sid(24)].datetime.astimezone(timezone('US/Eastern'))  
    if dt.year == 2012 and dt.month == 6 and dt.day == 1:  
        if dt.hour == 15 and dt.minute == 0:  
            order(sid(24), 50)  
            log.info("Ordered 50 on %s" % dt.ctime())  
    if dt.year == 2011 and dt.month == 12 and dt.day == 1:  
        if dt.hour == 15 and dt.minute == 0:  
            order(sid(24), 10)  
            log.info("Ordered 10 on %s" % dt.ctime())

Giving log output

2011-12-01 handle_data: 20 INFO Ordered 10 on Thu Dec 1 15:00:00 2011  
2012-06-01 handle_data: 15 INFO Ordered 50 on Fri Jun 1 15:00:00 2012

and Transaction List

2011-12-01 20:01:00 Unknown BUY 10  $388.53 $3,885.30  
2012-06-01 20:01:00 Unknown BUY 50  $563.94 $28,197.02

I think an upcoming problem here is that if Quantopian eventually has non-US stocks, then you might need to know which exchange or timezone stocks belong to.

Incidentally, sometimes the securities are listed (e.g. AAPL) and sometimes they come up "Unknown", like in the above sample. Why is that?

correction: 3pm. whoops.

@James,

Yes, you did it perfectly. Furthermore, I think you are right that it is too onerous to make a rule about the time of day, and it will become really confusing if you are dealing with exchanges in multiple timezones.

Talking internally, we hit upon the same problem case, and one idea is to include a new entry in the data object: local_datetime. That way you could just access the datetime in the exchange's timezone for static rules like your example above, but you could do comparisons between exchanges using the datetime property in UTC (i.e. a normalized datetime).

What do you think?

thanks,
fawce

Disclaimer

@fawce

That's a good solution and only adds a single attribute to the structure! However, the local_datetime will have overlapping dates once a year (and presumably a 1hr break in the dates once a year) as DST comes in and out of effect. (see pytz)

I'm not sure how much that really matters since there is always UTC to fall back on. I suppose some algorithms looking for a specific time every day might fail in that one case, but then its always going to be out of trading hours anyway?