Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
market-on-close (MOC) orders?

Will market-on-close (MOC) orders be added to the backtester (see https://www.interactivebrokers.com/en/?f=%2Fen%2Ftrading%2Forders%2Fmoc.php)?

I often see posts by folks interested in closing out positions by the close of the market. An MOC order type would be the way to go, correct?

28 responses

For now, we're only supporting some basic order types through IB. There's a few reasons for that:

  1. We want to provide as solid a product as we can. Smaller feature sets are easier to make robust.
  2. It's possible to create complex market close orders using the simple tools.
  3. We don't want to go too far down the IB path - we want to support other brokers as well

We'll see how it goes with feedback and development, but my gut is that the more complex and exotic order types offered by a given broker will be painful to implement. We want to keep the abstraction level in our code as much as we can so that we can connect to more brokers using the "basic" order types.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Dan,

The tricky part in backtesting, it seems, is accounting for days when the market closes early. I came across:

http://www.nyse.com/pdfs/closings.pdf

Any ideas on how to handle this (aside from manually typing in all of the associated datetimes)? Perhaps Quantopian could set up a list, accessible with Fetcher?

Also, for paper/live trading a list of planned closings is required (in the absence of a MOC order type), correct?

Grant

I agree, Grant, that's the type of thing we need to make more easily available. For now, manual entry of the dates is the only option. We do have a date library for early closes deep in our code, but it's not exposed in the user interface yet. Definitely on the to-do list.

It's worth pointing out that even if you know the planned closings, there is no 100% sure way to get an order executed before the market closes. Unexpected trading suspensions, low trade volume, etc. are all potential pitfalls.

Regarding that date library we use internally, I believe you can access it from your algo via from zipline.utils.tradingcalendar import get_early_closes It takes as arguments a start date and an end date and returns those NYSE planned early closes (since 1993) that fall in the given range.

-Rich

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Rich,

I'll give it a try. You might consider a usage example on your help page, since I've seen the question come up more than once regarding closing out prior to the end of trading.

Grant

Hello Rich,

Works!

I've attached a backtest. A few questions:

The log output for the first three datetimes is:

2008-01-03PRINT1999-12-31 00:00:00+00:00  
2008-01-03PRINT2007-07-03 00:00:00+00:00  
2008-01-03PRINT2007-11-23 00:00:00+00:00  

Why the 1999 date? My start date was in 2007.

Also, do all of the dates returned by get_early_closes correspond to 1 pm closings?

Grant

Why the 1999 date? My start date was in 2007.

That's a bug. I've fixed it in https://github.com/quantopian/zipline/pull/206 . Note that even after my fix is reviewed and pulled into master, there will be an additional delay before we release it to the web site.

Also, do all of the dates returned by get_early_closes correspond to 1 pm closings?

Yes, according to http://www.nyx.com/holidays-and-hours/nyse .

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hello Jonathan,

The 1999 date is still being reported:

2008-01-03PRINT1999-12-31 00:00:00+00:00  
2008-01-03PRINT2007-07-03 00:00:00+00:00  
2008-01-03PRINT2007-11-23 00:00:00+00:00  

Would you expect it to be fixed by now?

Grant

Hi Grant,

That fix was pulled into zipline, but it hasn't been deployed to the Quantopian servers yet. We'll definitely reply again when it is!

-Rich

Hello Rich,

Looks like the problem is fixed now:

2008-01-03PRINT2007-07-03 00:00:00+00:00  
2008-01-03PRINT2007-11-23 00:00:00+00:00  
2008-01-03PRINT2007-12-24 00:00:00+00:00  

Thanks,

Grant

Good catch Grant! We did in fact deploy the fix.

Best,
Rich

Hi guys,

Here's an example implementation. Seems to work fine for checking if the market will close early (I ran the example on daily data, but the idea is to run on minute data, and close out positions prior to market close, for example).

I noticed some comments on github that this may not be the most efficient approach (calling get_early_closes for every call to handle_data). Is there a better way?

Grant

Thanks Grant, nice solution.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Here's another version that uses get_datetime().

By the way, how does get_datetime() work? The help page states:

get_datetime()  

Returns the current algorithm time.

Parameters

None

Returns

Returns a Python datetime object with the current time in the
algorithm. For daily data, the hours, minutes, and seconds are all 0.
For minute data, it's the end of the minute bar.

How is the "current algorithm time" defined? Previously, datetime was associated with time stamps of entries in the trade data base. A bit confusing...is the backtester still event-driven, or does handle_data get called for every calendar day/minute?

Grant

I noticed some comments on github that this may not be the most efficient approach (calling get_early_closes for every call to handle_data). Is there a better way?

Call it in initialize, specifying a time range from the earliest date you intend to run your backtest until today, and store the result in your context for use in handle_data.

You can do get_datetime().date() in context.early_closes in handle_data to find out if the current day has an early close.

How is the "current algorithm time" defined? Previously, datetime was associated with time stamps of entries in the trade data base. A bit confusing...is the backtester still event-driven, or does handle_data get called for every calendar day/minute?

The backtester is still event-driven. That is not going to change. ;-)

handle_data gets called at the end of each time period, but if there were no trades for a particular entity during that time period, then data will contain the bar for the last time period for which there was a trade, including the earlier timestamp. So you can compare the value of get_datetime() with the timestamps on the data for individual equities to find out whether they traded in the current time period, or if not, how long ago they last traded.

Thanks Jonathan,

When I get the chance, I'll try your recommended approach of calling get_early_closes only once when the algorithm is initialized. Eventually, I'd like to work up an example and post it, since closing out positions at the end of the trading day seems to be a common practice.

Regarding get_datetime(), I'll have to write some code to understand it better. From your description, it sounds like get_datetime() will return a value every calendar day (or every calendar minute) if I call it within handle_data. The effect, I suppose, is that handle_data then gets called every calendar day/minute. Or do you still restrict the calls to handle_data to be only when there are trades?

Grant

get_datetime() will return a different value every time you call it, but the values won't necessarily always be for sequential time periods, because handle_data() will only get called for any particular time period if at least one of your algorithm's securities traded during that period.

Thanks Jonathan,

Makes sense now. If at least one of the securities being backtested traded, then get_datetime() will provide the current datetime. Otherwise, get_datetime() won't be called, since handle_data() won't be called. The basic "event-driven" architecture is the same, and you've just provided a utility function to output the current datetime of the backtest.

Also, unless something has changed, there is no way to force the backtester to call handle_data() for every day/minute the market is open; I need to use a proxy security (e.g. SPY). Correct?

Grant

Hello Jonathan,

Attached, please find my attempt at an efficient early close indicator. Please have a look...something isn't quite correct, since it only picks up the early close on July 3, 2013.

Grant

Please have a look...something isn't quite correct, since it only picks up the early close on July 3, 2013.

That would be a bug. I've submitted a Zipline PR with a fix. The usual disclaimers apply... Once it's merged into Zipline, it then has to be released to the web site, which could take a few days or even more depending on our release cycle.

In the meantime, you can work around it by creating datetime()s to pass into get_early_closes() without any HHMMSS data assocated with them, and with timezone set to UTC, e.g.:

import datetime  
import pytz  
from zipline.utils.tradingcalendar import get_early_closes  

start = datetime.datetime(2001, 1, 1, tzinfo=pytz.utc)  
end = datetime.datetime(2014, 1, 1, tzinfo=pytz.utc)  
context.early_closes = get_early_closes(start, end)

Thanks Jonathan,

See attached for another work-around.

I noticed some odd behavior. When I run the attached code in the quick backtester, the log output is:

2012-07-03PRINT2012-07-03
2012-11-23PRINT2012-11-23
2012-12-24PRINT2012-12-24
2013-07-03PRINT2013-07-03

The dates above agree with those on the chart created with the record function.

And when I run a full backtest, the dates are the same:

2012-07-03PRINT2012-07-03
2012-11-23PRINT2012-11-23
2012-12-24PRINT2012-12-24
2013-07-03PRINT2013-07-03

However, the dates indicated in the chart created with the record function differ--they are one day early in the results pane of the full backtester. But I just noticed that below (in the shared backtest), the dates are correct. Strange...

Grant

Hi Grant,

I ran a full backtest of the algo you just attached and zoomed in on the results, and the peaks in the record graph show up on the correct days for me.

Remember that when the number of data points you're graphing gets large enough, we start to aggregate the data, and that can cause it to seem to move around a bit. Perhaps that's what you were seeing?

jik

Jonathan,

It's subtle. The date displayed near the Week/Month/All buttons is off by one day when a full backtest is run and the results are displayed. For the attached backtest, it will display as July 2 for the peak, but it should be July 3, when the market closed early.

Just let me know if you can't reproduce the problem...I'll send you some screen shots.

Grant

I see what you mean now. I was looking at the date on the axis below the graph, not the date to the left of the Week/Month/All buttons. I can reproduce the problem you're seeing -- that date is a day earlier than it should be. I've opened an issue about it in our internal bug database. Thanks for pointing out the problem!

Grant,

I believe that this may work for market-on-close orders. By converting to local time EST, pytz accounts for daylight savings time

log: 2013-03-07moc_order:43DEBUG2013-03-07 20:55:00+00:00
2013-03-08moc_order:43DEBUG2013-03-08 20:55:00+00:00
2013-03-11moc_order:43DEBUG2013-03-11 19:55:00+00:00
2013-03-12moc_order:43DEBUG2013-03-12 19:55:00+00:00
2013-03-13moc_order:43DEBUG2013-03-13 19:55:00+00:00

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Seong,

I hadn't considered factoring in daylight savings time.

Grant

Grant,

I saw your comment from before, thanks for catching that. I've updated and cleaned up the code. I believe just using early_closes, as you're passing it into the function already, should work.

from pytz import timezone  
from datetime import datetime, timedelta  
from zipline.utils.tradingcalendar import get_early_closes  
import pytz

def initialize(context):

   context.aapl = sid(24)  
   ## Initializing start and end dates for security trades  
   start_date = context.aapl.security_start_date  
   end_date = context.aapl.security_end_date  
   context.early_closes = get_early_closes(start_date,end_date).date

def handle_data(context, data):  
   ## for testing  
   moc_order(context.aapl, 1, 5, context.early_closes)  
def moc_order(sid, amount, minutes, early_closes):  
   ## converting to eastern time becaue pytz brings it into post adjusted dst  
   loc_dt = get_datetime().astimezone(timezone('US/Eastern'))  
   ## setting the early closes  
   date = get_datetime().date()  
   ## checking if an early close, if it is, close at 12:56 EST  
   if date in early_closes:  
        if loc_dt.hour == 12 and loc_dt.minute == (60-minutes):  
            log.debug(get_datetime())  
            return order(sid, amount)  
        else:  
            return  
   ## because pytz already accounts for DST, close always at 3:56 EST  
   elif loc_dt.hour == 15 and loc_dt.minute == (60-minutes):  
        log.debug(get_datetime())  
        return order(sid, amount)  
   else:  
        return