The reason to cancel all open orders at the end of the day is because prices can move significantly overnight, especially for thinly traded securities. We took a more prudent and conservative approach and didn't want to expose an algo to any wild fluctuations. To be clear, if you have a partially filled position, any remaining open orders for that position will be cancelled at the end of day. We're flexible and there is room to change this behavior, depending on the feedback from live traders and how they like the experience. Also, I want to point out that all paper and live trading agos with IB are relaunched every morning and will follow the algo's logic.

And you're correct, an order cannot be filled at market open. The first order will be placed at 9:31AM and be filled within a few seconds, and you will see the fill in your dashboard at 9:32AM.

Thanks Alisa,

Makes sense, I suppose, since the algorithm would have no way to monitor overnight events that could influence prices, and no way to modify orders accordingly (if I understand how live trading will work).

I hadn't considered the daily re-launching. Another (minor) consideration in algorithm writing.

Grant

Hi Alisa,

The backtester (and presumably live trading) calls handle_data at 4 pm (assuming a normal trading day). However, the market closes at 4 pm, and open orders are cancelled? But the algorithm can still do stuff after the close. But the algorithm is shut down when the market closes, and is then relaunched the next day. Kinda confusing.

A few questions:

• When, precisely, are the EOD orders cancelled? For example, under live trading, when handle_data is called at 4 pm, will any open orders already have been cancelled automatically?
  
• Generally, I'm confused about what happens under live trading when handle_data is called at 4 pm. The market is closed, right? But the algorithm is still running, so it can do stuff, like still submit/cancel/query orders, etc.?
  
• When you say "algos with IB are relaunched every morning and will follow the algo's logic" what does it mean? For example, if the algorithm stores data in context.algo_data will it be available when the market opens again?
  
• My understanding is that you will eventually update the backtester so that it automatically cancels all open orders at the EOD . Will you also emulate the daily algorithm relaunching?

Grant

Alisa,

Another question is will there be the option to manually intervene and submit orders via IB directly, after the market is closed (e.g. to submit an order to be executed immediately upon market open)? Since the algorithm is shut down, this should be possible, right? For example, if company XYZ goes belly-up overnight, then it would be advantageous to sell ASAP the next day. Algorithmically, there would be no way to catch the open, since stop loss orders would be cancelled, right? And new orders couldn't be submitted until 9:31 am.

Grant

Hello Alisa,

Another follow-on question is did you consider synthesizing an empty minute bar of data, timestamped at the market open (9:30 am)? If so, then handle_data would first run on the empty bar, allowing trading as close to the open as possible, within Quantopian/IB timing limitations.

Grant

One of my favorite strategies in the old days was to fade hype-driven opening gaps alongside the NYSE specialists - if there was a too-big gap up being indicated, submit a ladder of limit sell orders. All the marketable ones got crossed in the opening auction, cancelled the rest, and then exited after 10m/30m/60m. I don't know if the market structure remains profitable for that sort of trade, it's changed a lot since then.

My point being - that sort of strategy would be impossible to automate, if I understand the above correctly.

Hi Grant,

Here are the answers to your questions:

1. In live trading and paper trading, when handle_data gets called at 3:59PM, it places orders. In paper trading, when handle_data() is called at 4:00, it places order. In Iive trading, at 4pm, no orders are placed, and any open orders are cancelled (are expired). When I say cancel I mean that under the hood, all Quantopian orders have an expiration time of 4:00PM. That way, even if your algo crashes or something goes awry, any open orders will not get placed after 4PM. At the end of the day, your portfolio will only have positions for your filled orders.

2. When your algo is relaunched in the morning, it checks the code logic against your existing IB positions. For example, if you have an order target percent, it will calculate the stock allocation in your positions and see what adjustments (buy/sell) need to be made in order to reach your intended allocation. If you're using history, it will pull in those values and use the logic to drive your decisions. If your algo stored data, this data will be carried over into the next trading day and will be available when the market opens. Essentially we're seeking to make it a seamless experience, as if the algo ran continuously overnight, without exposing it to any sudden overnight market fluctuations.

3. As you've pointed out, open orders are not cancelled in backtesting and zipline paper trading, so they will persist the next day. Our hope is to identically align the backtest simulations with the live trading experience, so that both show identical behavior.

4. You are more than welcome to tweak your algo during non-market hours. To do so, stop the algorithm, make the changes, and then redeploy to the market, which will relaunch when the market is open. You can make any adjustments you'd like including, but not limited to, stock orders, order targets, universe selection, and calculated transformations. If while backtesting, you found a minor tweak to your strategy that will increase performance, you can stop the algo and add the change, and deploy.

5. In the future world of Quantopian, we hope to be able to trade at the market open - this is a request we've heard frequently and we're listening! This depends on the level of feedback we receive, particularly from the beta and live trading testers. But you're correct that at the moment, this functionality is not available. The first order is placed at 9:31AM and can get filled within a few seconds depending on the liquidity, and will appear on your live trading dashboard at 9:32AM when the next handle_data runs.

Best,
Alisa

Hi Simon,

That sounds like an interesting strategy! But unfortunately, you're correct, it wouldn't work here because you don't have the opening auctions and opening cross information. You will only see the transactions during market hours.

Thanks Alisa,

Your thorough response was helpful. Regarding "If your algo stored data, this data will be carried over into the next trading day" are you referring to data stored in context?

Also, I gather that the nightly automated algorithm restart differs from a manual stop/pause and then re-launch by a user in that any stored data in context would be lost. Correct?

Grant

That's exactly right, the data stored in context will be carried over to the next trading day and the information will not be lost.

When algorithms are restarted by Quantopian prior to market open, this is more of a cautionary and conservative measure on our part. However, when you manually stop your algorithm, tweak the code, and redeploy it - this now becomes a new, separate algorithm. It will not have your returns, history, data stored. Since you altered the code (maybe a small or a significant change) we now view it as a completely new algo. Perhaps this behavior is not ideal, the jury is still out on that one. But we're actively listening to making amends along the way.

Try to paper trade your algorithm to see the first-hand the behavior. You can do this risk-free at no cost to get more familiar with the intricacies! A couple notes to keep in mind:

• IB paper trading and Quantopian Zipline paper trading both use the same Nanex price feed
  
• Both stop the algo at 4PM and restart in the early morning in preparation for the market open
  
• Only difference: IB paper trading (and live trading) cancels any open orders after 4PM (as noted above). In Zipline, these orders persist the next day. Ideally in the future this behavior will be more closely aligned.

Forward testing is a great way to see first hand the real-market experience.

Thanks Alisa,

Slowly, I'm working to launch another Quantopian Zipline paper trading algo.

Regarding manually stopping the algorithm, ideally, data could be carried over into the effectively new, separate algorithm. Basically, this would amount to storing a file with Quantopian, and then being able to read it in upon launch. Otherwise, even a minor bug fix or algorithm tweak would result in a loss of historical data that may have taken days/weeks/months to accumulate.

Grant

It would be great as a customer of your product, to add a functionality that you can store the context in memory into an AWS S3 bucket.
Once we need to update the algo and relaunch, we could have the option to load the context from the AWS S3 bucket.
This would be ideal, because I store a lot of information of my operations and if I relaunch an algo, I will lose the strategy and operations, which would force me to sell/buy everything before I relaunch my algo.