Hello David,

I took a look and you're correct - the limit order execution isn't quite right. Thank you for pointing that out! We always appreciate the feedback. Here's a short description of what we've found:

• When we check the order triggers for stop and limit orders, we use the data bar's current close price.
  
• If the bar price triggers the order, the order goes into the execution engine.
  
• The execution engine then applies the slippage model.
  
• The order is filled using the slippage-adjusted price.

We need to take the slippage model out of it for limit orders - that will be a better model.

We're also thinking about modifying the slippage model instead of removing it in these cases. What do you think about looking at the high and low prices of the bar, and then filling the order depending on availability? For instance, imagine you have a limit order to buy 100 shares if the price dips to 10.00. In a one-minute bar, the high is 10.05, the low is 9.95, and the volume is 100. That order shouldn't completely fill - it should only fill for the parts of the bar that are under 10. Do you think we should head in that direction?

Thank you again for bringing it to our attention.

Presumably creating a generic model for lightly quoted high volume instruments or highly liquid low volume instruments could prove challenging--different instruments are will have rather different behaviors wrt to filling a resting order despite possibly having similar volume and liquidity averages over a bar.

Either way what you are describing sounds reasonable. I would expect the possibility of a full, partial or no fill from your backtest engine if the market trades at/through the resting limit order price depending on factors such as average volume and average liquidity over the bar and would always expect to get filled at the specified limit price or better.

Thanks.

Thank you for the feedback! We will work on providing the fix, and possible improvements, to the limit order execution.

How do limit orders function currently in the live beta?
Thanks.

Hello Alisa & David,

As Rich Frank helped clarify in https://www.quantopian.com/posts/usage-and-syntax-of-stop-orders, under live trading with Interactive Brokers (IB), all orders (regardless of type) are sent immediately to IB as they are encountered in the algorithm, with IB managing any limit triggers (although I haven't yet seen a detailed mapping between Quantopian order types and the IB order types that will result). The implication is that the order fulfillment engine for the backtester needs to be modified to better model live trading. Currently, for a market order submission during bar 0, the order will be filled at bar 1, using the closing price of bar 0 (assuming no slippage). However, a better model for live trading might be to fill closer to the open price of bar 0 (although this is still dicey, since under live trading, the order could be submitted up to ~ 50 seconds into the bar). Additionally, as Alisa suggests, using the high/low levels for the bar might be a better way to determine the trigger levels for limit, stop & stop-limit orders, combined with an appropriate slippage model.

A note to Quantopian is that the validity of the backtester model can be benchmarked using paper trading at IB, so whatever route you go, do a study (or hire someone to do a study) and publish the results publicly. Also, it seems that IB would be the best consultant on how to modify the backtester to match live trading. Have you talked with them?

Grant

Has there been any work on the limit order model? I placed a short vwap order in the "Quantopian Open" that was filled 70 bps below it's limit.

Bump. Still a problem? Re: the limit order partial fill suggestion from two years ago, if the low of the bar was even one cent below the limit price, the limit order should be completely filled. Markets are generally not allowed to trade through a price without clearing all resting liquidity at that price. E.g. If my limit bid is at 10, I must be completely filled before the inside ask price can drop to 10.

Bump. It seems like this is still a problem. Is this being worked on? My backtesting is essentially useless! My algorithm relies on limit orders being executed properly.

Bump. This MAJOR bug needs to be fixed, my algo is also using limit orders.

This bug makes my algo noticeably more profitable (in back tests). Live trading seems to be always necessary to verify behavior as well as results. Please fix this bug.

Assuming that any limit order would be completely filled if price traded a tick beyond it is not a valid assumption. In real life, if you have a decent sized bid sitting there visible on the book, the HFTs will happily provide liquidity a penny above you.

In practice, using limit orders, all of your losing trades will get complete fills, but some of your winners (ones where your limit price was near the low of the day) will get zero or partial fills. This is why I prefer to use other order types such as "market if touched" or "limit if touched" with extra room.

These order types can be backtested in Quantopian by checking the low (buys) or high (sells) of the last minute and simulating a normal market order with slippage assumptions. A minute can be a long time, but the deviations average out over a number of trades.

Was there any work done on this? I noticed that limit orders are still only triggered according to the close price of the following bar, so are market orders, which leads to inaccurate filled prices, and cases in which limit orders aren't filled at all (although according to the values they should've).
This occurs especially on daily frequency, I presume it's because the probability of a higher difference in price as oppose to minute frequency)
Thanks!

If this could be addressed, maybe Q could try allowing limit orders in the contest but it's complex.
All of the limit orders above are surely referring to limit on existing positions to sell for profit if the price goes up to that limit.
There's also limit to buy or buy more.
Stops can also be on both buy and sell, then there's the third type, StopLimit, and partial fills and different slippage models, and backtesting vs paper trading (15 minutes delayed) vs real money.
Coping with stop & limit is complex.

Is there any work done to fix this very simple but MAJOR BUG?

Can you elaborate maybe with an example.
By the way I have a bit of new info, it looks like limits are triggered on previous minute close (rather than high or low in the case of buy).

There are two issues:

• Limits are not filled when price is hit which causes incorrect fills especially in tight siutuations like whipsaws
  
• Slippage is also applied to limit orders - this is kind of mind blowing stuff, you can get filled at the wrong side of the limit because of the slippage.
  

For some traction, attach a minimal backtest that demonstrates a point where a limit was expected and not triggered (or visa-versa), something like this to show some minutes of OHLCV and price history could be useful.

log.info('{} history\n{}'.format(  
    stock.symbol,  
    data.history(stock, ['open', 'high', 'low', 'close', 'price', 'volume'], 5, '1m')  
}

A backtest provides concrete content for discussion. Then if it reveals an issue and a change made to address that, it can also be used to verify.

Blue seahawk - have you even read the thread you are replying to?

Please read the second post in this thread from quantopian posted in 2014.

I've been analyzing limits in actual code last couple months and comparing to another site.
The point--and it's a good one: "A backtest provides concrete content for discussion."
On the other hand, writing a backtest to demonstrate a problem isn't easy. I too have questions by the way.

An update and maybe clarification and direction when using limit orders. First off, there hasn't been any changes to how limit orders are filled since the original 2014 post. Specifically, the steps that zipline follows are:

1. Check if an order is triggered. The data bar's close price is compared to the limit price.
   
2. If the order is triggered the order goes into the execution engine.
   
3. The execution engine applies the slippage model to determine the filled shares and price.
   
4. The order is filled using the slippage-adjusted price.

One issue is the current default slippage model used in step 3. (see https://www.quantopian.com/posts/changes-coming-to-the-default-slippage-model). It's FixedBasisPointsSlippage (see https://www.quantopian.com/help#ide-slippage). This always 'slips' the buy/sell price by a fixed amount which, in the case of limit orders, may be over/under the limit price. This default slippage model is meant to be used in conjunction with the contest which requires the 'order_optimal_portfolio' method for ordering. It isn't so much a 'bug' as much as the wrong application of this slippage model when using limit orders. If one isn't developing for the contest and doesn't want to use the standard 'order_optimal_portfolio' method for ordering then the default slippage may not be a good choice.

That's the update and clarification part of this post. Now for some direction.

The default FixedBasisPointsSlippage can be used for limit orders but the results will be negatively skewed (since orders may fill worse than the limit price). A better approximation when using limit orders may be to use VolumeShareSlippage.

set_slippage(slippage.VolumeShareSlippage(volume_limit=.01, price_impact=0.0)

The key is 'price_impact' = 0.0' (volume_limit' can be whatever seems appropriate). By setting this to zero, the fill price will be the close of the previous bar. Because the order was triggered, this will always be equal or better than the limit price but never worse than the limit price. Note however this is optimistic (since one will often get a better price) but at least the limit price will be respected. Getting a better price actually does happen in live trading when a limit order is placed and it's a 'marketable order'. A marketable limit order is a buy order with a price at or above the lowest offer in the market or a sell order with a price at or below the highest bid in the market. Marketable orders are submitted as a 'market order' by the broker and therefore fill at market which may be better than the limit (more on the bid and ask issues later). So, VolumeShareSlippage can be used for market orders but it will optimistically skew results

Probably a better slippage choice would be to write a custom slippage method. Return the fill price to be the limit price. This would be slightly negatively skewed (because some orders would fill better than the limit) but not crazy and most of all not overly optimistic. The added benefit of a custom method comes if one uses a combination of limit orders and market orders. It get's passed the order which can be inspected and, based upon the order type, different slippage could be applied to each order type.

That's the direction. Use the default slippage to get a pessimistic model then switch to VolumeShareSlippage and get a more optimistic version. More than likely reality may be somewhere in between. To reduce that margin maybe write a custom slippage method to replace the default and reduce the pessimistic side of the model. The actual may be something in between the two.

Now a few added thoughts.

Remember that a backtest isn't reality. It's important to look how sensitive performance is to the slippage model. If an algo swings from bad to good based upon a change in slippage would one really want to risk real money?

@Mikko M hope this helps. If there are some specific anomalies you are seeing with orders and have questions please post them. The devil is always in the details. As @Blue mentioned it's helpful to have some specifics when looking for them.

Thank you Dan,

I have actually been using the volumeshareslippage mentioned but some algos return crazy looking results (22+ sharpe etc) which I presume are due to incorrect fills.

I think then the next step will be to model a custom slippage model then as I like the Q environment (large scale precise stock data is expensive!) - too bad you removed the live trading possibility!

ps. if anyone else is interested in working a slippage model that or is aware of already implemented model that is semi-realistic preferably a bit pessimistic then please post a link or message.. I'll probably open a seprate thread later for the slippage model implementation..

edit: Here is one implementation.. I totally forgot this thread :-I https://www.quantopian.com/posts/simulation-of-non-marketable-limit-orders

I would lean towards the pessimistic model for a number of reasons.

Since Quantopian uses only 1-minute granularity on the price data, it can be ambiguous whether the order was marketable or not when it was submitted. In those cases it's safer to assume it was not.

Also, Quantopian may give very generous price improvement on non-marketable limit orders, which is obviously unrealistic. To illustrate lets say the price is currently at $5.10 and you place a limit buy order for $5. Now lets say 20 minutes later the price dips down to $4.50 within the span of a 1-minute bar. As I understand it volumeshareslippage will give you a fill at the $4.50 price (the close price of the minute bar), even though in reality you would have gotten filled at $5.

Quantopian doesn't have any data on quotes or spreads, so you have no way to know whether the historical fills were filled at the bid or the ask. As an example, if you have an odd lot limit buy order, there's a chance that in the real market you would only get filled once the ask hits your limit price while round lot bids may be getting filled at better than your price while you sit there not getting a fill, because of how orders are matched.

HFT pray upon limit orders. They employ methods such as "sub-pennying" to basically cut in line to get ahead of you in the queue and will only give you the fill once they've determined that the price is moving against them/you. They take on nearly zero risk and you only get your fill once it's a losing prospect. To be safe, assume you're always last in the queue -- meaning your order for $5.00 won't get filled until price hits $4.99.

To assume zero price impact will be in many cases unrealistic as well.

These are all things to take into account if you want to develop a custom slippage model to more accurately model limit order fills.

I totally agree and I think the slippage model should be as pessimistic as possible (but still obey the market rules) - probably it's wisest to always fill at the limit price - it's better to get a little bit better results in live trading than to only then realize that the algo was a loser all along.

This thread is the reason (a real "anomaly") why I returned to this issue as the results seemed too good to be true: https://www.quantopian.com/posts/long-only-non-day-trading-algorithm-for-live#5ccab7fcee5166004005ee2b