Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Quantopian 2 partial fills

When running my algo I noticed these:

2003-08-01 WARN Your order for 11027 shares of SHY has been partially filled. 6870 shares were successfully purchased. 4157 shares were not filled by the end of day and were canceled.

This algo balances monthly so I want to keep the order open until it's filled. Is there any recommended way to keep orders open until they are filled or is it assumed that every user will implement reopening functionality themselves?

10 responses

My first reaction is that I think that this should signal that there is something else worrisome going on in your algorithm. You're either trying to get a big chunk of the volume over the whole day, or you're trying to fill a large order with too few minutes left in the day. If you have a market order being canceled at the end of the day, it means something odd is going on.

Other that that, I don't have a specific best practice to recommend in this case. I can imagine saving a "target portfolio" on your rebalance date, and then checking that was obtained successfully. Or, I can also imagine saving the order_ids of each order, and then the next day using get_order() and verifying that the order amount matches the fill amount, and then re-ordering any discrepancies. But I really would focus on my first point: a cancelled market order signals that you're probably trying to buy too much of something that is too illiquid.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Dan

Same questions I rised yesterday 3 times:

https://www.quantopian.com/posts/take-quantopian-2-for-a-test-drive#5713506a38c23bf1ea000276
https://www.quantopian.com/posts/take-quantopian-2-for-a-test-drive#571674c85e96cf575a0002aa
https://www.quantopian.com/posts/take-quantopian-2-for-a-test-drive#571674c85e96cf575a0002aa

and never got the answer were from came that 2.5% of bar volume limitation in default slippage model for market orders?
I also strongly support others requests for Good Till Date and Good Till Cancel order option.

Hello Dan,

I actually know the reason for the trades at my example were not filled, it's because of the long backtest with ETF that was just opened near those dates so there is not enough volume. However my question was about common framework, there should be some common way to guarantee execution when volume is there or you will end everyone coding their own implementation (some will be more buggy that others) of the same issue which is huge waste of time (I actually have a functioning execution framework myself but it's a PITA to copy it from algo to algo and update all changes by hand to every algo).

Monitoring execution is of course important too and you are quite right that at real world this algo would be problematic as it's trading quite large sums on unliquid asset.

Maybe the question should be: Should there be some kind of common extra order layer in top of raw execution or possibility to include user created libraries? OCOs would be quite nice, icebergs and other slightly more sophisticated order methods would also be quite nice in real world execution. As you don't allow libraries to be used you will end up with people copying same ordering code from algo to algo and that will lead to huge version control mess with every algo and tens-hundreds of thousands of lines of duplicate, deprecated code that could be in some common libary (not necessairly from Quantopian if you allow using external libaries too).

I have a liquidity filter (at least 10m shares traded per day) and trade at market open, and I still get partial fills by market close. My algorithm seems to perform the same, so I've not looked into it too much. I wonder if the moving average volume overestimated liquidity for some stocks?

I had the same problem with my algo. The initial version of it had only liquidity filter. However, after revising the logs I have found out that major part of the partially filled and cancelled orders were related to penny stocks. After introducing an additional filter in the pipeline the problem seemed to be solved

Mikko - the ability to have reusable code is a highly requested feature. It makes a lot of sense - it's something that we want to build in the future.

Vladimir - In our modelling and experience, the new default slippage is a good one. It's not perfect, of course. Slippage is going to vary by security, time of day, market conditions, a whole bunch of things. We aren't claiming to have solved that problem; sometimes, the default model isn't the right model. At those times, one can take advantage of the slippage API and make what ever adjustments are appropriate.

Nikolay - That sounds great. That's exactly the type of improvement that we hope comes out of this change.

@Dan: I would like to suggest that a new parameter be placed in the standard "set_slippage(slippage.VolumeShareSlippage" code: Min Size Round Lot=YES.

  • If we can't fill 100 shares at the assumed slippage, the slippage is wrong.

You show great courage with respect of product cycle updates. Thank you!

I face the same problem being unable to exit some stocks EOD.

My algo now cancels all orders 2 mins before market close and loops context.portfolio.positions for existing positions. Stocks with existing positions are stored in a dictionary and set to sell at tomorrow open.

The not-so-easy problem is, for Q's contest, I believe we must end the day in all cash, or hold at least 1 long/short position?

My intraday algo fails to exit some stocks on a few days during the 2 year backtest. So the workaround again, is to go long/short ONE stock of SPY, 1 min before close if some positions are not closed. It's quite troublesome.

John Jay - Thanks for the suggestion. Definitely doable - also doable in a custom slippage model if you are interested in that route yourself.

Kayden - It isn't necessary to be hedged 100.0% of the days. If you're hedged all but 2 days, you still get the hedged badge in the contest.

@Dan: Thanks for the clarification, that's very good to know.