Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Strange orders

The algo tries to close a buy order of 4959 SPY shares on 04/06/2016 by first selling 8899 shares and then buying back 3940 shares on 04/14/2016 . How can this happen?

14 responses

I still cannot figure out the cause of the strange orders. Any help from Quantopian will be appreciated.

Can you explain a little about this algo and then I will try to have a look?

Thanks. The algo buys SPY when there are two consecutive losing days and tries to get out with 2% profit or stop loss.

I have had a look and can only put it down to external factors on Quantopian's end. There is nothing I can see wrong with your code, but I could be wrong!

Thanks Max. I think Quantopian people should take a deeper look.

Hi Ricardo, what you're seeing here is happening because of partially-filled close orders. At 1:04 on 4/14/2016, your algorithm decides to close the SPY position. However, the volume during the next minute isn't very high, so the sell order is only partially filled. The backtester plans to finish the sell order during the next bar.

Meanwhile your algorithm still wants to close the position and makes another sell order at 1:05. Now there are two duplicate sell orders, one which is left over from the partially-filled order at 1:04 and one from 1:05; both are filled completely at 1:06, and the algorithm has overshot; it now has a short position. Finally it has to buy back some shares to cover and get to the target percentage of 0. Let me know if you'd like more explanation on this.

The bottom line is that order_target_percent and order_target do not take orders that are already open into account, so opening multiple of these orders can cause your algorithm to overshoot. A good rule of thumb is to always cancel all open orders for a security before placing another targeted order for it. You can do this using get_open_orders and cancel_order, documented here.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Nathan.

You wrote:

"Meanwhile your algorithm still wants to close the position and makes another sell order at 1:05. Now there are two duplicate sell orders,"

Apparently the algo (order_target_percent ) knows how much it has to sell in the first round i.e., it knows the original open position size. Why it is not updated to know how much to sell in the second round after the position size changes due to the first sale?

at 1:08 the position is flat. Why does the aglo buy 745 shares and then sells 745? That does not make any sense.

It appears to me like a deficiency in the open position updating which you then require the algo developer to handle. In any case, is there any code example for doing that? Thanks again.

Order target percentage sets an order. Since every minute you are evaluating whether to set the order or not based on if you have any positions at all you run the risk of having 50% of your original position and then making a subsequent order to sell that 50%.

If you add a boolean flag to your code that is triggered when you place a sell order and do not sell if it is triggered then this won't happen.

It makes sense that order target percent can't use open orders to set the target. In real trading the open orders information may be out of date by the time you get it

James, that does not explain this:

"at 1:08 the position is flat. Why does the algo buy 745 shares and then sells 745?"

Ricardo, here's a full rundown of what happens:

  • 1:04, long 4959: tries to close, opens order for -4959; -1764 are filled
  • 1:05, long 3195: tries to close, opens new order for -3195; -3195 are filled for 1:04 order and -3195 are filled for 1:05 order
  • 1:06, short 3195: tries to cover, opens order for 3195; 2450 are filled
  • 1:07, short 745: tries to cover, opens new order for 745; 745 are filled for 1:06 order and 745 are filled for 1:07 order
  • 1:08, long 745: tries to close, opens order for -745; -745 are filled

So the algorithm overshoots twice. That's why it buys 745 shares too many; it's trying to cover the short position it developed by overshooting the sell order and overshoots again. Then 745 shares are sold to finally close out the position.

James's explanation for why this burden (cancelling old target orders) is shifted to the algo writer is good; we can't reliably update new target orders on our end because the numbers for the status of open orders only update once every minute. It works much better to have the algorithm cancel the old target order manually before placing a new one.

Nathan, thanks for the explanation, whether I agree or not makes no difference, if you could provide the fix to the algo I would appreciate.

How does canceling work if the order is fulfill before it can be cancelled? Is there an error or a way to detect this condition? A lot of times this is all asynchronous so even the cancel order is its own order. If you cancel then place an order without waiting for the cancellation to succeed you could still overshoot.

Ricardo, for more on avoiding over-ordering, I would suggest taking a look at Lesson 10 of the Getting Started Tutorial.

James, I agree that working with orders in this way in live trading can be finicky; the best way to make sure nothing goes wrong is to either avoid placing new orders if ones already exist for the same security, or to wait a full minute after canceling the old one to see how much of it was filled.

Nathan, I used the code in Lesson 10 and it works fine. I appreciate all the help.