Problem with Backtest

Back to Community

Problem with Backtest - Bug Report

posted Oct 28, 2014

I was playing with the Sample Algorithm for a Basic Strategy just to learn my way around the Quantopian platform and Python. I encountered what appears to be a bug in order_target(). On one day, Sept 19, of the backtest, excessive fills occurred which were confirmed by a corresponding negative cash balance.

On Sept 19, order_target() was called twice. The target was 436 shares at 14:16. The target amount was changed to 401 shares at 14:18 on the log. The total fill was 803 shares on a 401 share order. The next day, a target_order() to close the position was placed. An overfill of -43 shares occurred. A buy was then apparently executed by the platform for 43 shares to compensate for the overfill. The overfill on the sell side concerns me also.

I'm a newbie here. So, if I'm confused, please show me the error of my ways. Thank you.

Please note that this code is only meant to demonstrate the bug encountered and has no algorithmic trading value.

4 responses

Seong Lee

Oct 28, 2014

Hi Robert,

What's happening here is that an order takes a few minutes to be filled (e.g. if you place an order at 3:00 PM, it'll get filled at 3:01PM at the fastest depending on your order size). So when you place an order for the second time (14:16 and 14:18), the full order hasn't been filled by 14:18 meaning that your cash value is still untouched. A quick way to prevent this from happening is to see if you have any open orders for that security:

if context.security not in get_open_orders():  
        # Calculate how many shares and place the buy order  
        number_of_shares = int(cash/current_price)  
        order_target(context.security, number_of_shares)  
        # Only record logs for problem date  
        if (event_date >= '2014-09-18') and (event_date <= '2014-09-23'):  
            log.info("%s : Order Target %i shares of %s" % (event_datetime, number_of_shares, security_symbol))

This will make sure that you aren't ordering twice when you currently have an open order for a particular security.

Thanks and let me know if you have any questions

Seong

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Rob Welsh

Oct 28, 2014

Thank you, Seong. I misinterpreted order_target(). I thought that it would automatically adjust my share amount to the target that I last presented to it and that I could update the target share amount as frequently as I wished.

Does that explain the -43 share overfill followed by the 43 share buyback on the next day? Is that also due to multiple calls of order_target()? That is the way that the sample algorithm was coded.

I suppose that I will avoid the use of order_target() to maintain better control. Thanks again.

Alisa Deychman

Oct 28, 2014

Your interpretation of order_target() is correct - it will automatically adjust your share amount to the target number. However, when it does this, it doesn't consider the status of open orders. It only looks at the filled orders and then submits an order for the difference between target and filled orders. So this can lead to a large queue of open orders as the order_target call tries to swing back and forth, overcorrecting itself. This explains the overfill followed by the buyback, because the target is trying to catch up.

To prevent this overcorrection behavior, you can add a check for open orders as Seong suggested. Hope that helps!

Disclaimer

Rob Welsh

Oct 28, 2014

Thank you for the clarification, Alisa. It makes sense to me now.

You've successfully submitted a support ticket.

Our support team will be in touch soon.