Grant,

Stop-limit orders can definitely be a little confusing at first glance. What's happening here is that your algorithm does actually place orders on the 25th for a 100 shares of AAPL (2 separate orders). The order object is indicating that the stop was never reached when it actually was because in this case, the stop-limit order causes the order to immediately be converted into a limit order when the stop price is reached(this also means that the 'stop_reached' boolean will always be False but the 'limit_reached' will change according to the appropriate conditions). And because in backtesting, orders aren't cancelled at the end-of-day, the now converted limit order is carrying through day-over-day; hence the orders executing on the 25th, a few days after you first placed the order.

Basically, this means that your first two orders have automatically been converted into limit orders with a limit price of $100.00 and that's why the 'stop_reached' boolean is always False. In the backtest that I've attached we can see that on the 25th, the first two orders placed on the 22nd and 23rd have reached their respective limit prices and so the orders have been filled on that day.

Also, because orders are filled on the next bar, the current order (and the stop condition, given this was a regular stop order) won't be filled until the next day so you could, if you wanted, store the id in 'context' like

def initialize(context):  
    context.past_id = None

def handle_data(context, data):  
    if context.past_id:  
           print get_order(context.past_id)  
    order_id = order(sid(24), 50, style=StopLimitOrder(limit_price=100, stop_price=99.99))  
    if get_open_orders():  
        context.past_id = order_id  

Hi Seong,

I guess I don't yet understand completely. Are you using OHLC values to determine if the stop is reached? I can't understand why the limit order resulted when the closing price stayed above the stop price of 99.99?

Also, what is the point of the 'stop_reached' flag if it is not used? Shouldn't you be retaining the stop price and setting the stop_reached flag to True?

I've attached an update to my backtest.

Grant

The log output:

2014-09-22PRINT101.07  
2014-09-22PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': 99.99, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 100.0, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': '9bc9b28c2d9647518168ca4e9aa8da43', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
2014-09-23PRINT102.64  
2014-09-23PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 100.0, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': '9bc9b28c2d9647518168ca4e9aa8da43', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
2014-09-24PRINT101.74  
2014-09-24PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 100.0, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': '9bc9b28c2d9647518168ca4e9aa8da43', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
2014-09-25PRINT97.87  
2014-09-25PRINTEvent({'status': 1, 'limit_reached': True, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': 1.5, 'amount': 50, 'limit': 100.0, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': '9bc9b28c2d9647518168ca4e9aa8da43', 'dt': datetime.datetime(2014, 9, 25, 0, 0, tzinfo=), 'filled': 50})  
2014-09-26PRINT100.73  
2014-09-26PRINTEvent({'status': 1, 'limit_reached': True, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': 1.5, 'amount': 50, 'limit': 100.0, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': '9bc9b28c2d9647518168ca4e9aa8da43', 'dt': datetime.datetime(2014, 9, 25, 0, 0, tzinfo=), 'filled': 50})  
End of logs.  

Seong,

Here's a variant--still confusing. The stop is reached, but then there is no buy at the limit. It appears that only closing prices are used, correct?

Grant

2014-09-22PRINTOHLC  
2014-09-22PRINT101.8  
2014-09-22PRINT102.14  
2014-09-22PRINT100.58  
2014-09-22PRINT101.07  
2014-09-22PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': 101.0, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 97.72, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': 'b7eca32b0bba43d4b3baf9c2807a6668', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
2014-09-23PRINTOHLC  
2014-09-23PRINT100.62  
2014-09-23PRINT102.9474  
2014-09-23PRINT100.54  
2014-09-23PRINT102.64  
2014-09-23PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 97.72, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': 'b7eca32b0bba43d4b3baf9c2807a6668', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
2014-09-24PRINTOHLC  
2014-09-24PRINT102.16  
2014-09-24PRINT102.85  
2014-09-24PRINT101.2  
2014-09-24PRINT101.74  
2014-09-24PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 97.72, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': 'b7eca32b0bba43d4b3baf9c2807a6668', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
2014-09-25PRINTOHLC  
2014-09-25PRINT100.51  
2014-09-25PRINT100.71  
2014-09-25PRINT97.72  
2014-09-25PRINT97.87  
2014-09-25PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 97.72, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': 'b7eca32b0bba43d4b3baf9c2807a6668', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
2014-09-26PRINTOHLC  
2014-09-26PRINT98.53  
2014-09-26PRINT100.74  
2014-09-26PRINT98.4  
2014-09-26PRINT100.73  
2014-09-26PRINTEvent({'status': 0, 'limit_reached': False, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': None, 'amount': 50, 'limit': 97.72, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': 'b7eca32b0bba43d4b3baf9c2807a6668', 'dt': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'filled': 0})  
End of logs.  

Grant,

Great questions! To answer your first one: Yes, the close_price is being used to calculate a security has reached its stop/limit price.
As for the rest of the questions, I'll try to clear up everything but I'm sure this is going to be an ongoing discussion so feel free to keep asking more.

I can't understand why the limit order resulted when the closing price stayed above the stop price of 99.99?  

I can totally see why you're confused. At first glance it seems like the stop and limit prices should create upper and lower limits of sorts, but when you look at how stop-limit orders really work you see that that's not the case. Stop-limit orders behave like this: let's say a stock is at $40.00 and you set the stop price to be $45.00 and the limit price to be $50.00. Once the stock crosses the $45.00 stop price, it's converted into a limit order. This means that as long as the stock's price is below $50.00, the limit order is good to go. So it's in two steps, each independent of each other where the limit order doesn't know about the stop order of $45.00; it only knows that as long as the stock is below $50.00, the limit order can be executed. This is why the trades were executed on the 25th since 97.87 is well below the limit price, and the stop price ($99) was crossed on multiple days (101, 102, etc.,). And the logs show that the only orders that were executed were orders that were placed on the 25th, not the 26th (both orders have the same id and the creation date is the 25th).

Ideally in the above example, the stock's price would remain above 45 and below 50, giving you the butterfly bounds that you originally hoped for, but in some cases that doesn't happen.

2014-09-25PRINT97.87  
2014-09-25PRINTEvent({'status': 1, 'limit_reached': True, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': 1.5, 'amount': 50, 'limit': 100.0, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': '9bc9b28c2d9647518168ca4e9aa8da43', 'dt': datetime.datetime(2014, 9, 25, 0, 0, tzinfo=), 'filled': 50})  
2014-09-26PRINT100.73  
2014-09-26PRINTEvent({'status': 1, 'limit_reached': True, 'created': Timestamp('2014-09-22 00:00:00+0000', tz='UTC'), 'stop': None, 'reason': None, 'stop_reached': False, 'commission': 1.5, 'amount': 50, 'limit': 100.0, 'sid': Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 9, 30, 0, 0, tzinfo=), first_traded=None), 'id': '9bc9b28c2d9647518168ca4e9aa8da43', 'dt': datetime.datetime(2014, 9, 25, 0, 0, tzinfo=), 'filled': 50})  
End of logs.  

And to answer this question:
Also, what is the point of the 'stop_reached' flag if it is not used? Shouldn't you be retaining the stop price and setting the stop_reached flag to True? Because the behavior is to convert the stop-limit order directly into a limit_order, the backtester treats the order as a strict limit order and not a hybrid order. This is why the 'stop' quickly becomes 'None' in many of the logs.

This is definitely an ongoing discussion and the backtester's behavior is very different from the live tester's in this case because all orders in live testing are cancelled at the end of the day, reducing the probability that something like this could happen. And because the ticks of each bar are day over day, the big fluctuations in the price might have led to this as well.

I hope I've answered your questions, but if not, feel free to keep on asking more!

Seong

Thanks Seong,

I found the relevant zipline code:

https://github.com/quantopian/zipline/blob/master/zipline/finance/blotter.py
https://github.com/quantopian/zipline/blob/master/zipline/finance/slippage.py

The logic for the stop-limit order is:

if order_type == BUY | STOP | LIMIT:  
    if event.price >= order.stop:  
        sl_stop_reached = True  
        if event.price <= order.limit:  
            limit_reached = True  
elif order_type == SELL | STOP | LIMIT:  
    if event.price <= order.stop:  
        sl_stop_reached = True  
        if event.price >= order.limit:  
            limit_reached = True  

I also sorted out that there are two separate boolean flags, stop_reached and sl_stop_reached. As best I can tell, stop_reached applies to stop orders, while sl_stop_reached is used for stop-limit orders. A call to get_order(order_id) only shows stop_reached; sl_stop_reached is not accessible.

The code that converts the stop order into a limit order is:

if sl_stop_reached:  
# Change the STOP LIMIT order into a LIMIT order  
    self.stop = None  

The problem I see is that the stop-limit order literally converts into a limit order, with no record of the stop value and no indication that the stop was reached. My sense is that rather than converting to a limit order, the stop-limit order should remain (with a stop value and stop_reached = True), and just become executable as a limit order.

Grant

Grant,

That's awesome you found the code! 'sl_stop_reached' and 'stop_reached' are independent in order to preserve distinct 'stop' and 'limit' orders but you bring up a good point; I'm looking IB's description of the stop-limit order and they seem to convert the stop-limit order directly into a limit order but it doesn't specify whether it executes within the boundaries of both the stop and limit order instead of just the limit order. We'll definitely have to dig a little deeper into it. Thank you for bringing it up!

Seong

Song,

The logic in the zipline code I posted above seems correct (see http://financial-dictionary.thefreedictionary.com/Stop-Limit+Order, for example).

Grant

Grant,

Yeah, we double checked with IB to make sure and everything checks out... and it does!! Good to go!

Seong

Updated track_orders for StopLimit orders. It can help turn the lights on.

2016-01-14 06:31 _orders:65 INFO    1   Sell -1 SPY now 189.67 stop 189.63 limit 191.19 cash 1000  2c31  
2016-01-14 07:36 _orders:65 INFO   66      Sold -1 SPY limit at 191.24   cash 1190  2c31  
2016-01-15 06:31 _orders:65 INFO    1   Sell -1 SPY now 186.80 stop 186.75 limit 188.29 cash 1190  ea33  
2016-01-15 13:00 WARN Your order for -1 shares of SPY failed to fill by the end of day and was canceled.  
2016-01-19 06:31 _orders:65 INFO    1    Canceled Sell -1 SPY limit at 189.95  
2016-01-19 06:31 _orders:65 INFO    1   Sell -1 SPY now 189.95 stop 189.91 limit 191.47 cash 1190  3781  
2016-01-19 13:00 WARN Your order for -1 shares of SPY failed to fill by the end of day and was canceled.  
2016-01-20 06:31 _orders:65 INFO    1    Canceled Sell -1 SPY limit at 185.13  
2016-01-20 06:31 _orders:65 INFO    1   Sell -1 SPY now 185.13 stop 185.09 limit 186.62 cash 1190  ed1b  
2016-01-20 12:21 _orders:65 INFO  351      Sold -1 SPY limit at 187.00   cash 1376  ed1b  
2016-01-21 06:31 _orders:65 INFO    1   Sell -1 SPY now 185.82 stop 185.78 limit 187.31 cash 1376  c6dd  
2016-01-21 07:21 _orders:65 INFO   51      Sold -1 SPY limit at 187.32   cash 1562  c6dd  

causes the order to immediately be converted into a limit order when the stop price is reached

Rather than a new order though, everything remains the same (order id, dt, status etc) except stop price is removed, the only change.

In the code I imagine this is happening:

if stop price:  
    if stop price met:             # Treat it as a stop order  
        Set stop price to None                <=== whoa  
elif limit price and limit_reached is False:  
    Check whether price met etc    # Treat it as a limit order  
else:  
    # Treat it as a market order  

Instead could be:

if stop price and stop_reached is False:        <== added condition  
    if stop price met:             # Treat it as a stop order  
        Set stop_reached to True                <=== yes  
elif limit price and limit_reached is False:  
    Check whether price met etc    # Treat it as a limit order  
else:  
    # Treat it as a market order  

That's the only change needed to make the behavior consistent and user could maybe use stop_reached for decisions, as long as the accurate stop_reached set to True would be ok with IB. Right now when limit price is reached, limit_reached is set to True while stop_reached remains False when stop price is met.

I know this is an old thread, but I'm confused by the thread itself. What I would like to do is trail a price on the way up, and if it reverses, get the stop price or better. Based on what I'm reading, that wouldn't happen. If I have stock at $100 and it goes up say, to 106. I want to place the stop order below the current price of 106, and the and the limit order to be triggered on reversal for 105. Would the function below accomplish that, or not?

             def trail_stop(context,data,*limit):  
                if context.trail_counter < TRAIL_INTERVAL:  
                    context.trail_counter += 1  
                    return  
                else:  
                  context.trail_counter = 0  
                  for s in context.portfolio.positions:  
                   if s not in context.closed:  
                     cp = data.current(s,'price')  
                     cb = context.portfolio.positions[s].cost_basis  
                     amt = context.portfolio.positions[s].amount

                     if cp > (cb * TRAIL_STOP):  
                         if (cp * TRAIL_STOP) > context.trail[s]:  
                             context.trail[s] = cp  
                             if limit == True:  
                              tc(sl,[s,0,cp,cp,True],["SL - Trail Price Moved => " + s.symbol + " => " + str(cp),'StopLimit Error'])  
                             else:  
                               tc(so,[s,0,cp,True],["SO - Trail Price Moved => " + s.symbol + " => " + str(cp),'StopOrder Error'])

            def sl(stock,amount,priceA,priceB,*cancel):  
                    co(stock,cancel)  
                    order_target(stock,amount,style=StopLimitOrder(priceA,priceB))

            def co(stock,*cancel):  
                if cancel == True:  
                 for s in get_open_orders(stock):  
                    cancel_order(s)  
                    print s

            # TRY CATCH WAPPER - ALLOWS FOR MESSAGES ON SUCCESS AND FAILURE  
            def tc(fn,prms,msg):  
                try:  
                    fn(*prms)  
                    print prms[0].symbol + ' => ' +  msg[0]  
                except:  
                    print prms[0].symbol + ' => ' + msg[1]  

Anyone?

One use for StopLimit (for long): Sell if price falls to the stop but only if it doesn't drop even further below the limit set even lower.
Imagine a LimitStop order that says: If price rises 9% then don't sell yet, let it skyrocket, but set a stop now, at 8% to sell if it drops that far.

What you're referring to, Blue, is called a 'trailing stop'. Trailing stops can be percentage or dollar-amount based. These are a common tool among intermediate-advanced traders, and are available on most of the major (ie Etrade, Thinkorswim, etc) trading platforms.

I don't believe I've actually seen one successfully implemented on Q yet, because the logic behind it... ie "if ABC stock starts to fall, place a stop-limit order X% below it" ...would result in soooo many orders being placed (and subsequently having to be retracted/cancelled as the stock regained its footing) that the algo would presumably eventually go haywire and fail.

Canceling and resetting stops can be done, just lots of work, Quantopian's ordering system is built well enough to take it. By LimitStop I meant just a single order and I think the trailing stop you refer moves automatically, wouldn't that be sweet.

I hope it is (robust enough).. have only done paper-trading so far on Q, no live-trading.

Upon rethinking, though, the algo could just as easily track the price minute-by-minute and then place a market order once said threshold (X%) is reached. It may not sell at precisely X%, but it would be close enough (assuming you set the threshold at, say, 6.8% when you're really aiming for ~7%).