Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Order Management System - Do I need one?

So I have a contest algo that is supposed to be short only on IVV and QQQ, and be dynamically trading a portfolio
of less than 20 long stocks twice a day.

I ran extensive backtests on it, and it had no problems, and it's run with $1M live-paper-contest trading no problem.

Now, after 2 weeks live-paper-contest trading, at$10M, I have a stock that is short $2M-$5M!!...obviously a huge problem...

Debugging a running contest algo is hard...you can hack pyfolio and get information, but that is a time consuming way to go, as I found out.
I ran a backtest separately for the time period that the algo has been live, and the results don't match up...the backtest runs fine!...hmmm...

I looked at the Log file, and I had enough info in there to get an idea of what might be happening, but need some help from you to validate/set-me-straight on the ways of ordering-logic, which I'm obviously not very good at.

Essentially, the algo trades twice a day by computing a portfolio, and liquidating those stocks from the morning portfolio
in the afternoon that don't appear in the afternoon portfolio. I liquidate by using order_target_percent()...in retrospect,
I obviously don't understand the real-life semantics of any of these ordering logic functions!

So one stock, AWGN, gets bought in day K for M%. The next day, in the morning rebalance, it gets sold... I order a target of 0% (e.g -M% )...
but then in the afternoon, it gets sold again at -M%...this is a mistake, and at the end of day, I seem to own around -2*M% of AWGN, due to liquidity problems of getting rid of the stock...and...it looks like to me that the afternoon ordering transaction does not reflect the partial order fill that got done in the morning rebalance...all orders are cancelled at the end of day, so at least we get a fresh start...but now we need to get rid of twice as much of the illiquid AWGN the next day!!...and this doesn't happen. This bad liquidity cycle keeps going, day after day, and before I know it...I'm short $5M on one stock!...disaster!!

Yes, I can fix this problem in lots of ways, and yes, I'm doing something stupid, but I'd like your help with pointing me towards
practices, documentation and code that'll make it safer to algo trade without disaster.

For example, is there code to do what the HFT people do and test the waters on what liquidity there is in the order book by issuing fill-or-kill orders in a probing kinda way?

Any help or comments appreciated.
alan

2 responses

There are several ways, which one you will use depends on the specifics of your problem. Generally you don't want to open any new buy orders while you still have a pending sell order because if the sell order fills you could use the money for your buy order. You also don't want to open any order for a security that another order already exists for. With proper order management you can perfectly control your leverage (pending liquidity)

1) Check for any open orders before opening a new order

if get_open_orders(): return  
for stock in list:  
    #ordering logic  

2) check for open orders on a security before attempting to sell

for stock in list:  
    if get_open_orders(stock): continue  
    #sell logic  

3) Canceling all orders at the start of rebalancing (this can be modified for canceling only sell orders or only buy orders)

def cancel_open_orders(context, data):  
    oo = get_open_orders()  
    if len(oo) == 0:  
        return  
    for stock, orders in oo.iteritems():  
        for order in orders:  
            #message = 'Canceling order of {amount} shares in {stock}'  
            #log.info(message.format(amount=order.amount, stock=stock))  
            cancel_order(order)  

4) cancel any existing order for a security before you attempt to open a new order on it.

def cancel_open_order(sec):  
    oo = get_open_orders(sec)  
    if len(oo) == 0:  
        return  
    for order in oo:  
        if order.sid == sec:  
            cancel_order(order)  

Luke,
Thanks for responding to my request positively!
While I've used the functions you've listed in certain situations,
it hasn't been clear to me why, when and in what order I need them, but will now think about it more.

I'm thinking of creating a user's portfolio and order tracking class, with an eye towards tracking, monitoring, and restricting certain actions
perhaps on a minutely basis:
- liquidity effects on portfolio assets (buy/sell/filled/slippage and commission ),
- portfolio rebalancing constraints and measurements (resolve ordering conflicts with respect to a golden target portfolio)
-internal/external split/delisting functions (i.e. neuter out assets that'll be delisted in the future, or validate a split against some external data source.)

This is probably too much to bite off, but something simple that applies a discipline across algorithms would be useful for me.
I guess that I need this class to get a grip on how a portfolio actually gets rebalanced, through time, given a target, and how to self-correct "mistakes" in
the asynchronous nature of the transactions.

Been looking at:
https://github.com/quantopian/zipline/tree/master/zipline/finance
https://www.quantopian.com/posts/helper-functions-getting-started-on-quantopian

Again, thanks for your help!
alan