Simon,

Would it make sense to actually check the volume of that ETF before ordering? E.g. if the amount of shares that you're about to order is greater than 5% of all volume, then don't order because that's , in your definition, an illiquid security (in this case).

if numshares < data[stock].volume * 0.2:  
       order_target_percent(stock, weight)  
       cash -= notional - context.portfolio.positions[stock].amount  

Idea credit to Naoki Nagai

@Simon T.

for which there is actually far more liquidity than is apparent based the traded volume.

Are you saying that there's size available outside the NBBO? And that you could tap that if there was a different market model?

With a custom VWAP you could, what exactly? You'd still be throttled by the volume printed to the OHLC bars Q uses to fill trades? I'm confused.

I must order these ETFs, the question is how to break up my "large" order enough that I won't get killed by the slippage model.

Are you saying that there's size available outside the NBBO

ETFs generally have the liquidity of their underlying baskets. Market makers may only post 10k at the inside market, but if you hit it, they'll refresh, because all they need to ensure is that they can keep replicating the basket. They'll sell you millions of shares as long as they can keep producing new units.

With the VWAP, yes, I'd still be throttled, but I am hoping that I could mitigate Quantopian's price slippage based on volume of traded bars by reducing my "large" order into 200 share chunks or something. Even worse, if I am trying to execute multiple legs of a hedged basket, if the VolumeShare slippage is different for each leg (ie they trade in different minutes for different amounts), while I'm legging into my basket my hedges are all off and I'm exposed.

Simon.

So you'd like to implement an opportunistic iceberging OEX algo on top of your alpha algo?

And you'd like to be able to plug in this OEX algo in as a standalone code block and just call it with with a set of position size intentions. The algo would manage orders for the open vs intended position imbalance for each leg and yet be cognizant of excessive legging. This OEX would have to primarily monitor available volume but try and maintain a level exposure over your basket of components. (Am I close?)

drool yes that is what I want lol!

but I don't want to write it :)

Yeah, neither would I... Coding these order ex strats in Quantopian is like eating with chopsticks wearing ski gloves; it's gonna be messy and frustrating but hopefully you'll get fed.

Hello Simon,

Sounds interesting. When you say "**far** more liquidity than is apparent based the traded volume" are you implying that the default slippage model is not accurate relative to your real-world trading experience at IB, either paper or real-money? Or are you somehow inferring this from analyzing the minutely OHLCV bar data provided by Quantopian?

I guess the Quantopian slippage model completely falls apart if somebody has shares they want to get off their hands, but you're the only guy interested in buying them. They'd gladly sell you everything you want, at a bargain price (like showing up late in the day to a tag sale...get this junk outta here!).

Grant

Correct, but it's deeper than that. Even if one uses quote or book data to estimate liquidity, it is still not accurate for ETFs. Whereas to buy shares in a stock, someone must sell you theirs, but with ETFs, they can simply create more shares out of thin air, as long as they can locate the underlying assets.

It is partly why it's still a bit strange that bond ETFs appear to be more liquid than their underlying bonds, though that breaks down in crisis.

@Grant, not speaking for Simon, but yeah, no doubt there is size in the market but to acquire it or distribute into it you would be buying at inflated or selling at deflated prices.

So to me that speaks to a modified market model which uses more than the printed traded volume, which as Simon alludes to, maybe only representative of available quantity in the market. If 1000 shares were exchanged during the minute of 9:30.001 - 9:31.000, then most likely there were 1000 shares NOT traded, but available outside the NBBO spread If one had submitted a market order to buy 2000 shares during that minute you probably would have been filled. 1500 for sure with some high degree of confidence. But at what price? NBBO offer + the spread? Market models are just that - models. They are best guess trading environment simulators. An "extended volume market model" could be built and offered that would help Simon's case; based on the above extra-spread size available.

Simon, you're handy with helpful links, what are the rules regarding dynamic allocation of new shares? It would seem that an ETF manager could saturate the market at will and create an arb op for their friends.

Well, there's a bit more information beyond volume on Quantopian. For the bar, if O = H = L = C, then there was only one trade, which could be used as an additional factor in a model, versus an undetermined number of trades in the bar.

It seems like part of the problem involves building a statistical model that predicts the probability distribution of the volume, given a trailing set of volumes. For example, if the volume is 200 for every other bar, and 100 for the others, then there is a 50% chance that for any given bar, the volume will be 200. So, in general you'd have to build up a histogram of volumes, with more weight placed on the most recent bars. The statistics would be used to determine how many shares to buy/sell in the next bar, and you'd then just roll forward, repeating the process.

Simon, are you actually trying to get predictive results out of the backtester, or is this a (legitimate) form gaming of the contest?

Grant

I would just say that the quantopian slippage model is naively punitive for infrequently traded ETFs, and since I have a decent algo which trades some ETFs like that, I would like to optimise my execution strategy to avoid the worst of the slippage. This means dripping into hedged positions 50 or 100 shares at a time, over the course of days.

The annoying part is that my execution algo would be an order of magnitude longer that my alpha model, and it's tedious work in a language I dislike, so I'll likely not bother.

The truth is that if there's an equity-based ETF that quotes 1000x1000 and trades 5000/day, you can usually trade orders of magnitude more shares than that without moving the market at all, as long as you don't stupidly sweep the book.

Simon, I think I see your slip showing... Better tuck that puppy in.

[As an aside, here's what some fatcats are doing with ETFs in order to hedge their engorged positions... Newsletter that focuses on odd positions.]

IIUC.... Simon is saying that Q uses the "volume" number based on trades executed at a given time (down to the minute at best) for what is the available number of contracts?

If so, I agree that is indeed a "naive" way to invent prior supply for a backtest. However, unless they have the entire quote history and construct the book for all those minutes (that's a lot of work and data storage.... and you thought your VWAP stuff was heavy...) then they've really little alternative, if they want to reflect any kind of notion of supply.

I find myself agreeing that in real life, if you say, wanted 1000 contracts of X at say, 21.50, you could probably submit orders for 100 contracts on successive minutes, whenever the offer was there and get filled, even though the tape the next day might say that only 500 trades took place at 21.50. The amounts on the bid and ask come and go (cancels) without simplistically being "eaten" by trades forcing the NBBO up or down. (ok, maybe you wouldn't always get all 1000 through at that price, but you might indeed get more than the next-day reported trade volume).

There's probably no practical way for Q to implement the come-and-go of all those HFT quotes :-)

If I've misunderstood, ignore all that :-)

This means dripping into hedged positions 50 or 100 shares at a time, over the course of days.

I see. Sounds like we're talking some pretty low volume, infrequently traded ETFs (ain't no SPY). Does Interactive Brokers (IB) paper trading handle things any better? Or do they also have a naive slippage/market model?

I cooked up something for my daily trading algo. It is very basic and slices orders and executes them every minute for the next 60 minutes.

class TradeOrder:
def init(self, stock, size):
self.stock = stock
self.size = abs(size)
self.longshort = np.sign(size)
self.stepsize = self.size // 50
self.stepsize = max(self.stepsize, 10)

def execute(self):  
    if self.size < self.stepsize:  
        order(self.stock, self.size * self.longshort)  
        self.size = 0  
        return True  
    else:  
        order(self.stock, self.stepsize * self.longshort)  
        self.size -= self.stepsize  
        if self.size == 0: return True  
    return False  

class LazyExecutor:
def init(self):
self.trades = {}

def add(self, trade):  
    if trade.stock in self.trades:  
        stock = trade.stock  
        oldtrade = self.trades[stock]  
        oldsize = oldtrade.size * oldtrade.longshort  
        newsize = trade.size * trade.longshort + oldsize  
        del self.trades[stock]  
        self.trades[stock] = TradeOrder(stock, newsize)  
    else:  
        self.trades[trade.stock] = trade  

def execute(self):  
    trades = dict(self.trades)  

    for stock in trades:  
        if trades[stock].execute():  
            del self.trades[stock]