The is the greatest post in a long time!

I know you can create your own "slippage" model in Zipline and pass this to the backtester. However would it be possible to use these slippage models in the optimizer?

I'm asking because the total transaction costs with some of the models listed in the lecture are concave. If you are using the convex solver in your optimization, it will not proceed. How would you handle that?

Interested too. I started to play around with the optimize API turnover constraint. Part of the game it would seem is to predict how much turnover makes sense on a daily basis dynamically.

Generally, when I watched the video, it was not clear where y’all are headed with this stuff? Will it be incorporated at the algo level or was it more for the Q fund, post alpha combination? I’d think that your prime broker would provide a service? What’s the game plan?

Hoping the platform will support multiple fill options via order_optimal_portfolio in the future. Appears to be a disadvantage if you can only support market orders at 5 basis points slippage when competing in the marketplace against other hedge fund algorithms that probably have 50+ fill options to lower slippage and transaction costs.

If the idea for the Q fund is to do alpha combination on a set of algos and then do another round of optimization and order management it is not obvious that anything else is needed at the individual algo level. In fact one could just skip the ordering altogether and just output the alpha vector from each algo to be combined with the whole lot of algos. In the context of the Q fund it would seem the value is in the individual alphas, not tasking users to figure out how to manage orders.

From this Case Study, I am inclined to infer that portfolio optimisation and order management are entirely the authors' :

Quantopian’s research team selects strategies to trade in the firm’s 1337 Street Fund.

“The fund’s mandate is an absolute return, pure alpha strategy focused only on
US equities.. The goal is to evaluate all the algorithms our users write and invest
in the ones that best fit our investment mandate.”

“The algorithms are set up as standalone strategies in Geneva ..and each
calculates its own P&L. That allows us to take the trading P&L for the portfolio
as a whole and split it up by author, so that we can easily track each author’s royalty.
The strategy level reporting in Geneva is also useful.. It allows us to give each author
a performance snapshot for their algorithm at the end of the month.”

Thanks Karl. Yes if algos are standalone then it is a different story.

Pipeline is based on yesterday's close price, the alpha vector that we pass to optimize from the before_trading_start pipeline analysis doesn't really make any requirement that the order be filled at market right away at the exorbitant slippage that they are incurring and modeling it at. Why waste dollars on slippage costs, when that is neither the requirement of every algorithm developer nor what one might consider the optimal solution for long short equity. It becomes a lot more easier for themselves and the user base if they expose in order optimal low slippage options (1-2 basis point slippage). Maybe lowering the fill per minute and spreading the order out over a longer time horizon. Ideally one would leave the fill methodology to the algorithm developer, he knows what best suits his strategy. Is he trading with the expectation that the fill happen rightawy at market or is he trading over a longer time frame where lower slippage is preferred.

Thank you for your comments and questions about the market impact model lecture. To address questions around adding slippage to the optimization model : the purpose of building a model that simulates market impact is to understand transaction costs on a per strategy basis. To get to a strategy level cost estimate we simulate market impact at the order level. At an order level we are asking : how might we be moving the market during the life of the order? and what are the spreads we are paying? how do the order attributes (e.g. time we trade) impact the spread? In its current state, our market impact model can be used to simulate transaction costs in a backtest or compare against live trading results. There are aspects of the market impact model that would be interesting to add to a pre-trade optimization such as the spread modelling. In general, there is uncertainty in pre trade transaction cost estimation on any single order. I have observed that in aggregate, over longer time periods, the market impact model converges to an estimate with less uncertainty. In live trading, execution decisions are based on many of the factors discussed in the lecture such as relative order size, liquidity, and urgency (information about intra-day or "short term alpha"). Users should focus on developing alpha vectors while taking into consideration the concepts introduced in the market impact model lecture.

Hi Michele,

Thanks for your response. I have some follow up questions to your post.

Regarding " In live trading, execution decisions are based on many of the factors discussed in the lecture such as relative order size, liquidity, and urgency (information about intra-day or "short term alpha")" are you talking in a general sense of Market Impact model or particular to Quantopian's execution decisions? I don't see how to provide hints like "urgency" in the order_optimal_portfolio, which you have highlighted as a factor that affects execution decisions. Is that something that is in the works? Regarding the other parameters "relative order size"and "liquidity", the fixed basis points slippage that is being used in the backtester doesn't seem to vary slippage costs based on those parameters. My understanding is that it is always using 5 basis points slippage, could you provide us insights on why the slippage is not a dynamically calculated value based on those factors you mentioned?

Providing hints to the ordering mechanism whether the alphas are short or long term, whether I care more about fill timing or lowest slippage options etc., can help us fine tune algorithm performance to a greater degree by giving us more options to play around with.

Thanks,
Leo

From a December 6, 2016 Quantopian blog post:

We're currently working on implementing a more accurate, state-of-the-art slippage model that we hope will shift the green distribution in the plot above to nearly match the blue one. In the meantime, if you are concerned about the accuracy of the slippage model, you can set a custom model for your algorithm.

I gather that the 5 bps currently used in the default slippage is kinda +1-sigma of the IB slippage. Seems like a reasonable compromise. I guess I wonder if for a diversified algo trading 100-200 stocks that complies with the contest rules, if anything fancier is needed as a kind of back-of-the-envelope model to compare one algo to another? Plus the more degrees of freedom, the greater the risk of over-fitting and "gaming" I would think.

Isn't every model that is written on Quantopian assumed overfit (default assumption). Isn't that why we have out of sample?

I am saying please provide more accurate slippage cost, as that will benefit both the user and Quantopian and possibilities for low slippage fill options (i.e urgency parameter that Michele wrote about). Not sure what that has got to do with overfitting and gaming.

The feature that seems a reasonable compromise to you (fixed basis points slippage) can also be gamed and used for over fitting if one really understands where the advantage has shifted from the change. Gaming and over fitting cannot be stopped by freezing the platform at a certain level or a certain degrees of freedom. You need to compete in the marketplace and want the best tools that are easily available to other players in the marketplace. The platform need to evolve in sophistication, accuracy and features over time otherwise you risk becoming obsolete/inadequate/replaced etc. etc..

Hi Leo -

Good points. A more accurate model would be best. I got a bit carried away with the contrarian line of thinking.