The Quest for Capacity: Optimizing Sharpe Ratio under Varying Capital Levels

Back to Community

edited

With interest rates at all-time lows, the pressures on institutional and private investors to seek new harbors for vast sums of capital are high. Quantitative hedge funds are thus racing to increase capacity of their portfolio of trading algorithms. Despite these market forces, surprisingly little public information is available on estimating and maximizing capacity.

In this post we will take a look at the problem of maximizing the Sharpe Ratio of a portfolio of uncorrelated trading algorithms under different capital bases.

What are the limiting factors of capacity of an algorithm?
Fixed transaction costs are usually not a concern at institutional trading levels as brokers offer very competitive transaction fees. The main headwind when trading larger amounts comes instead from slippage and the fact that large orders drive up the price (for more information on the impact of slippage on trading algorithms, see also my other blog post on liqudity).

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

18 responses

Simon Thornington

Nice!

Alan Coppola

Thomas,
Nice job!
I take it that these are the theoretical curves, based on a certain models for sharpe and slippage.
Given that the Q IDE is one big simulator, would a potential next step be to run the Q Simulator on some actual portfolios, compute the same curves, and see how the Q IDE simulated curves match up with the theoretical curves you have so nicely generated.
In the wireless-channel-simulation world, this is akin to getting theoretical BER(BitErrorRate) curves for certain channel models, and then seeing what you get with modelled channel impairments.
alan

Thomas Wiecki

Alan,
Yes, here I used theoretical curves to demonstrate the logic. In reality, we are estimating the Sharpe decay curve by running a slippage model on the algorithm returns under different capital assumptions. The logic outlined here after we have generated these curve remains the same. We are also in the process of comparing how accurate these simulations are to the actual slippage we are seeing so your comment is spot-on. The connections to wireless-channel-simulation are very interesting.

Disclaimer

Thomas Wiecki

Blog post with some nice additional visualizations now live here: http://blog.quantopian.com/optimize_capacity/

Disclaimer

Simon Thornington

Do you guys have any data yet on actual slippage experienced at various order sizes and stock liquidity profiles? Or is all of this still based on that arbitrary quadratic slippage model?

Thomas Wiecki

@Simon Very active topic for us right now. Seems like zipline slippage is underestimating at low order sizes and overestimating at high order sizes. Definitely something we need to improve. For more realistic simulations we are using the Almgren slippage model: http://www.cims.nyu.edu/~almgren/papers/costestim.pdf

Disclaimer

Simon Thornington

Nice. I am very curious to see how it calibrates to your live-trading fill data; the results I expect will be different and better than that paper, due to the positive influence of HFTs on spreads.

Grant Kiehne

Hi Thomas,

What is the justification for your Sharpe versus market value curves?

er_funcs = (  
    lambda x: np.exp(-x/50)*0.5 -0.2,  
    lambda x: np.exp(-x/40)*0.2 - 0.02,  
    lambda x: np.exp(-x/70)*0.15  
)

vols = (  
    0.06,  
    0.06,  
    0.07  
)

Does this imply a certain theoretical form of slippage? Or are you basing the curves on empirical evidence?

the problem of maximizing the Sharpe Ratio of a portfolio of uncorrelated trading algorithms under different capital bases.

I'm a little confused by this approach, since presumably you would never have uncorrelated trading algorithms (unless by some magic, the ones you are selecting for the Q fund are perfectly uncorrelated). Are you just making the simplifying assumption to do a "back-of-the-envelope" illustration? Or would you have factored out the correlations, so that you are only dealing with the uncorrelated returns?

Very active topic for us right now. Seems like zipline slippage is underestimating at low order sizes and overestimating at high order sizes. Definitely something we need to improve.

In the context of your internal discussions, is there any dialog around how you might share with users actual trade data (owned by Quantopian) so that they could empirically determine slippage? As you say, "The main headwind when trading larger amounts comes instead from slippage and the fact that large orders drive up the price" but if users don't have access to the data, then they won't be able to manage it. Also, I would think that given the vastness of the industry, there would be vendors who would provide accurate slippage models, on a case-by-case basis. Maybe you could purchase the data/models? Or maybe the data/models tend to be proprietary to trading firms?

For more realistic simulations we are using the Almgren slippage model

If it is more accurate, then why not update the default Quantopian slippage model, required by the contest? Or maybe that is the plan?

Best regards,

Grant

Thomas Wiecki

Thanks for the great questions, Grant.

Does this imply a certain theoretical form of slippage? Or are you basing the curves on empirical evidence?

These are roughly what the curves look like with the Almgren slippage model. They are also chosen to show some interesting patterns in optimal allocation when changing capital levels.

I'm a little confused by this approach, since presumably you would never have uncorrelated trading algorithms (unless by some magic, the ones you are selecting for the Q fund are perfectly uncorrelated). Are you just making the simplifying assumption to do a "back-of-the-envelope" illustration? Or would you have factored out the correlations, so that you are only dealing with the uncorrelated returns?

Yes, back-of-the-envelope. In reality we would take these into account, this is just an abstraction and I mention it for completeness here.

In the context of your internal discussions, is there any dialog around how you might share with users actual trade data (owned by Quantopian) so that they could empirically determine slippage? As you say, "The main headwind when trading larger amounts comes instead from slippage and the fact that large orders drive up the price" but if users don't have access to the data, then they won't be able to manage it. Also, I would think that given the vastness of the industry, there would be vendors who would provide accurate slippage models, on a case-by-case basis. Maybe you could purchase the data/models? Or maybe the data/models tend to be proprietary to trading firms?

Good question. I think the types of analyses we're developing would definitely be useful in the hands of users to get a sense of their slippage. Not sure yet how exactly we'd do that but in principle I completely agree.

If it is more accurate, then why not update the default Quantopian slippage model, required by the contest? Or maybe that is the plan?

That's certainly the plan. It does require some more data that is currently not available to the backtester at that level so it's not as straight forward to implement as I would like. It makes sense to me to try and at least make the current slippage model better as that can be done easier. E.g. adding a minimum slippage of a few bps seems like a sensible change.

Disclaimer

Grant Kiehne

Thanks Thomas -

By the way, I'm seeing your response as a set of 5 identical postings above (and I got 5 e-mails, too). Nothing like driving home your points with repetition, Thomas!

Might want to let Q support know, since if this is a bug, it'll create a real mess on your forum.

Thomas Wiecki

Haha, I hope I made myself clear ;).

Disclaimer

Grant Kiehne

Another angle is that knowing the slippage on a short time scale is knowing how the market will respond. At short enough time scales, it would seem it would be a cash cow to know the slippage. This must be a HFT game--putting in orders, cancelling them, following up with orders to sell/buy, once the price has gone up/down--arbitrage based on the slippage model. Kinda interesting.

In the case of Quantopian/Point72 (I view them as the same now, correct?) and the long-short "investment" style algos (versus active market participation, in-your-face "trading"), it would seem you are just trying not to take it on the chin. There is no arbitrage in knowing the slippage model. Is this an accurate assessment?

Grant Kiehne

A larger question is why are you working on this in the first place? I'd think Point72 would know all about it (and your new CIO, as well). A "solved problem" in the industry, in other words. Or is it more of a quickie illustration, rather than a suggestion that you are starting a the ground floor in your understanding? Not trying to be a jerk...just dawned on me that you may be re-inventing the wheel. But then, sometimes it is best to start from scratch, to fully internalize a problem and its solution (and certainly if you intend to improve upon the current state-of-the-art).

Deleted User

In experimenting with algos that might meet the contest guidelines (in particular $10M starting capital and default slippage settings), I'm finding it somewhat too easy to boost alpha by broadening the universe to include more thinly traded stocks and letting them take as long as they need to for orders to fill, even if it wraps to the next day (which requires extra code to implement).

Running such an algo live would result in much quicker fills (since they're market orders), but with far greater price impact. This would probably make live results significantly different from backtest results even if the "alpha" in the backtest would have persisted without the live trading interference. Forward testing with paper trading will, of course, not uncover this problem.

What is the latest thinking as to how price impact can be effectively simulated?

Thomas Wiecki

Our current slippage model underestimates slippage, so we will fix that and make it more realistic. It is also too aggressive in splitting large orders up over multiple bars.

Disclaimer

Alan Coppola

@mhp - yes, we had to go to weekly rebalances, and keep transacting on subsequent days to reach our rebalance percentages with the $10m base. OBTW...we've noticed that the fills have gotten magically better about a week ago...so that is good!

@Thomas - Super! Glad the slippage model will be improved! Any further validation work to sync the backrest data with reality is much appreciated, and brings with it a feeling of confidence in the value of back testing.

alan

Grant Kiehne

@ Thomas -

There is also the characteristic that orders are not filled by the backtester if there is no historical trade (event) in a given minute. It requires actual trade data, not extrapolated/interpolated. It is not obvious that this needs to be changed, but my understanding is that it is very rare for orders not to be filled almost immediately in the real trading world.

Thomas Wiecki

@Alan: Completely agree, Gus did a deep dive into that so we have something to work with.
@Grant: Good point, that should definitely be fixed too. Although I predict that this doesn't occur for liquid stocks like those in the Q1500.

Disclaimer

You've successfully submitted a support ticket.

Our support team will be in touch soon.