Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Dynamic Portfolio Selection with the Kelly Formula

Hello Everybody,

Investopedia describes the Kelly Formula (criterion) as a method used by gamblers and investors to determine what percentage of their bankroll/capital to use in each bet/trade to maximize long-term growth.

In finance the Kelly formula is generally the mean excess return divided by the variance in the returns of an investment strategy. For multiple strategies, it becomes (inverse covariance matrix) * (vector of mean returns). It is often used in risk management applications, like deciding on an appropriate amount of leverage, and allocating capital between multiple strategies. I did not use the excess returns in this example though.

Here's a breakdown of the strategy

  1. Assume each security in a percentile of the DollarVolumeUniverse is an independent investment strategy.
  2. Calculate the Kelly leverages of all securities in the universe
  3. Use the result as a ranking and select the top N securities as the new portfolio.
  4. Drop the independence assumption of the selected securities and recalculate the Kelly leverage using the covariance matrix and mean returns.
  5. Invest an amount proportional to the Kelly leverages in each security.
  6. Repeat (quarterly in this case)

It's not perfect and under construction, but I wanted to get it out there so I can see any variations you come up with. It seems to be hit and miss and tends to want to short a lot of the portfolio. I added a parameter to limit the amount of short exposure, which helps out, but I'm I haven't found any optimal settings yet.

Please share any thoughts and modifications. The backtests take a long time to run so be prepared to wait a while.

Happy tinkering,
David

P.S.
For those that like a challenge, It would be cool to see an algo that calculates its own Kelly leverage and uses it to decide on the leverage going forward.

13 responses

very cool, i was just thinking about kelly portfolios, it's nice to see your implementation

i really like your event manager, its very handy.

related reading i found interesting (when they talk about rentech)
http://www.nuclearphynance.com/Show%20Post.aspx?PostIDKey=67428

I am also very interested in this. Following has a multivariate case: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2259133

Thanks Fred, the event manager has become the first thing I paste into new algos, glad others are finding it useful too. Interesting paper Suminda, it looks doable in Quantopian, I'll have to wrap my head around it first.

Here is an interesting result from this, it has the context.short_pct = 1. I have had better results from keeping it mostly a long strategy and dropping the dynamic selection. I'll do a separate post for that version.

The following formula is what needs to be implemented which is the final conclusion for the unconstrained version.

\vec{u^{\star}} = (1+r) ( \widehat{\Sigma} )^{-1} ( \widehat{\vec{r}} - r )

It appears the current formula works well in down markets - though is there a way to increase performance in bull markets?

Suminda, this is a version that implements the unconstrained version from that paper. I'm not sure I interpreted it correctly, and it's incomplete. This version doesn't invest in the bond asset when it should. It behaves in a similar way to the last one, the short side needs to be kept in check. This one is also done with a static portfolio

Great effort but may be some small thing has been overlooked. I am also not sure at this point that it is.

May be you can get in touch with the author of the paper. He might be able to give you some insights.

This is a great post, thanks for sharing. The EventManager class looks really useful and certainly something we thought about providing in Zipline/Quantopian for your convenience.

I also saw that you got acquainted with a bug in history:

# Conversion to make history columns sids with a workaround  
# for a bug that causes extra columns in history from time to time.  
sid_map = lambda data: {i.sid: i for i in data.iterkeys()}  
def map_columns(df, data):  
    mapping = sid_map(data)  
    mapped_cols = {}  
    for i in df:  
        if i in mapping:  
            mapped_cols[mapping[i]] = df[i]  
        else:  
            log.debug('sid(%s) missing from data!'%i)  
    return pd.DataFrame(mapped_cols)  

Note that this bug should now be fixed on Quantopian (https://github.com/quantopian/zipline/commit/96bdb22db926b84cc6df59fa790d1e38799d0d96). Please let me know if it doesn't work correctly for you.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

This is very useful. Perhaps you can add more scheduling options also. This should be at the exchange level.

It looks like the bug fix worked!
Here's a version without the workaround, I also dropped the second portion of the algo, this version just selects the top N kelly scores with the independent returns assumption and buys those securities.

@Anony, I thought about the asset returns as the potential upside of a winning bet. I'd say that the variance in the returns has more to do with the probability of winning the bet.

How is covariance^{-1}*return is derived from Kelly criterion. Any body can explain it?

Although I'm not familiar with the term "Kelly Portfolio", I would guess that it relates to Edward Thorp's paper on the Kelly Criterion in Blackjack, Sports Betting, and the Stock Market. The OP cites an Investopedia article, which describes the "Kelly formula" as (in Thorp's "discrete" format) as f* = p-q/b where f* is the fraction of the player's "available resources" to bet on the stock, p is the probability of winning, b is by how much, q is the probability of losing, and "a" (missing from the Investopedia formula) is for how much (losing, assuming you cut your losses quickly and don't just let it go to nothing). In my opinion, since the trader would not want to let that happen, this should not be effectively declared to be the case. A more appropriate form of the "Kelly Criterion" is f*=p/a-q/b, in which some traders I know of set to something as low as 0.01, or 1% of the traders position in the stock (NOT of the player's available resources!) , i.e., the player's loss tolerance as a fraction of position size taken.

Of all the variables in the formula (a, b, p, and q), "a" is only one the player has direct control of as the trade is taking place. Variable "b" you just want to be as high as possible, but you won't know for sure unless (1) you were right about what direction your trade would go, and (2) you didn't cut it short before it could run its full course. In choosing a value for "b", you can only make your best guess as to how far it will run, then go with that when you calculate your position size, i.e., f*. You can only control "p" and "q" (where p+q=1) through the selection process of the stock, as is the case for "b". That brings us back to "a", the only variable you control directly once the position's been taken. And by clamping down on "a", you raise the value of "f*", something the "cut losses quickly" heuristic doesn't say anything about. The heuristic's good, but it doesn't say you can ramp up your trading fraction, whereas the Kelly Criterion does, amd which in my opinion is more than enough reason to use it whether you go full Kelly or not. Anyhow, that's the story of "a" in the Kelly Criterion and why it's important. Otherwise, in an otherwise perfect trade you'll just be leaving money on the table to people who know less about trading than you. And while charity's fine (subscribe to mine!) that's not any of us are here for.

Regarding the "discrete" case of the Kelly Criterion that uses "a" and "b", Thorp derives the KC in his paper for "even money", but in the stock market that is not the case, which is why I do it for "uneven money", where the player can win or lose any fraction or multiple of the amount bet. In the even money case, "a" and "b" (or in the Investopedia article, just "a"), are assigned values of 1, which is clearly not the case with the stock market unless your goal is to lose all your money every time you lose some of it. And while some might say that to do so is simply being prudent, in the engineering world I'm from they also say "garbage in = garbage out". And as Edward Thorp once said after reading Benjamin Graham's book on Security Analysis, "I was both surprised and encouraged by how little was known by so many." Happy trading everyone!