Thanks for sharing this strategy James. This is an excellent example showcasing the usage of pipeline! The 'out of the box' performance is quite well even if I lower the number of stocks in each long short baskets and reduced the initial capital to 10k. The Momentum defined at line 42 looks more like a MeanReversion. Thanks again for sharing!

Really? I'm surprised it worked well with such a low amount of capital. Could you post your backtest, I'd love to see that?

It didn't for me :) Ten stocks per leg and 100k, backtest from 2003, is doing awfully...

The trouble with these long-short strategies is that while they are great for institutional investing, and of course for the fund, they are impossible to run as an individual. That means either writing one for someone else made-to-order, or writing one for Quantopian in the hopes that it gets chosen, with the downside that if it doesn't, it's wasted effort. I am hoping to come up with an algo that might be suitable for the fund, but which I could still trade for myself, so that if it doesn't get chosen, it's not wasted effort, but it definitely precludes algos which need hundreds of stocks for diversification and millions of dollars. If I write algos that I want to trade myself, which may or may not be suitable for the fund, at least I know the effort won't be wasted!

The changes I made in the attached backtest were: 20 stocks/leg and 10K for the initial capital. Aside from the volatility, it did surprisingly well from 2011 to 2014. :)

James Christopher notes an issue regarding de-listed stocks. I am assuming they get stuck in the portfolio since an algorithm does not know they have become delisted?

If so, the simple solution is the addition of a list classifying each stock as active or inactive. If inactive the stock is automatically exited on the final day.

I'm afraid I am still grappling with the basics of Zipline so I have no idea as yet how it deals with this issue.

Ah yes, further research shows this up as a major flaw in back testing. Which needs addressing on Zipline . If it is of any help, my partner and I use CSI data which updates a CSV file daily showing, among other things, when a stock becomes delisted. This is then dealt with in back testing automatically by the program we are currently using which automatically exits on the final day on back testing any stock which is delisted. If forward trading you need to be aware of upcoming delisting and take appropriate action. Or not!

The way Zipline currently works therefore makes a complete nonsense of back testing which includes delisted stocks. Unfortunately.

And not including delisted stocks makes a nonsense of any stock back test. Obviously.

Personally, rather than obsessing over the Quantopian competition/ hedge fund, my recommendation would be for people to concentrate on helping to improve Zipline. My partner has enabled us to use Zipline with our own local CSV files for instance since the use of data going back to 2002 is of little real value to us in back testing.

We are currently working on integrating FX conversions so that we can include a provision to switch base currencies and include non base currency instruments.

Quantopian is a wonderful effort and to open source Zipline is incredibly generous.

But is has some way to go before it becomes a genuinely useful tool in the real world for those who seek to back test and trade on a comprehensive and international basis.

Getting to grips with Zipline is a huge struggle partly because of the ever changing list if dependencies and version numbers and the complexity of getting the C compiler working in Windows.

But anyone who really wants to help develop the product (Quantopian/Zipline) into something of true use probably needs to get to grips with it.

Updated version this time focusing on improving the momentum aspect.

Simply put this makes it so our top ranking stocks (the ones we plan to hold long positions in) are filtered so that we consider only those with positive momentum, and likewise our lowest ranking stocks (the ones we plan to hold short positions in) are filtered so that we only consider those with negative momentum.

Seems to result in ~60% increase in returns and a marginally increased beta, while maintaining its previous volatility and drawdown characteristics.

New version focusing on improving the Quality factor.

Before I used a single metric, Return on Equity (ROE) to measure profitability. I wasn't really happy as ROE explains only a part of a company's profitability (how efficient it is at using shareholders capital). Consequently I did some quick research and figure the addition of two more metrics to form a composite score, similar to the Value score, could hold water.

• Return on Assets (ROA): how efficient management is at using its assets to generate earnings
  
• Gross Margin Ratio: how profitable a company sells its goods/services

The new composite Quality score shows ~45% increase in total returns for this backtest, however it also results in marginally increased volatility, drawdown, and an even more negative beta.

James - I really like this algo, but it has a big flaw in that it doesn't liquidate when a major drop in price for a security occurs. If you test it in 2008 - it loses everything because it keeps holding on to Fanny Mae (FNM).

Yeah, I noticed that too, fairly depressing, especially considering the nice run up it has before it crashes.

That area is one of many where there is room to improve upon on this algo. I've done absolutely nothing yet to control risk, beta, order execution, slippage minimization or anything along those lines. I've mainly been focusing on making the factors more robust/true to what they represent.

Sorta odd that the Q's data providers don't include information on scheduled delistings. I would think that anyone doing R&D at an institutional level would have these data. One could imagine a helper function in the Q API that would aid in managing delistings (e.g. get_delisted()).

To go at a multi-factor problem, I suggest having a look at response surface methodologies (e.g. http://www.itl.nist.gov/div898/handbook/pri/section3/pri336.htm). There is a systematic way of sorting out which factors are significant, finding interactions, curvature, optimizing, etc. One issue is that to run the "experiments" you need a script to launch lots of backtests (preferably in parallel) and then collate and analyze the results. You can see from the table 'TABLE 3.21 Three-level Factorial Designs' on http://www.itl.nist.gov/div898/handbook/pri/section3/pri336.htm that the minimum number of runs (i.e. backtests) to definitively sort out what's going on goes up rapidly with the number of factors (e.g. 6 factors results in 729 runs for the full model).

I'm having difficulty understanding this part in the original post:

ranks = results.rank().mean(axis=1).order()  

Do you calculate the mean value of all four factors like (f1+f2+f3+f4)/n and then rank them? Shouldn't you calculate the mean first before ranking?

By ranking each of the factors individually we can normalize the data, if we were to just average each value of each factor for each stock we'd get some pretty weird and meaningless results. (What does the average of the P/E ratio and a momentum measure even mean?)

Ah ok I understand now. Yes, adding those numbers together would be meaningless. Better to give them numerical ranks.

Lovely idea good sir and would be wonderful except the fact that in principle. VALUE and fundamentals are irrelevent in the current market. Using swing trading techniques with a very broad stroke of macro is what currently matters right now.

@James - have you figured out a way to fix this - without curve fitting - for the recession?

@daniel you could add a fourth factor, mean reversion, which works like momentum but in the opposite direction and with a 1 month time frame. This is actually the 1 month that is excluded from the momentum calc. These will do well in periods of high volatility, like 2008

A fourth factor could totally help, though I do caution a little doing that. The main reason is that the way the factors are combined in this algo is by a simple mean of their ranks, consequently, the more factors you have the less likely you are going to get concentrated high or low rankings. There are many better ways to expose yourself to these factors. Another thing to take note is that any signal generating factor has times when it is and isn't working.

I've been backtesting a long only version of this strategy, combining that with the Simple Downside Protection mode outlined by Dr. Gray here leads to some pretty good out performance will bearably drawdowns.

Be aware that you're using quarterly ratios (I'm referring to ROA, ROE, gross margin, etc...). Is it you real intention?
I'm also implementing a similar multifactor pipeline algorithm but - as I need TTM ratios - I incurred in problem described here:
https://www.quantopian.com/posts/error-in-fundamental-data#56f5da99f44e1db34b00064e

Yeah. I'm aware of that, I think the idea here is to just use the most recent data available. Also note that way to make the fundamental factors more robust is to z score the factor value, that way you don't have to worry about the scaling issues associated with different time frames.

A percentage rank test is a good alternative to a z score. It checks the percentage rank of the result in the last N occurrences.

It looks like this has problems with KGC in early December 2004. I can't see why it took such a huge position there, but it seems to think it loses all of it. Looks like bad stock split data? I guess that might explain the oversized position if it's confused.
http://www.dividend.com/dividend-stocks/basic-materials/gold/kgc-kinross-gold-corp/
lists a 0-1 split on 2004-12-05 and 100-1 split on 2004-12-06.
What do you normally do in this type of situation? Ask to get the data cleaned up? Manually exclude KGC?

This is one of the most popular algos on the site so I thought it would be cool to share how it has been doing since I published it. Here is an out of sample tear sheet with Bayesian stats on the latest reply's code. I ran this live for the first half of 2016 and can confirm the results were very similar.

Classic!

I fear so. I was talking to someone in NY banking about one of their big HF customers.

They have a huge number of analysts producing algos and when the algos go sour they get binned and new analysts taken on. A huge turnover of analysts. And we can see why.

This trading strategy gets a 90% haircut in 2004. And if it could happen in the past, it might in its future. Anyone has a correction for that?

Hi James!

First off, I wanted to say that I came to Quantopian to implement this exact kind of strategy! I read up a ton on factor investing in academic papers but I know next to nothing about programming (I did economics and math in undergrad but didn't take any CS/programming courses unfortunately) and I came here to practice both programming and to implement what I've read about! I'm envious that you could whip this up in an hour - I'm going to have to work hard to program that well haha.

I added a better factor for quality - gross profitability a la Novy-Marx (2013). It's simply gross profits divided by total assets. I am personally convinced that GP as defined by Novy-Marx is the measure of quality to use (see his paper "Quality Investing" here).

Substituting ROA/Gross Margin with gross profitability ups the returns from 192% to 301% over the same time period. The Sharpe ratio nearly doubles. The draw down is slightly lower but the beta is slightly higher.

NOTE I edited out a lot of my comment because I figured out the issues after writing this comment. It was unnecessary to put here, as I simply didn't read the momentum factor correctly in the original code. END NOTE

Also, low volatility may not be necessary either - see (again!) Novy-Marx on the defensive/low vol/low beta factor here.

I'm still trying to read and understand your code here so I can tweak it on my own instead of just sitting in the peanut gallery making passing comments.

Cheers,

Cory

James (Christopher)
A key question for you. For which I have no answer at all. Your algo looks immaculate - you designed it based on fundamental principals and it did well in the test period. Out of sample has been disappointing.

What lessons should we learn from this? Do you consider that you fitted the algo to the data (consciously or unconsciously)? What do you think was / is wrong with the concept? Or is the algo going through a rough patch from which it will recover?

James -

This whole multi-factor equity long-short jazz seems to be what y'all are after, given the design of pipeline, your Alphalens and workflow (https://www.quantopian.com/posts/alphalens-a-new-tool-for-analyzing-alpha-factors), the move to $10M capital for the contest, etc. Perhaps it is obvious to those in the industry, but why would one think that it is the way to go? Has such an approach been proven out over 20-30 years? Is it a recent fad? What's the backstory?

Grant

Cory I ran a longer backtest on your algorithm and it seems to perform very poorly before 2012. It then 'switches on' and performs well, and it looks as if it might be moving into another period where it begins to perform poorly from the end of 2016 onwards.

I'd guess this is due to regime changes in the market, and factors coming in and out of relevance. I'm not certain though.

Hey Joe,

The algorithm fails in 2004 because Quantopian's data is messed up.

The huge drop in 2004 is a result of KGC going through two stock splits but it shows like an 8 mil loss.

As you know stock splits should be neutral wrt price.

Anyway, I re-ran this code while manually excluding KGC from being purchased or shorted. This is a crude fix but the backtest survives the bloodbath in 2004.

Unfortunately I ran it last night on my laptop. I use Comcast's Wi-Fi hotspot near my house and it crapped out on me in the middle of the night so a full backtest didn't occur. I will run one later today.

For a trading strategy to have some merit it must first pass the test of time. And second, it also has to outperform its benchmark, otherwise an index fund would have been a better choice than having a program churning and churning to produce very little in added benefits.

I skipped 2004 entirely, this way skipped the KGC problem.

Only changed the momentum to its latest value and got the results of the attached backtest. Changing the momentum calculation from close[-20] / close[0] to close[-1] / close[0] on a 252 lookback period should have had a minor effect. You made it more responsive to the latest data and not something from a month ago. Even if it is was to have a minor impact, 20 more days over 252, the effect was to delay the recognition of the financial crisis and then misreading its exit. Mishandling the financial crisis had a major impact of this trading strategy.

Can not say that I like the general behavior of this trading strategy in its present state.

Joe:

I removed KGC from the algorithm to avoid the weird 2004 data issue. This may have hurt the performance over the entire time period, but unfortunately this is the quick fix I was able to make.

As you can see, it performs pretty badly coming out of the recession. There may be a reason with the data, again, but I am not sure. I will have to delve into it when I get off work. Two years of flat returns boggles my mind.

The algo doesn't perform well towards the end of 2016, but it does peak at 360% returns.

Grant:

The long-short algorithm that systematically buys factors comes from Fama and French 1992 and subsequent research on what drives stock outperformance. Value and momentum premia have existed for over 200 years of stock data.

The implementation of the factor strategies are new, however.

It is worth mentioning that individual factors will underperform the broad market at times. Factor premia come about over long periods of time. This isn't a get-rich-quick strategy. But, combining factors that have negative correlations - like momentum and value - can increase the ability for these factors to outperform.

The four-factor strategy outlined by James (adapted from GSAM) is not uncommon. It combines the value, momentum, defensive and quality factors. However, the formulation of the different factors will affect their performance - as I've demonstrated above, changing quality from weak ROA, ROE or Gross Margin to Novy-Marx's Gross Profitability greatly enhanced the returns.

Guy:

You shouldn't use a 12 month look back period for momentum. It is well documented in the momentum literature that you need to use a 12 month minus 1 month (or 6 month minus 1 month) return for structuring your momentum strategy. The reason being is that there is a one-month short term reversal issue as well as liquidity factors to consider. James is very right using close[-20] / close[0] for his momentum factor.

Remember, momentum is not well understood. It is essentially a technical - charting, just more...robust. The t-months minus 1 is bog standard.

Novy-Marx also wrote a paper showing 12-month minus 7-month is more informative than 6-month minus 1. I will add that later.

Here is some information regarding momentum construction:

https://www.aqr.com/~/media/files/fund-documents/aqr-momentum-index-methodology.pdf

http://pages.stern.nyu.edu/~lpederse/papers/ValMomEverywhere.pdf (page 936 has other sources)

https://www.msci.com/eqb/methodology/meth_docs/MSCI_Momentum_Indices_Methodology.pdf This is MSCI's construction of momentum. It uses z-scores and 6-month momentum, but the idea is the same.

Cory, the chart below is about momentum flutter. It is a stochastically generated chart with drift. If I press F9, a new chart is displayed. The momentum line could be chosen to be something else than the linear regression presented here. It would not matter much since there is little predictability in its construction except for its small upside biased drift.

Momentum Flutter

http://alphapowertrading.com/images/divers/Momentum_Flutter.png

That you use data 1 month or 6 months old, you still need to construct your 12 month lookback. There is no added value in taking data 1 month or 6 months ago for decision surrogate. However, they will still catch the upward drift while it last. And the market has been going up for the past 7 years. So, there was plenty of time to catch this drift, not by precognitive analysis, but just because it was there.

How long this momentum will last is another story.

Guy,

I think you misunderstand "momentum" in financial economics literature.

Jegadeesh and Titman wrote the seminal paper on it, using J-month portfolios.

Since then, economists have found that 12-1 month momentum predicts stock returns. It is distinct from the stylized fact that the stock market goes up in the long run (equity risk premia is positive).

Also, evidence suggests momentum is a persistent factor that explains stock returns, and has explained returns for over 200 years and in different asset classes.

You don't have to believe the evidence, but that's why 12-1 is in the original code.

Cory, I understand perfectly. I have not said anything different than what you just expressed. But, see, about anything, that you would want to analyze over a 12 month period, delayed data or not, could help “predict” where the market is going as long as, in general over the long term, it is trending up. There is memory, there is persistency in average market prices. All fine.

But, there is this big but. If there was that much predictability in market prices, one could certainly devise trading strategies that would perform a lot better than this one, generate a lot more alpha than having such a hard time beating a benchmark. It does not really matter what the foundations of this predictability is. It should show its presence in any long term portfolio backtest. And it is not what I see here...

But, see, about anything, that you would want to analyze over a 12 month period, delayed data or not, could help “predict” where the market is going as long as, in general over the long term, it is trending up. There is memory, there is persistency in average market prices.

That's not what momentum is saying.

Momentum is the idea that there is a premia on stocks that exhibit high (relative to their peers) momentum over stocks that exhibit low (relative to their peers). So, a long-short portfolio of stocks with high momentum on the long side and low momentum on the short side will provide alpha over the long term.

It isn't saying momentum predicts where the market is going. It's saying that momentum is a signal of stock returns.

If there was that much predictability in market prices, one could certainly devise trading strategies that would perform a lot better than this one, generate a lot more alpha than having such a hard time beating a benchmark.

You do realize that the value, quality, momentum, defensive, etc, factors in academic finance are puzzles right? They're unexplained, as of yet.

They don't give alpha every year. They may not give alpha over many years. But in the long run these factors do provide alpha. Mixing them together gives you a strategy that can generate alpha over the long run. See, for example, MSCI's smart-beta index based on size, value, quality and momentum.

It does not really matter what the foundations of this predictability is.

Well, it does. Quantopian doesn't want algos that aren't based in economic intuition. If you just have some data mined factor you're not going to get consistent results.

It should show its presence in any long term portfolio backtest.

Again, it does. Please - read Jegadeesh and Titman 1993, Carhart 1997, or Asness et al 2013. If not, the Alphalens preview actually develops the 12 minus 1 momentum strategy as its example.

And it is not what I see here...

And, of course, this isn't a completed algo. I also object to the idea that it has a "hard time beating its benchmark" when there's only a two year stretch of bad performance (which could ultimately be a KCG-data type issue - who knows?).

Cory, so, for now, based on the backtest I did, using the present “predictive” powers of that trading strategy I have to conclude that the obtain results were unimpressive.

Maybe, all I saw, just as you said, is an incomplete, an half-baked trading strategy.

I'll wait for it to come out of the oven. Impress me.

Sorry on that one.

No intention of offending anyone. Just ran the backtest giving it a longer trading interval to see how it might behave. If a trading strategy can't last, I lose interest. Plain and simple.

I certainly did not like the output of the backtest I performed which did not include the KGC 2004 problem since the start date was 2005. Whatever kind of predictability that strategy might have, based on my backtest, it spent over 6 years in a row underperforming the index.

Are there published examples of long-short funds that one could hold up as benchmarks? For example, I found Vanguard's VMNFX:

https://personal.vanguard.com/us/funds/snapshot?FundId=0634&FundIntExt=INT#tab=1

5-year annual return is 3% with $1.8 billion in the fund, which I guess is better than parking your money in a bank CD, but it isn't in the category of legendary hedge fund returns.

Vanguard Market Neutral Fund seeks long-term capital appreciation while limiting exposure to general stock market risk. The fund’s performance is not correlated with the returns of either the broad equity or bond markets. The fund aims to generate a return that exceeds that of 3-month U.S. Treasury bills in up and down markets by following a long/short market-neutral strategy. The fund seeks to invest on a market-neutral basis in a diversified portfolio of securities. The advisor attempts to fully implement its process by holding both long and short positions in approximately equal aggregate amounts while maintaining near-neutral exposure to industry and market-capitalization risk factors. The advisor uses proprietary computer programs that rank stocks based on the assessment of each stock’s return potential. Securities with the highest rankings are purchased, while those with the lowest rankings are sold short.The fund is managed without regard to tax implications.

Sounds a lot like what Quantopian is promoting.

I see http://etfdb.com/etfdb-category/long-short/ so the same thing is available in ETF-land. Maybe I should figure out how to enter one of these in the contest?

Vanguard Market Neutral Fund seeks long-term capital appreciation. It looks like it has done nothing for 8 years.

I think it's doing its job as long as it's helpful in diversifying a traditional stocks and bonds portfolio and has a positive expectancy in the long run after fees. Actually, if it's negatively correlated at tail events I'm not even sure the latter is required.

https://www.portfoliovisualizer.com/backtest-portfolio?s=y&timePeriod=4&startYear=1985&firstMonth=1&endYear=2016&lastMonth=12&endDate=08%2F31%2F2016&initialAmount=10000&annualOperation=0&annualAdjustment=0&inflationAdjusted=true&annualPercentage=0.0&frequency=4&rebalanceType=1&showYield=false&reinvestDividends=true&symbol1=VMNFX&allocation1_1=50&allocation1_2=70&allocation1_3=90&symbol2=VBINX&allocation2_1=50&allocation2_2=30&allocation2_3=10

Sorry, that was the wrong link. This one illustrates the diversifying effect of including small amounts of the vanguard market neutral fund in a traditional 60-40 portfolio.

It's mildly increased the sharpe ratio and helped improved the cagr/maxDD ratio too:

https://www.portfoliovisualizer.com/backtest-portfolio?s=y&timePeriod=4&startYear=1985&firstMonth=1&endYear=2016&lastMonth=12&endDate=08%2F31%2F2016&initialAmount=10000&annualOperation=0&annualAdjustment=0&inflationAdjusted=true&annualPercentage=0.0&frequency=4&rebalanceType=1&showYield=false&reinvestDividends=true&symbol1=VMNFX&allocation1_1=0&allocation1_2=10&allocation1_3=20&symbol2=VBINX&allocation2_1=100&allocation2_2=90&allocation2_3=80

Burrito Dan
Boring, dull stuff but I suspect that in the real world a more reasonable representation of what is achievable than much of what we see in the world of hedge funds.

Portfolio Visualiser is an excellent tool and I particularly like their use of long term data. Are you the proprietor?

Burrito Dan'

But it actually loosing money since Dec 31 2007

Initial Balance Final Balance CAGR Std.Dev. Best Year Worst Year Max. Drawdown Sharpe Ratio Sortino Ratio US Mkt Correlation
$10,000 $9,812 -0.22% 4.78% 8.59% -11.38% -19.53% -0.07 -0.10 0.11

Hey Anthony, no, I run an excellent fast food burrito chain for a living :)

I would classify the vanguard fund as a hedge fund by its investment style, but from a regulatory and fee point of view it sounds like mutual fund.

Anyway, i get tired of the "60-40 beats hedge fund" argument. You can diversify even with a low expected return, and diversification can be a free lunch, where the sum is greater than the parts.

@Burrito Dan

The Vanguard Market Neutral Fund is a mutual fund, it doesn't just sound like one ;)

Maybe I am understanding this incorrectly, but when I tried the backtest, I rebalanced monthly, and then checked the transaction log. It made 12,000 transactions over two months. From what I understand a transaction is buying or selling multiple shares, so theoretically it should have made 400 trades per month. Why is this happening?

Hi James,

Instead of taking the mean of ranks as you do, I want to convert them to normalized scores and then take their mean. Something like this for each rank:

score = rank.argsort() + 1.0  
score /= score.shape[0] + 2.0  
score = sp.stats.norm.ppf(score)  

Could someone show me how to do this with this algorithm please?

Best regards,
Pravin