Eyeballing shorts doesn't seems very deterministic. Have you considered introducing net_leverage as a metric and requiring

abs (net_leverage) <.75

or some % that has the desired effect?

Sorry, perhaps it's written somewhere else, but what specifically are the new requirement for hedging? Are the contest entries now getting the badge only at some specific level of simultaneous notional exposure long and short?

Or are the future disqualifications going to be a post hoc judgement call...

Dan, I think this is fair borderline on generous, so thumbs up.

The contest entry list does feel like a bit of a bazaar at times. The more it is aligned with the fund's requirements the more motivating in my view. This would include downgrading entries with:

• Insufficient hedging (as you describe above)
  
• Concentration risk (e.g. >10% in a single stock/etf)
  
• Passive algos (i.e. not Active by Jonathan Larkin's criteria in his blog post)
  
• High correlation to strategies already in the fund (although this might be difficult to make transparent in practice)

I also note the contest rankings are often a few days out of date. I've seen it a few times, but today is a good example. The rankings didn't run on Monday morning, so they are showing "last updated: 4 days".

Maybe the rule should read delta neutral, or (near) neutral and give some range or neutrality i.e +/- 20%.

The specific rule is important, because there are instruments with internal leverage that might provide an effective hedge even if only 1/3 of the capital is allocated to them. This problem will be worse when futures are allowed, though presumably that will herald a complete overhaul of the margin and notional exposure calculations.

I totally agree with Simon, rules should be extremely clear.

Why isn't this rule already captured in the requirement for low beta?

From Simon: there are instruments with internal leverage that might provide an effective hedge even if only 1/3 of the capital is allocated to them

Sure (e.g. SPXS, as noted here: https://www.quantopian.com/posts/zero-beta-using-spxs-3x-bear-etf). Why is this a bad thing though? The use of "internal leverage"?

I think explicitly geared ETPs are already forbidden. I was talking more about whether they intend to mandate some kind of dollar neutrality or something else. Attached is a silly strategy which would probably pass all the automated tests, yet if I understand correctly, would be rejected if it won.

EDIT: if anyone is feeling adventurous, you could probably turn this into a conforming strategy by trading the top 100 stocks of the consumer cyclical vs consumer staples directly, rather than using ETFs. In fact, you could hedge with the top 100 in the S&P 500 by market cap too.

Eyeballing shorts doesn't seems very deterministic.

Could also look at correlation between market and algorithm performance under samples from the time-series model used in the Bayesian tear-sheet.

Why do you have to short to hedge? In most cases, a long T-bond positions is actually a better hedge.

Quantopian - "We want low beta, high sharpe, perfect hedging and low correlation to the rest of the algos. We want a unicorn!" Good luck with that, guys. Meanwhile, back in the real world, where quant pay is higher than $10,000 for six months... At least have the integrity to acknowledge that hedging can be accomplished in a long-only portfolio.

Why do you have to short to hedge? In most cases, a long T-bond positions is actually a better hedge.

Totally agree with this as well.

Dan comments above:

Just about everything also gets evaluated for an allocation, but that process is separate from the way we declare a winner.

Maybe the two processes should be reconciled? One would like to think that high ranking in the contest would indicate a high probability of getting an allocation, right? Presumably, if Quantopian aims to fund a decent fraction of its 80,000+ users, both the contest judging and allocation evaluation processes will need to be largely automated (unless the plan is to fund only 30 or so algos, in which case, a manual process might suffice). In the end, it would seem that there would be enough data to identify and rank promising algos automatically. Can the HAL 9000 be programmed to sort it out? "Just what do you think you're doing, Dave? Your algorithm is grossly overfit!"

@Simon,

Thank you very much for sharing the algo. Could you please describe shortly what it does?

Why do you have to short to hedge? In most cases, a long T-bond positions is actually a better hedge.

Um yeah...why bother?
Dual Momentum

See "my" 50/50 system:

    50/50  
1m LB  
CAGR    Max DD  Vol  
mean    6.86    12.73   7.61  
median  6.88    12.95   7.63  
min 4.84    8.79    5.35  
max 9.26    16.34   9.15  

Thank you very much for sharing the algo. Could you please describe shortly what it does?

It just switches between consumer staples (defensive) and consumer discretionary (aggressive) ETFs based on the slope of the VIX term structure, and hedges with S&P 500. All static.

Here is simple question. Let's assume I have a magical algo that uses sp500 and bonds to balance excellent score and I'll win the competition with it.

If I use spy + some bond etf to hedge it will be disqualified because it's "all long".
If I use spy but instead of going long some bond etf I go short inverse bond etf. The algo will be long/short algo and not be disqualified.

Why? What is the difference in risk between these algos?

It was my understanding that those would both be disqualified for not having enough assets presenting some sort of concentration risk. But that is neither here nor there.

It would be interesting to hear from the Q powers-that-be why the long-short hedging jazz is so special? Why is it such a big deal? Is there some proven magic in it? I keep hearing folks say it ain't all that special. Well, why is it special? I kinda see that if one can get a nice smooth long-term return and then apply 6X leverage, as Q has announced as part of their plan, it could work out nicely. Is that why it is so exciting? There's been very little information on why the contest rules and fund criteria are what they are...is it just that it is what Point72 is asking for? For those in the industry, maybe it is obvious, but it is not so clear to everyone here.

By the way, I sorta have to congratulate anyone who managed to game the contest a bit. It would seem to be in the spirit of hacking Wall Street (as http://blog.quantopian.com/quantopian-manifesto/ with its raised fist would imply--power to the people!). Kinda like a trick play in football--if it is within the rules and it works--awesome! In the end, just a game.

When the Q fund concept was first kicking around, I thought the idea was that user's retail trading strategies would somehow align with the needs of the fund. In fact, I recall that Q was looking for users to share their real-money trading results (with the money coming from their own pockets). The Q open was meant to be a means for folks without lots of cash to still be able to be in the running for an allocation. Now we are at $10M minimum in capital, and strategies dealing in large portfolios that are probably not in alignment with the user's retail trading practices. Any thoughts on going back to the idea of supporting low-capital retail trading? There is a gross mis-alignment between the Quantopian retail side of the business (Robinhood?) and your push into institutional-style scalable strategies. Was it simply concluded (hopefully with data) that your user's retail algos wouldn't work? You'd never reach your $10B AUM goal, even if you could scale the user base to the sky?

Correct me if I'm wrong, but:

1. Beta is cheap to buy. So why impose a) beta -0.3 to +0.3 limitation, and b) short position requirement, when Quantopian as a fund can hedge out market exposure itself from any algo to meet the desired beta target? In the end, Quantopian will pool a certain number of algos into a portoflio, each of those algos will have a certain beta, and portfolio of algos will have its own overall beta. So why can't Quantopian itself hedge out beta (with short market position or a long negatively correlated asset) on a portfolio level?

2. If beta -0.3 to +0.3 limitation and overall market neutrality is so important for Quantopian, why SPY is used as a benchmark by default? Why cash is not used for benchmarking then?

3. If we are here to beat the market, why the contest rules do not take into account performance-per-unit-of-beta type of measures? For example, why Information Ratio is not a criteria for ranking algos?

My speculation is that at some point over the past year, Point72 said "Show me X and we'll provide Y." I ain't no business guy, but if a customer shows up with $250M, it is a no-brainer that their need becomes your need, and by extension, the need of anyone participating in the contest and trying to get a fund allocation. The risk is that what the customer thinks he needs is not really what he needs, but everyone falls in line since he has money. One way to mitigate this lemming-effect risk would be for Quantopian/Point72 to support a channel for non-conforming algos, under the assumption that this whole formula of scalable, institutional-grade long-short equity could end up a disaster (or at least under-perform, relative to alternatives). The thing that is not discussed in any of the press releases is what Point72 expects to pay for work by Quantopian quants, relative to full-time ones. The cost of having an on-staff, full-time experienced guy with an advanced degree sitting in an office in the U.S. is very high ($300K per year? Maybe more in expensive locales). Why not outsource globally? Hey, how about global temp workers that we don't even have to pay? Quantopian to the rescue! In fact, if the workers are also users and do lots of retail trading, they'll be paying to work, effectively (since I have to think that Quantopian will need to profit in some fashion from its "free" trading, if it isn't already). So, it is basically a cost-benefit-risk analysis. The algos are black-box from unknowns around the world. Their performance may not be super-duper, but they are cheap and their are lots of them, so in a relative sense, it is a great deal for "alpha."

One risk, presumably covered by paperwork signed by Quantopian managers, is that they have not utilized information proprietary to others (e.g. covered under NDAs, terms of their present employment, etc.). Or insider information? Eventually, there may be a manager who gets sued/investigated, along with Quantopian/Point72. But I'm guessing the lawyers have thought this all through, and have means to ensure that managers are on the right side of the law, or at least that Quantopian/Point72 aren't to blame.

It would be interesting to hear from Fawce and Steve Cohen on how they are approaching the business and legal end of things, but I'm not gonna hold my breath. Am I on the right track? Or totally off-base?

Dan,

" Everyone who has read our commentary about the contest knows that we are looking for long/short algorithms with low exposure to the market. The nominal position is a fig leaf, and that fig leaf doesn't cover much. The algorithm is short "enough" to get a badge on the leaderboard, but it is not hedged by any reasonable measure."

Then you exclude 95% of retail quants.

What do you mean by::

Short enough
hedged by any reasonable measure

The only effective hedge is via options or futures. Do you offer that? Long-short is not a hedge.

Anthony FJ Garner and Mikko

The 50/50 or 60/40 strategic allocation in stocks/bonds is the most stable hedge historically. Most other long/short allocations suffer from non-stationarity of statistical properties.

Limiting research to long/short algos with low beta motivated by contest rules severely limits the scope of Q platform.

In my opinion, the only restriction should be low beta. Of course, Q may have its own objectives and this is to be respected.

Michael
The so called "Dual Momentum" strategy or the "binary" and 50/50 strategy outlined on my website.

Binary = all bonds / all equity depending on momentum indicator, monthly re-balance. 50/50 same thing but 100% bonds or a max 50% equity/50% bonds.

Tests well since 1753 in the UK and 1870 in the US. Also tests well more recently on iShares Country funds.

But I suspect you mean the more common simple monthly re-allocation to achieve re-balancing back to the relevant proportions?

I share your suspicion of all such strategies but equally as I have often said, the stock index itself is a systematic strategy in its own right. And if the stock index does not work, then I'm not at all sure what one has left?

Dual Momentum

And if you are not a believer in deep learning, ANNs etc take a look at Q "rival" Numerai

I'm certainly going to submit some forecasts however sceptical I might be.

Anthony,

The problem with momentum backtests is that they reflect gains from periods when momentum was not employed and investing was mostly value-based. Therefore, the backtests are unrealistic because they do not account for the impact of momentum strategies on prices but only extract post hoc alpha from value investing effects on prices, which are now basically absent. I am not sure if this is clear.

I looked at Numerai but I have objections about their use of probability since I do not understand to which of the four definitions of probability this corresponds to. See for details: http://www.priceactionlab.com/Blog/2016/03/traders-abuse-probability/

When I use something along the lines of probability in my p-indicator, that is a deterministic calculation I call a directional bias. Machine learning gives a stochastic calculation with high variance based on training set, algo structure,properties, etc. Each time you run a learning algo you will get a different probability or model for it for same training set. The high variance means that the actual probability is 50%, i.e., a coin toss. This is no different than the odds of classical TA with chart patterns, MACD, etc. It is an exercise in futility. If trading was a matter of probabilities only, all math professors would be very rich. On the contrary, professors have failed, often with a big bang (LTCM) The main problem is the variance and the fact that although probability exists as a limit, we may not have enough data to know its value and actually we do not. Therefore, probability measures, as with my p-indicator, can only be useful in the context of discretionary trading and when considered in specific context, rather than a form of systematic trading. Again, I am not sure this is entirely clear.

Out of sample metrics of disqualified algo:

annRet_pt 20.29%
annVol_pt 7.58%
sharpe_pt 2.48
maxDD_pt -3.37%
stability_pt 0.93
sortino_pt 3.66
beta_spy_pt 3.16%

corr_pt 0.07

Is it not hedged enough by any reasonable measure?

Vladimir, stats are good but it is my understanding from email exchange I had today that they are now looking for long-short algos only, possibly due to requirements from investors. How long was your out-of-sample and how many trades were placed?

Fair point Michael re lack of TF back a few hundred years. However my more recent tests on iShares seem to suggest the idea of switching risk on and off still works well to achieve better risk adjusted returns than the indices. And for investment purposes what are the alternatives? Active fund managers are always long and so DD is every bit as bad as B and H of the indices. Of course if it ceases to work, as it might, you will be faced with the inevitable whipsaw losses which can be devastating. No easy answers other than, perhaps, very wide diversification over asset classes and modes of investment.

Michael,

How long was your out-of-sample and how many trades were placed?

It was in contest paper trading out-of-sample for 6 month.
There were 54 trades.

I agree that they should focus on improving their ranking criteria. 0.86% is close to zero returns.

Antony Jackson, I don't think so. My algo ranked 6 has a sharp ratio = 3.399 , wich is bigger than the 5 algos ahead of me.

From the contest results, I guess the most important things for Quantopian are the low Beta, low Volatility, and the Stability. but what's the point
of your algo if it doesn't make money.

A has 100% CAGR and 50% volatility. B has 10% CAGR and 5% volatility. A and B have the same Sharpe ratio of 2.

There is a widespread confusion about performance metrics vs. metrics useful in statistical inference. Sharpe is not a very useful performance metric. SPY buy and hold total return since inception has 0.5 Sharpe yet a few not data-mined strategies can beat that.

Sigh... The contest is an opportunity for Quantopian to reflect back to the community quantitatively which algos are attractive for the fund. Without this sort of feedback, it is kind of a dud, in my opinion. Presently I'm ranked 53 in https://www.quantopian.com/leaderboard/23 out of 590 entries, but what does this tell me? Am I anywhere close to getting anything, or not at all on the path to the land of milk and honey?

It seems that the fund team should chime in, and re-rank according to what they see as potentially (not necessarily) fund-able. Sounds kinda high-touch, but it would only be a monthly exercise.

"it is all about beta"

Maybe you are correct but this tell me nothing. MOM, the neutral momentum ETF lost 18.3% in 2016.

I'd be interested to know how Q look at long/short strategies that keep most, say 90%, of assets in cash-like short maturity bond ETFs in order to decrease volatility while still utilizing the excess "cash" to generate stable returns, utilize long positions with 9% to generate alpha, and hedge the market risk with 1% by shorting SPY. This kind of strategy could have everything they're looking for, ie. it's low volatility and highly stable, generates high sharpe and low drawdown, and has a beta of 0.00. However, it might have large exposure to a single asset (short-term bond ETF), and looks like the short position is only nominal (1%).

Also, instead of shorting SPY, the market risk could be negated by longing an inverse S&P 500 ETF or implied volatility ETF such as VXX. I assume such strategies would not be welcome since they appear to not contain any short positions, despite being hedged?

Three years ago, I was happy (my algorithm won the third prize in the Quantopian Open 14 contest and was ready to get first prize in 15 contest) and at the same time unhappy (in retrospect all three of my algorithms were disqualified as insufficiently hedged using the “fig leaf” indicator in favor of those participants who built high-quality algorithms) with this verdicts:

The nominal short position is a fig leaf, and that fig leaf doesn't cover much.
Going forward in Contest 15 and beyond, these unhedged entries, the entries with only a nominal short position, will be disqualified.
We think that this path is the best, balanced outcome that is available to us.

After the review process of Contest 14, we have determined that your algorithms in the Quantopian Open are not truly hedged.
As a result, your submissions have been disqualified from contests 15-21 (Aug 12, 2016, 2:44 PM).

Since than a disqualified algo in live trading on 10M initial capital doubled the money, 2.5 times beat the market with Annual return 25.8%,
Sharpe ratio 2.01, beta 0.43, alpha 0.19, on leverage 1.0.

Live trding satistic as of 08/14/2019:
Return 103.89
Alpha 0.19
Beta 0.43
Sharpe 2.01
Sortino 2.91
Volatility 0.11

Not considering possible leverage:
The investor missed the opportunity to earn 8 million for 10 million investment.
Quantopian missed the opportunity to get 1 million in balance sheet.
I lost the opportunity to earn 1 million in royalties.

Who did win?
Was the algo really unhedged?
Was "the best, balanced outcome" really the best?

I hope for comments from those who participated in the decision (there is not much left at Quantopian).
May be those contestants who built high-quality algorithms will share their live trading results for the same period?

Below is notebook "A disqualified algo: out of sample backtest and live trading performance".

Wow, interesting (and interesting original post)! Looks like your algo performs really well in bull markets, but how would it have performed had we had a market crash? Are you able to run a backtest from 2007-2010?

@Vladimir, why are you not able to fully hedge your strategy? Surely with 0.19 alpha, long-short hedged you should be able to achieve at least 0.095 alpha, which is quite respectable still. Feel free to PM me if you'd like to collaborate on hedging your strategy.

Also, did you never consider selling your strategy elsewhere to a fund that has a more aligned risk profile?

Joakim,

Sure, here it is.

Thanks Vladimir! Looks impressive, though it's difficult to see how it performed during the years I specified (the GFC period essentially).

Viridian Hawk,

At the time of disqualification hedging requirements was:
"Your algorithm must be hedged to the market.
It should hold both long and short positions simultaneously, or be entirely in cash.
Hedged strategies reduce their market risk and correlation risk to individual positions."

This algo met all of them in 6 month paper trading:

It was hedged by long position in US bonds (beta_spy_pt 3.16%).
It had both long and short positions any time.
It has correlation to the market (corr_pt 0.07).

annRet_pt 20.29%
annVol_pt 7.58%
sharpe_pt 2.48
maxDD_pt -3.37%
stability_pt 0.93
sortino_pt 3.66
beta_spy_pt 3.16%

corr_pt 0.07

Was it not hedged enough by any reasonable measure?

Vladimir, it sounds like a great strategy, just not a good fit for the financial product Quantopian is currently selling to their investors. For example, your beta of 0.43 is too high for the Q Fund. You can do one of two things:
1) Adjust the strategy such that it meets Quantopian's current requirements, or
2) License it to somebody else, make your $1mm and be happy.

@Vladimir, what's your Size exposure? If you're trading mostly microcaps your returns might be coming mostly from unrealistic fills.

Joakim,

Quantopian Open had no restrictions on Tradable Universe, Optimize API, Sector Exposures, Style Exposures, Turnover, Position Concentration, Net Dollar Exposure.
It allowed leverage up to 3.0.
So I used as tradable instruments 3 most liquid not-leveraged ETF and leverage 1.0.
With default slippage and commission model I never get any warning of unfilled orders neither in backtest nor live trading.

Here is backtest results you requested.

I see, thanks. That's quite impressive!

What about my questions?

Who did win?
Was the algo really unhedged?
Was "the best, balanced outcome" really the best?

I hope for comments from those who participated in the decision (there is not much left at Quantopian).
May be those contestants who built high-quality algorithms will share their live trading results for the same period?