Hello Tim,

What have you tried? Some tips from a hack:

1. My sense is that an algo needs to treat long and short on a equal footing. You can't sorta fix up a long-only strategy by adding in shorting on an ad hoc basis.
   
2. I've noticed that as I reduce the number of stocks in my universe to below about 50 (w/ $1M initial capital), volatility starts to creep in. And concentration in only a handful of stocks is a problem, too. I don't know of any general rules here, but it is something to consider.
   
3. One thing to try is using an ETF to null beta. If your stocks are drawn from the general market, then try SPY. Or use a basket of sector ETFs. This allows you to go long, long/short, or short in company stocks, and then make up the difference (either long or short) with the ETF. At any point in time, your beta should be relatively low, without having to find N stocks to long and M stocks to short, in equal proportion.
   
4. Try setting commissions to zero and turning slippage off, so you are not confused by their effects.
   
5. Use a gross leverage of 1.0, for development.
   
6. Backtest as far back as possible, and try to ignore recent performance.
   
7. My sense is that if you can get a beta-neutral strategy simply to eke out a consistent return (even if it is zero) over 10 years or more, you can then tinker with it to see if it can be improved. I recall that when Quantopian kicked off the fund concept, they said they would accept a Sharpe ratio as low as 0.7, with annualized returns as low as 7% (presumably at a gross leverage of 1.0). So, you don't need to achieve rock star status.

Cheers,

Grant

Thanks, Grant,

Your advice makes perfect sense and in fact I do stick to these guidelines and have also tried many of the approaches that you mention.

I have attempted to use the long and the short leg on equal footing, as well as shorting SPY only to reduce market exposure. In all the cases the drop in the performance of the algorithm in term of returns was so bad that it rendered it useless.

I think the problem may be that momentum strategies that I have been employing simply do not work with shorting, as pointed out also by Andreas Clenow in another thread. And I have scoured the web in vain for tips on criteria for shorting based on the fundamentals.

That leaves pairs trading I suppose. I still need to get to it, though ...

Regards,

Tim

I have the same trouble with shorting. I think the problem is that it is difficult to find alpha or excess return relative to the market. If your strategy moves nowhere with an equal long short strategy, it probably means you're just betting on beta or playing with stocks more or less volatile than the market. Thus, positive beta vs negative beta in a long-short strategy will leave you with zero beta and returns.

I believe you are right, Minh, a meaningful short strategy should be able to yield good returns on its own, i.e., when the long leg is not even present. Otherwise, what's the point? Sure, you can get smaller drawdowns in times of crisis, but this can be more easily and more efficiently achieved by applying an absolute momentum criterion (filter) to a market index ETF, for example, keeping the entire approach long.

In this sense, I have never been able to come up with a meaningful short strategy. I guess it simply means that I am not good enough at this entire game...

You might also consider the time scale. For example, try trading every day, using smoothed minute data (with no commissions, and the slippage model disabled). There could also be a time-of-day dependence. Basically, you have frequency of trading, degree of smoothing, and time of day to explore. Recently, I kinda stumbled into something that may be decent by backtesting with daily trading, but then found I could trade less frequently.

Assuming momentum works for rising prices (or dollar-volume) and dropping prices (perhaps differently depending on the direction?), then there is the problem of how to weight each stock. Say you have 50 stocks (selected by some criteria?). Which should be long and which short? And how should the portfolio be weighted? How to maximize the profit, but maintain diversification? Etc.

Thank you again, Grant, for your guidance, I should definitely try the varying time scale approach that you suggest.

As for selecting which stock to go long on and which to short, I currently sort the stocks according a rough approximation to the smoothed first derivative of time dependence of the daily close price. The top (positive) 10 values indicate a long position, the bottom (negative) 10 a short one. I rebalance monthly with equal weighting of the stocks in the portfolio.

Tim -

Yeah, you might try more frequent trading, just for yucks. And you can vary the look-back period, too. For example, trade every day, trying look-back windows of 1-20 trading days.

Also, as I suggest above, try not using daily closing prices, which are simply the last trade of every day. I've tended to synthesize prices from minute bars, using a Pandas smoothing function (e.g. prices = pd.ewma(prices,com=390)).

Intuitively, on a monthly time scale, I have to wonder if mean reversion is killing you. Picking 10 long and 10 short "screamers" and then hoping that they keep trending over the next month might not be valid. Rather, one might expect the stocks to relax back to more realistic (mean) values. I ain't no expert though. Maybe somebody who actually knows what they are doing can comment?

@Grant,

I'll certainly try to implement your suggestions, but let me just explain that although I rebalance on a monthly basis, I use a lookback window of a year (daily close prices). The method actually works pretty well with a long leg only, especially when coupled to an absolute momentum filter (signal) for the market as a whole (using SPY as a proxy). When times are bad, the algo simply invests fully into government bonds.

I see. So when times are bad (measured how?), you could go short instead of going into bonds, but you would not have a long-short algo suitable for the Q contest/fund.

Again, no expert here, but my sense is that you won't be able to do a long-short algo for the contest/fund with a year lookback window.

I figure momentum trading is the opposite of mean reversion. Maybe stocks that tend to trade well above their recent mean prices on a consistent basis are momentum-ish? I guess there needs to be some way to determine whether a given stock will revert to its mean or continue higher, shifting the mean?

For deciding the market direction, I use the same momentum calculation as for the ranking of the stock, but I apply it to SPY and simply look whether the value is positive or not.

To look for rising stocks I take the ratio of a short and long MA. For spotting mean reversion behaviour one would need to have a less crude measure of the first derivative and and look at its shape, rather than just the current value, I guess.

I am attaching a backtest of the long-only algo, just as an illustration.

Sorry, the algo above actually takes the average return as a measure of stock performance.

Here's the algo that uses the ratio of two MA's.

The performances of the two algos are identical.

For what it's worth, here's a beta-neutralized version, using 50 stocks (instead of 10). --Grant

Howdy Tim,

I agree with Grant on a shorting period of 1 month is too long. Try your strategy for a shorting period of 5-8 trading days.

Thank you, Toan, I shall certainly heed yours and Grant's advice.

There's a technique I picked up from http://icml.cc/2012/papers/168.pdf, described briefly in the last paragraph of section 4.3. One approach is to pick a maximum trailing window length, say 5 trading days. In a loop, you would look back 1 day, 2 days,...up to 5 days. For each look-back, compute the expected (forecast) return, r1, r2, r3, r4, & r5, and the respective portfolio weight vectors, w1, w2, w3, w4, w5. The portfolio vector for your rebalancing then becomes:

w = sum(ri*wi)/sum(ri), where i = 1 to 5

It is simply the weighted average of the portfolio vectors. The returns are relative, and thus ri >= 0.

Splendid idea, thanks Grant!

Tim.

If I'm correct your algo is allocating random instruments. See these lines

def rebalance(context,data):

    P = data.history(context.secs, 'price', 100, '1d')  
    V = data.history(context.secs, 'volume', 100, '1d')  
    H = data.history(context.secs, 'high', 100, '1d')  
    L = data.history(context.secs, 'low', 100, '1d')  
    s2 = np.log(H / L)**2  
    z = P * V / s2  
    x = z.tail(10).median() / z.median() - 1  
    w = (P * V).median()  
    w = w[w > context.portfolio.portfolio_value]  
    x = x[w.index]

    w = P.tail(10).median() / P.median() - 1  
    w = w[w > 0]  
    x = x[w.index]  
    w = V.tail(10).median() / V.median() - 1  
    w = w[w > 0]  
    x = x[w.index]  
    w = s2.tail(10).median() / s2.median() - 1  
    w = w[w < 0]  
    x = x[w.index]

    # record(n = len(x))  
    longs = x.order().tail(10).index

the last order sets NaNs at the end of the list and therefore you are allocating into instruments that have NAN values in X. This is also the reason your two examples have exactly the same returns.

You should change the line to include dropna() if you don't want to use NA values.

longs = x.dropna().order().tail(10).index  

Here s backtest with dropna() included.

Mikko,

I owe you a big thank you for this crucial insight.

Regards,

Tim

In Andreas Clenow's book, Following the Trend: Diversified Managed Futures Trading, he talks about shorting in momentum strategies that kind of pertains to what you have been talking about if I remember correctly. The thing that surprised me was when he broke down the long and short sides of the strategy detailed in the book. The shorting side barely made any money and when it did it only about 20-25% of the time and during sustained market downturns. I want to say the returns were on average around 2% annually. The point is the shorting side many times doesn't increase overall returns particularly during the market conditions you can test on Q's platform (which in my opinion lean maybe a lean little too much toward the bullish side).

That being said what Grant said up towards the top of the thread regarding using the sector as a equal weighted short seems to work well to reduce volatility. The key thing the consider is to try to find a ETF that is most correlated with your holdings. By finding a correlated ETF to your holdings it will work best as a hedge to your strategy sometimes this is can be SPY sometimes it will be IWM, QQQ, or maybe a lesser known and trading ETF it just depends on what market segment your strategies holdings are focused around.