From what I've seen so far with testing fundamentally based algorithms, equal weights aren't very effective at generating alpha and weighting positions can significantly improve the algorithm. Haven't seen much discussion on investing sites about weighting stocks in the portfolio but you could try something like weighting stocks based on their rank in the list or weighting stocks based on the number of quarters in which they passed the screen.

Thanks! One thing though - don't these choices for weights seem a bit arbitrary? I mean, the choice of filters is always logical and based on sound financial theory (e.g. low P/BV ratio => stock undervalued => screen stocks with low P/BV values).

Similarly, shouldn't there be a logical way (as in logically following from financial theory) to determine weight allocation? The only method for weight allocation I know of is Markowitz optimization, and there are probably similar ways to optimize allocation based on past data, but these optimizations are computationally demanding. Isn't there any research or literature on logically arriving at a suitable, non-computationally intensive, weight allocation scheme?

Maybe there's a way to use machine learning to determine the optimal weights, it probably would be computationally intensive though.

I'd recommend checking out this post that was produced on The Efficient Frontier: Markowitz portfolio optimization in Python. This should allow you to be able to derive a non-arbitrary rule to define your weights.

Also, I agree with Eric. For your example of P/BV, you could apply higher weights to out-preforming/under-performing companies. Possibly, investigate even utilizing a series of bins to separate the companies into their respective buckets, then specify the needed weights with some ranking system. Then, in order to check if the derived weights are beneficial, compare to see if they capture more alpha than an arbitrarily set portfolio.

Thanks for the reference!

Possibly, investigate even utilizing a series of bins to separate the companies into their respective buckets, then specify the needed weights with some ranking system.

By that do you mean deciding on a bin size, say k, followed by dividing the stock universe into k-iles (e.g. deciles if k = 10) and ranking all stocks from 1 to k based on which k-ile they lie in? Also, what exactly do you mean by "arbitrary set portfolio"?

Exactly, you got it. By 'binning' I mean this or this. So you would set an increasing weight towards each bin, which would lead to a higher weight for higher preforming companies.

By an "arbitrarily set portfolio" I was talking about a portfolio where the weights are just arbitrarily set. The reason we would want to compare the two (one being the basic weight portfolio, the other being the portfolio set by a created system of ranking) would be to see if there is actually any additional benefit in doing so.

Removed that algorithm because it used a huge amount of leverage. That wasn't the right way to implement the weighting strategy, the bin method seems much better.

Here's a question:

You are saying that an algorithm gives you a list of stocks to buy, but you don't want to equally weigh them because some are better than others. If you know the criteria that decides which stocks are better than why not adjust the logic such that it only gives you the best ones? If you don't have the criteria that determines which stocks should be more heavily weighted than equal weight is your only option.

You are saying that an algorithm gives you a list of stocks to buy, but you don't want to equally weigh them because some are better than others. If you know the criteria that decides which stocks are better than why not adjust the logic such that it only gives you the best ones? If you don't have the criteria that determines which stocks should be more heavily weighted than equal weight is your only option.

That makes perfect sense. I hadn't thought of it that way! Could you please clarify another thing: continuing with the P/BV example, let's say we only have 4 stocks A, B, C and D in the bottom quintile (on the basis of P/BV value), with P/BV that of A > that of B > C > D. Since the 4 stocks have different values of the ratio, we want to distinguish between their weights. One way to do so is on the basis of ranking, like Eric said. So the weights assigned to A, B, C, D could respectively be 4, 3, 2, 1. But then one could just as well use squared weights, 16, 9, 4, 1 or any other monotonically increasing function of the rank.

Do you think it's advisable to try out different such functions, or would it result in overfitting?

I don't think it would hurt to test a few different weighting schemes, if you have enough stocks and enough time data to get yourself a decently representative sample. If you're using pipeline and hold enough stocks then you should be able to get a good idea of the performance.

Personally I would not do most weighting schemes other than market cap and equal weight unless there's a significant, consistent increase in performance and the scheme has some logical reasoning behind it. As you said, we may overfit, and some weighting schemes may leave you too exposed to one specific stock or industry.

You may find the below code somewhat useful. I use it to prevent overexposure to industries when equal-weighting.

def concentrate(context,data, index,sectors, maxlen=6,persec=2):  

    zipped = zip(index,sectors)  
    trading_list =[]  
    sector_lib={205: 0, 206: 0, 207: 0, 101: 0, 102: 0, 103: 0, 104: 0, 308: 0, 309: 0, 310: 0, 311: 0}  
    for stock,sector in zipped:  
        if len(trading_list) == maxlen:  
            break  
        if sector_lib[sector] <persec:  
           trading_list.append(stock)  
           sector_lib[sector]+=1  
    return trading_list  

Just feed the function a list of stocks you're considering, a list of their corresponding sectors in order (this can be done with a dict much easier, but this is really old code when I wasn't that experienced), the maximum number of stocks you want to hold, and the max number of stocks you want in any sector. If you want to go from a list of say 50 potential stocks to say, 10, be sure to rank your list of stocks based on P/BV first, or whatever your factor may be. This can be pretty easily adapted for market weighting too.

Hello, great discussion. I wanted to jump in with the following thoughts. Most strategy development falls into the following format: maximize expected OOS alpha subject to risk constraints. Phrasing the problem as an optimization problem has many benefits in terms of ease of thought and implementation. Here we define alpha as excess returns that are unexplained by a chosen set of risk factors. The reason we use alpha and not returns is because generally risk factors reflect effects that are cheap to purchase in the market otherwise and are not in high demand. The nice thing about this framework is that you can adjust the risk factors to your preferences. Say you don't care about market exposure, then remove that as a risk factor and you're good to go.

An optimizer will do its best to optimize your expected alpha given risk constraints. But it's only as good as the estimates of OOS alpha and OOS risk. Defining your expected alpha (AKA alpha expression, alpha vector, ...) is what you're doing when you construct a model/factor. Often you accept that your estimate of future alpha will be very noisy. Also it often helps to think of your alpha vector as more proportional to or correlated with true future alpha, as getting it to match exactly is witheringly difficult. Just getting to correlated with future alpha is fine, because then you'll still average out to be okay over many positions. Why do you need risk constraints? Because with no constraints the optimizer would put everything into the one stock that had the highest expected alpha. There's a really good explanation of this here.

So, is equal-weighting better than alpha weighting? It really comes down to your alpha expression. Some alpha expressions are super noisy and you basically accept that you can't do better than classifying as stock will go up or down, which is what equal weight long short is. Some alpha expressions do have more information in them that the optimizer can extract. In general you want to let your knowledge of the alpha expression drive a hypothesis about the weighting scheme that you can then test. You don't want to overfit by trying a bunch until one works. The nice thing about the optimizer framework is that it allows you to swap the alpha expression in and out fairly simply. That said, alphas do have to be designed with constraints in mind. You can't just take an alpha designed to be long only and enforce a long-short constraint. The optimizer will mangle it beyond all recognition and there will likely be no alpha left.

Please see this template for an example of everything discussed. Included are a bunch of constraints around sector neutrality and market exposure. We plan to be releasing updated constraints soon that reflect what Quantopian would like to see in a candidate for allocation. https://www.quantopian.com/lectures/example-long-short-equity-algorithm

I forgot to mention, a great sanity check on whether a weighting has value is just to map it to equal weighting and see if it reduces the returns. If not then you're likely no better than an up/down classification.

@Delaney: Thanks for the post! I will go through the posts that you've linked to in detail. Had a doubt about optimization: I've read from a couple of places that portfolio optimization/tangency methods don't work very well at the level of a single alpha if the alpha is based only on 1 or 2 ideas.

When a portfolio manager has multiple alpha (i.e. multiple algorithms), then in order to combine them to form actual trade orders or portfolio, he/she turns to optimization.

Is this correct? If I were to make a trading strategy that's say, based on only 1 fundamental factor, or only 1 technical indicator, would it be inadvisable to decide stock weights in such a strategy based on convex optimization or tangency methods?

Great question. You are correct, that's why generally the resulting weights are an alpha expression, or combination of alphas. The reason for this is just that in today's market, finding any one edge consistent enough to trade on is very difficult. Individual alphas are too noisy and the optimizer wouldn't work that well. In general you want to combine multiple alphas in order to create a diversified final alpha expression. The more different the component alphas, the greater the diversity of ideas and the better the resulting expression. Mathematically, the reason this works is that if you have many models, each with a signal and noise, combining them averages the signals, but reduces the noise, therefore increasing the signal to noise ratio. We're working on a lecture to explain this in more detail. You'll find that the more alphas you feed into your expression, the better it does. Unless of course you combine alphas that interact in a weird way or conflict with each other. In general any type of trading will be difficult using only one alpha.