@Grant: That's a fantastic idea.

Here is a paper I found very interesting: http://alphapowertrading.com/papers/OnLinePortfolioSelectionMovingAverageReversion.pdf

A tongue in cheek summary:
The algorithm works under the assumption that stocks revert to their mean and continuously rebalances its portfolio based on whether a stock decreased or increased at t-1. There is a nice overview of related algorithms, most of them assume that the stock-price at t-1 will revert to the same value at t+1 (e.g. $2, $1, $2). Obviously this doesn't happen very often: a mean-reverting price might look like 2, 1, 1, 2 in which case you would lose money.

The new contribution is to predict that the price will instead revert to its moving average, in which case you can also turn a profit in the 2nd case.

I think this should be fairly straight forward to implement in Quantopian. It is also a recent publication. Finally, I'd be really curious if we can replicate their results ;).

Thanks Thomas,

When I get the chance, I'll read it. Seems like there could be an overall upward or downward trend...perhaps a simple moving average is sufficient to account for this, versus a more complicated least-squares polynomial fit, for example. My intuition is that if the price versus time can be approximated as linear, then the moving average will work. If it is quadratic or higher order, perhaps the model falls apart?

@Grant: That's an interesting theory. Being able to simulate some stock prices with e.g. quadratic or higher order price changes to test it would be useful here.

My gut feeling however is that trying to fit anything higher than linear will easily overfit, but maybe you have other experiences on this?

Thomas,

It's been awhile, but there are clear-cut accept-reject statistical criteria for limiting a fit to a specific polynomial order (works for multivariate fitting, too). I'll see if I can dig something up to pass along. My experience comes from the response surface methodology used in manufacturing process improvement, but it is quite general and I believe could be applied to quantitative trading algorithms (e.g. for extrapolating a trend). Overfit should be avoided and the assessment can be done statistically.

Thomas,

There is a discussion on Wikipedia of using the p-values of the polynomial coefficients to determine their significance (see "Example with real data" here). So, in fitting a security attribute (e.g. price) versus time, one could select the order of the polynomial based on the p-values. Over a wide enough time window, the price versus time trend should be quadratic- or higher-order.

@Grant That's a good idea. Alternatively one could use the Akaike Information Criterion (AIC) for model comparison (i.e. linear vs quadratic). The key is that more complex models can always fit the data better which is why AIC penalizes the complexity of the model.

I would expect, however, that even with a good fit it might be impossible to make accurate predictions over a wider time window.

In any case, I hope the batch transform ships soon so that we can actually explore all these ideas :).

Thanks Thomas,

I started a "plug and chug" approach to a Quantopian implementation of the algorithm in the paper you shared (I'll have to do some homework on the Langrange multiplier technique). Perhaps you (or somebody) can clarify something for me. On Page 5, under Algorithm 2, Step 6, a normalization is required. What does the notation mean? How would I implement it in an algorithm?

Thomas,

There seems to be some guidance on the normalization in the MATLAB code here and in this referenced paper.

Hi Grant,

Are you referring to: b_t+1 = argmin ∥b − b_t+1∥² ?

The way I understand this is that they have to do this because the b_t+1 from the line before is not necessarily in the portfolio-simplex (i.e. the space of valid portfolios; here, their constraint is that the b has to sum to 1 and that all entries are positive). Graphically speaking, you can imagine a sphere in a 3d space. All points on the surface of the sphere are valid portfolios to use. However, if the optimization in the prior line selects one that is outside (or inside) of that sphere, what do you do? Here, they look for the closest point that is still on the sphere (by finding the one with the closest distance) and choose that instead. They "project" it down to this subspace.

This is also related to the Lagrangian multiplier. If you e.g. want to minimize a function you can just take the derivative and set that to = 0. However, what if you only want to find solutions that sum to 1? The derivative doesn't care. What you can do then is add Lagrangian multipliers that enforce this constraint if you do the derivative. I'm not 100% sure but I think they use it here to enforce that b sums to 1. They do not constrain that all entries in b are positive however which is why you have to normalize it. This however they already taken care of.

Now the question is how to normalize. They don't go into a lot of detail here but point to this paper:
Efficient projections onto the l1-ball for learning in high dimensions, John Duchi, et al. ICML 2008.

In the MATLAB code you posted there is a reference implementation (simplex_projection.m). I would probably have to take a look at that paper to see what's going on, but maybe we can get away with just porting that code to python ;).

Let me know if any of this is unclear or you have any other questions. This is fun :). If you want I could also try to port the normalizing function for you to import.

Finally, I think Quantopian/zipline really needs a function to rebalance a portfolio. You give it a vector of your your prior portfolio and what it should look like and it issues the buy/sell orders to achieve the desired portfolio. What do you think?

Hello Thomas,

The math is kinda beyond my experience, but I can probably hack my way through things since I found the MATLAB code (and another paper by the same author). When I get the chance, I'll take a look at the simplex_projection.m, etc. to see if I can sort out how to get it all to work in Python/Quantopian.

Regarding the "rebalance a portfolio" function for Quantopian, makes sense to me to have it as a standard, canned offering.

Another related reference:

http://www.springerlink.com/content/vx7j46h8848085tu/fulltext.pdf

@Grant: Yeah, that seems to be the predecessor. The other one I posted above seems like a smaller change on top of this. They do have the same problem of requiring the renormalization step. Has that caused any problems for you? I'm happy to give it a shot if you'd like! Also, feel free to just share what you have so far and I see if I can push it a little further.

Thomas,

Here's what I have so far:

#  
# Algorithm based on this publication:  
# http://alphapowertrading.com/papers/OnLinePortfolioSelectionMovingAverageReversion.pdf  
# 

def initialize(context):  
    context.stocks = [sid(8554),sid(19920),sid(22739)]  
    context.price = {}  
    context.x_tilde = {}  
    context.b_t = {}  
    context.b = {}  
    context.b_norm = {}  
    context.eps = 1.1  
def handle_data(data, context):  
    x_bar = 0.0  
    m = len(context.stocks)  
    sq_norm = 0.0  
    dot_prod = 0.0  
    b_tot = 0.0  
    # find relative moving average price for each security  
    for stock in context.stocks:  
        price = data[stock].price  
        x_tilde = float(data[stock].mavg(3))/price  
        if data.portfolio.positions_value > 0.0:  
            b_t = data.portfolio.positions[stock].amount * price  
            b_t = b_t/data.portfolio.positions_value  
        else:  
            b_t = 1.0/m  
        x_bar = x_bar + x_tilde  
    x_bar = x_bar/m  # average predicted relative price  
    for stock in context.stocks:  
        sq_norm = sq_norm + (x_tilde-x_bar)**2  
        dot_prod = dot_prod + b_t*x_tilde  
    lam = max(0,(context.eps-dot_prod)/sq_norm)  
    for stock in context.stocks:  
        b = b_t + lam*(x_tilde-x_bar)  
        b_tot = b_tot + b  
    for stock in context.stocks:  
        b_norm = b/b_tot  # new portfolio  

The normalization is not per the paper. Feel free to take a shot at porting the entire algorithm to Quantopian...then perhaps you can explain the theoretical/mathematical details to me! I'll keep plugging away at trying to understand what's going on here. I think that I can understand the Lagrange multiplier business, but the re-normalization seems trickier.

@Grant this is awesome. I made a backtest of your code (plus one line at the bottom to pass validation), so others could more easily clone it: https://www.quantopian.com/posts/on-line-portfolio-selection-from-grant-k

Hello Thomas,

Please see John's link above...I'll post some questions there.