Hello Taibo,

A few ideas:

• Did you consider using the batch transform for exposing a trailing window of sid data (see the help page for an example)? Although perhaps equivalent to your approach, it might improve code readability. Note, however, that the batch transform as it is presently implemented is inefficient, so your backtest will get bogged down if you switch to minutely data and call the batch transform every minute.
  
• You might consider using the statsmodels module. It has an ordinary least squares model (see http://statsmodels.sourceforge.net/stable/generated/statsmodels.regression.linear_model.OLS.html#statsmodels.regression.linear_model.OLS).
  
• Regarding the volatility, have you considered doing a statistical check for goodness-of-fit? You could incorporate the goodness-of-fit into your buy/sell decision (e.g. effectively weight the decision based on the strength of the trend).
  
• You might consider placing some realistic limits on your position size. See the example on the help page:

# Setting our maximum position size, like previous example  
    context.max_notional = 1000000.1  
    context.min_notional = -1000000.0  

@Grant: I agree that the batch_transform and statsmodels should make the code more readable. Note that the batch_transform should not be much slower than this approach. The cost of the batch_transform is associated with recreating the dataframe each call. This also happens when using np.insert and np.delete as they don't do in-place manipulation but return a new array.

Thanks @Grant and @Thomas for your ideas. I didn't think much about readability - which I definitely should have. The point here is that the linear coefficient approach can very accurately capture the overall trend of the stock, so I didn't set the limits - certainly in real trading scenarios setting limits would be a better idea.

As for volatility, I actually ordered amount as (coeff*500) - a quantity linked to the overall performance ("weight the decision based on the strength of the trend"). I don't know if that's what you meant, but any other ideas are as always enthusiastically welcomed.

OK, I converted the code to use the batch_transform and statsmodels.OLS to estimate the regression coefficients. As you can see, this greatly simplifies the code. Moreover, despite doing daily updating the batch_transform it runs very quickly.

Thanks Thomas.

Taibo (and Thomas),

I suggest adding a test for goodness-of-fit (see here for an example). Then, one could examine (ideally plot) the goodness-of-fit parameter(s) versus time to get a sense if the strength of the model varies with time. Also, statistics can be used dynamically to select the best model versus time...over some time periods, perhaps only a straight-line fit (or constant) is justified by the statistics, but over other time periods, a quadratic model might be justified. The statistics can be used to pick the order of the general polynomial model.

Also, you might try running the algorithm on SPY to see if you can outperform the Quantopian S&P 500 benchmark with the same or lower risk.

Grant

Out of curiosity, I ran the same backtest as Thomas', but with SPY. Not so good... --Grant

Hello: shouldn't you be shorting when slope < -.10?

Hi Allan,

I was just tinkering around. I didn't dig into the details. Feel free to clone the algorithm, and modify it. Let us know if you can improve it!

Grant