SPY & SH algorithm - please review

posted Oct 25, 2013

Good idea? Bad idea? What's missing? --Grant

21 responses

Grant,

I think it's a great idea and love your capital_invested function

Though, and this is going off of what Peter Millington said last night at the Quantopian meetup, it may be more robust to use more than one quantitative indicator. I'm thinking along with the z-score threshold add in a moving average indicator, or use of bollinger bands for the S&P. Definitely just throwing ideas out there and I could end up being completely wrong! I'll get a version of mine uploaded soon and let's see if we can keep this going

-Seong

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Seong,

Dennis C. contributed the capital_invested function (see https://www.quantopian.com/posts/spy-and-sh-z-score-indicator). This is not the first time I posted the idea...thought I'd revisit it to see if anyone has interest.

Yes, please feel free to play around with the algorithm. It's sorta curious that the return really takes off after July 2011. Any idea why? Was there some fundamental shift in the market?

Grant

Regarding adding additional parameters/degrees of freedom, I'm kinda wary, since over-fitting is a risk. However, adaptively optimizing the existing parameters would be interesting (e.g. window length and z-score threshold). --Grant

Grant,

I'm very curious about how we can go about overfitting (I'm going through http://www.autonlab.org/tutorials/overfit10.pdf and thought it might be useful to share). What I'm thinking is that balancing a number of parameters offers consistency yet too much does pose a high risk of overfitting // will spend some time looking into it more as having some sort of sanity check on algos would be nice

As for the July 2011 I'm as curious as you! Maybe we are just being 'fooled by randomness' as Taleb would suggest

-Seong

Disclaimer

Seong,

Here's a quick "poor man's" attempt at adjusting the threshold adaptively:

    if retrns < context.retrns:  
        threshold = 2  
    else:  
        threshold = 1  
    context.retrns = retrns

Of course, the more complex the algorithm gets, the greater the risk of getting fooled (especially for me, since I'm generally clueless about trading).

In any case, I'm kinda intrigued that just trading in SPY and SH and using no external signals, one could beat the market.

Thomas W. has brought up the idea of walk-forward optimization a number of times...maybe this would be an opportunity to try it?

Grant

Hi Seong,

Here's an extreme case (with borrowing on margin)...seems like it's printing money, but there must be a "gotcha!" (or a mistake). I welcome your feedback (or if it's real, fund it and buy yourself a sports car).

Grant

Hi Seong,

It would make sense to do an offline analysis of SPY & SH, using data from Yahoo or elsewhere. I think that this will be the only way to get an understanding of what's going on here. Naively, I'd think that SPY & SH would be "joined at the hip" but the backtest results suggest otherwise.

Grant

Grant ,

From my understanding, I would think that the SPY and SHY would be inverse of each other as bonds and stocks generally follow a pattern (e.g. when equity markets dips bond market rises as investors seek a safer asset) perhaps 2011 suggests that investors "came out of their bond shell"

-Seong

Disclaimer

Not SHY, but SH (ProShares Short S&P500 (ETF)). This is the mystery, since "ProShares Short S&P500 (the Fund) seeks daily investment results that correspond to the inverse (opposite) of the daily performance of the S&P 500 Index (the Index)." My intuition is that sometimes they get out of whack (to use the technical lingo). --Grant

Bernd N

Well, it is quite impossible to replicate the opposite of the S&P. First of all they need to borrow the stocks which they are going to short, which costs money or sometimes isn't even possible (2008 crisis for financial stocks for example). Secondly, returns over a longer period of time are multiplicative, not additive. So rebalancing is required, although I'm not sure if this is relevant for an 1x short index. For higher leverage ETF this eats away returns. See here an example where I go short SH vs the benchmark (I assume that is the S&P 500). It outperforms because SH goes down faster than SPY goes up.

Oct 26, 2013

Thanks Bernd,

I modified your code slightly to make it easy to switch from short SH to long SPY. Here's the long SPY backtest, for comparison.

I'm perplexed by the result, and wonder if something is missing from the backtest. Is there an additional cost associated with shorting not captured in the commissions model? Or is the return simply higher with the SH short because there is more risk (since SH is actually an investment in derivatives, as I understand).

Grant

Nov 7, 2013

Here's another installment with a cointegration test:

c_t = cointegration_test(dv[:,0],dv[:,1])  
    if c_t[1] >= 0.05:  
        return  
    elif c_t[0] >= c_t[4]['1%']:  
        return

def cointegration_test(y, x):  
    result = stat.OLS(y, x).fit()  
    return ts.adfuller(result.resid)

Grant

Nov 8, 2013

Here's a version w/ QQQ & PSQ. I have no idea if any of this makes sense...just tinkering. If anyone has insights or improvements, please share. --Grant

Here's the algo run on minute bars, with orders limited to once per day (it appears that there may be a little slippage, since some days I see multiple fills). I have not done much optimization. And there may be other problems, but I figured I'd get it out in case someone would like to contribute.

Any guidance on how to speed up the backtesting would be appreciated. This backtest took ~ 40 minutes to run.

Grant

Grant,

I believe your batch_transform function is what's slowing you down so much. Is it your intention to get minute prices and volumes?

-Seong

Disclaimer

Thanks Seong,

The batch transform may be the problem. Yes, I'd like to use minute prices and volumes--is there any alternative in a minutely backtest?

Grant

Grant,

I believe there is, but to confirm, you want to take minute prices and volumes over 15 day lookback period correct?

Disclaimer

Yes, 15 days (390*15 minutes). But my hunch is that some sort of smoothing/averaging would help (beyond what is done already with the z-scores). --Grant

Nov 23, 2013

Grant,

I believe that using an array to store your past prices instead of using batch_transform would be a lot faster (this will all change with the history api). And you could make sure the size of the array never goes past 390*15 by insuring that

if len(array) > 390*15: del array[0]

Would something like that fit what you're looking for?

Disclaimer

Nov 23, 2013

Thanks Seong,

I'll try building my own accumulator, and will look forward to the new history API.

Any idea of the most efficient way to write the accumulator? Seems that I should be looking at a numpy ndarray. The issue, though, is that I shouldn't be shuffling everything around in memory every minute, right? I've never used a deque; would it be better?

Grant