Nice! I remember a strategy to find high beta-bias stocks vs low beta-bias stocks, where one would sort the universe on beta_up / beta_down, long the top percentile and short the bottom. Probably suffers from the same problem as momentum at market bottoms, but I haven't backtested it in a while.

Hello Anony,

A couple quick thoughts:

--The algo doesn't perform very well beyond the "Great Recession" downturn, right? So, I'm wondering what is gained?
--Aside from the bond component, you are dealing with sectors of the S&P 500 (SPY). Would it make sense to measure the volatility relative to SPY? It seems that the overall market volatility should be factored out, to measure the sector-dependent volatility. Kind of a fuzzy thought at this point, but I have a simple algo in mind to see if it makes sense.

Grant

Just a hobby for me, at this point. As you must already figure, I have not found the holy grail. And I do not have the time/energy/experience for more than some sort of plain vanilla monthly/quarterly asset allocation algo. At this point, I'm waiting to see how much that's gonna cost through Quantopian/IB. It might make sense--we'll see. --Grant

Hello Anony,

In your VolBias code, why do you use the (i+1) multiplier:

        upPortion = 0  
        dnPortion = 0  
        span = len(self.Close)  
        for i in range(0, span):  
            if (self.Close[i] > self.Open[i]):  
                upPortion += (i + 1) * (self.High[i] - self.Low[i])  
            else:  
                dnPortion += (i + 1) * (self.High[i] - self.Low[i])  
        factor = upPortion / (upPortion + dnPortion)  
        self.Weight = factor  

Grant

Hi Anony, I am about to take a stab at your very interesting idea. Where did you come up with the initial thought? Also, how did you pick the portfolio? I see you mention something about "Q", what's that?

Hello Anony,

I started to play around with your approach, using my own coding style (see attached for a draft). Would you mind fleshing out a bit the indicator and trading rules? I'll see if I can replicate your result.

Sorry, I'm not yet up on Python object-oriented programming, so with a little guidance I should be able to decipher your code.

Grant

Hi Anony, thanks for the explanations, I'll be porting your algo to my quantshim framework too, as this is the best way for me to learn. Your Security/SecurityMgr objects are actually already similar to mine, so think it should be easy-peasy.

Thanks Anony,

Another detail:

record(Weight=context.SecMgr.GetSecurities()[0].Weight,\  
           Trigger=context.SecMgr.GetSecurities()[0].Trigger)  

Are you just plotting the weight and trigger for one security, as a check that the algo is working properly (and perhaps to estimate your entry/maintenance/exit conditions)? Or is something else going on?

Also, I gather that your linear weighting versus time (the period(0...n) mutiplier) serves to bias the VolBias toward more recent changes, correct? In this industry, I see that this weighting is conventionally called a weighted moving average (WMA) (see http://www.tradestation.com/education/labs/analysis-concepts/a-comparative-study-of-moving-averages). Correct?

Grant

@Anony, do you mind explaining the reasoning/mechanisms behind your slippage model, I commented out the set_slippage line and it doesn't perform as well with the default slippage model.

Okay, that makes sense. For minute data would you say that the default slippage model makes sense because the trades are executed in the following minutes and not a full day later?

Made a little headway on a minute bar version. No trading yet, but it looks like the weight and trigger values are in the ballpark. I decided to use the Pandas canned ewma instead of the homebrew weighted moving average, since it is basically the same kinda thing--most recent events are weighted most heavily.

I'll add some comments to the "cryptic matrix alchemy" just to show there's no magic involved.

--Grant

Anony Mole,
What's the entry and exit logic with the trend variable coding newbie here but v interesting

One possible source of bias is that you've hardcoded the DJIA components, when actually those components change over time. What that might mean is that you've introduced some look-ahead bias into your algorithm, in that you've picked stocks in 2003 that you shouldn't have known would become successful enough to join the DJIA (Cisco, United Health, etc) and likewise you've avoided some stocks which you shouldn't have known would leave the DJIA (Altria, Honeywell, AIG, Citi, GM, Kraft). Ideally for the DJIA component selection to be valid you would have to adjust the DJIA components over time (not too taxing as they don't change frequently, could upload a CSV to Dropbox and import without too much effort).

That said I haven't looked into how much those specific picks drove the results of your latest version.

I think one other thing to consider is whether your slippage model is a reasonable approximation of what will actually happen in live trading. I converted one of your algorithms (the third one you posted in this thread) to trade in minute mode with the default slippage model and configured it to trade at the first possible opportunity - in other words, at the open of the next trading day following the weight and trigger calculations. I ran them side by side for the period 2007-03-09 - 2007-12-31 (I actually ran the daily mode test from 2007-01-01 but due to warm up it doesn't trade until this date, so I delayed the minute mode algorithm accordingly which requires no warm up). I used a relatively small initial budget ($10K).

The performance varied widely even though the weight/trigger calculations and entry/exit parameters were identical (I logged them all). Essentially what happens is that on any given day, the trade price in minute mode trading at the open varies from your "trade at open" slippage model by a range of -1.5% to +0.99%. In this case that means for the first few months of the test, the minute mode algorithm underperforms the daily mode algorithm significantly (average price discrepancy is +0.037% per day in favor of the daily algorithm). The price discrepancy shifts back in favor of the minute mode algorithm for the second half of the test but by then the daily algorithm is already 7% ahead due to the compounded advantage.

I need to clean up the code for the minute mode version and then I'll post it (later today) and I'll also upload the Excel analysis of each individual transaction with price discrepancies detailed.

This has me wondering about having the minute mode algorithm try to trade at the end of the day or near it on the day used to calculate VolBias rather than at the open of the next day, to minimize price drift. I am also wondering about the default slippage model and how realistic it is. Does anyone know how the Quantopian team arrived at the VolumeShareSlippage default parameters?

@Anony, and everyone else

I finally cleared my plate and started looking at your strategy! I started by taking your latest version and ported it to my QuantShim framework. The backtest results don't currently match yours, so I need to work through and find what's causing the discrepancy. That said, the results are approximately the same, so that's a good sign.

My first bit of feedback would be a bit of a worry about the increased trade frequency, why do this? Cutting the initial capital down to 10k (something I could realistically afford for a strategy like this) cut returns from 438% to 190%, on this super-optimized backtest.
I'll give more feedback once I get my port to achieve the same returns as yours. After that, I'll get this running in minute mode.

btw, if someone is interested in using QuantShim, please keep in mind that history values are inserted in-front, which is different from how quantopian likes to show it.

@Anony, as Matt said, I bet there's some survivorship bias. It's time for bed for me, but tomorrow I'll try using the quantopian 10% universe stuff and see how that goes. fyi when I used the 9 sector ETF's my returns were not nearly as good... but again, I don't have the same returns as your implementation yet, so I might have a critical bug somewhere....

How would we go about implementing this strategy with Set_Universe? And has anyone been able to run it in minute mode?

@Nihar: It would be a bit of work to use set_universe with anony's codebase, as he'd need to either move the weight logic somewhere else, or dynamically construct his security objects.

Thankfully, the quantshim port I did will make using set_universe very easy once I get the returns to match Anony's

fyi I found the bug in my port of Anony's "DJIA frequent trade" run (I was missing a parenthesis) so now the returns roughly match. The couple-percent difference I think is due to how my framework dynamically handles rebalancing.

Next I will try running with 10% universe and then get it running in minute mode. After that, I'll integrate whatever optimizations Anony has come up with since :)

@Anony: re your inverted weights, in my testing with various portfolios this only has a slight (if any) impact on returns. for example your initial "DJIA" portfolio has almost the exact returns. I presume this is due to your weights hovering in the 0.5 range to begin with.

I've attached a run using the top 10% universe. This guarantees a bias free portfolio, and the results are significantly worse, but still promising due to the potential of decreasing the excessive trades

However, I don't think I'll be converting to minute mode just yet. the 10% universe took more than 2 hours to run, so I'll play with the other interesting improvement ideas first. Minute mode will take 10x or so longer to test so I think we can do that a bit later :)

Took a stab at converting this to minute mode while maintaining the analysis/execution on a daily basis. Here's what I've changed:

• No custom slippage - uses default slippage
  
• The analysis is still done daily, but at a time that can be (eventually) controlled: right now, security manager update is done once a day early during trading. If any orders are to be placed, they're placed at the same time early in the day (this can be modified to place buy orders early and sell orders nearer to close times etc)
  
• For the analysis, it calls on the history function to load the previous day's final ohlc data.
  
• Modified the security manager to take symbols instead of sids (my own preference really and to understand when some securities started - will set back to sid for the next run)
  
• Set the start date to 2008/3/19 - to account for unique symbols and shorter for quicker results (will go farther back with sids instead of symbols)
  
• Capital_base set to $10k instead of $100k. I had noticed higher returns with larger starting capital in daily tests.

Otherwise, I've kept all else the same. Still needs a weightPeriods warm up etc. This backtest took under 2 hours to run. While survivor bias is important to eliminate, this particular minute mode implementation behaves like a robust defensive strategy, at least. Comparing to daily mode, and at the risk of stating the obvious with most investments, sensitivity to entry/exit plays a major role.

Why do the results deteriorate in minute mode? We are still only trading once a day, which is very similar to daily trading, right?

On the other hand, even with set_universe, the results still look pretty good (4% alpha), so the strategy is likely viable if we can make the minute mode version track the daily version more closely.

Presumably the edge is dependent on joining the opening auction.

Can't say for sure but there are differences between daily and minutely testing. For one, the slippage model in minutely has much tighter constraints in volume, and the close_price of the first bar is not the day's open. Slippage can cause spreading of an order over several bars resulting in higher uncertainty in the cost basis. This post may be helpful.

Would like to know the simplest way to keep it from going in the hole.

1970-01-01  Starting cash 100000.0  
2003-10-20  New cash low 835.4615  
2003-10-21  New cash low -2133.098  
2004-06-23  New cash low -2150.923  
2004-09-02  New cash low -2416.6681  
2004-09-30  New cash low -3030.1226  
2006-01-27  New cash low -3313.521981  
2007-04-23  New cash low -3799.721181  
2007-09-27  New cash low -5850.262881  
2009-04-08  New cash low -6899.456317  
2009-04-22  New cash low -10969.950317  
2009-06-11  New cash low -18195.840017  
End of logs.  

Here's a variant, run on minute bars. A couple notes:

--No commissions (set_commission(commission.PerShare(cost=0.0)))
--As I understand, Quantopian still cancels orders at the day's end, so it won't work as written in live trading

Hopefully, the code is readable. Questions/comments/improvements welcome.

Grant