Here's the alpha decay and risk exposure notebook. The short Value exposure I'm rationalizing as 'quality ain't cheap.' Also, all of these tend to be short volatility, which has worked pretty well the last decade or so, but no guarantee that it would continue to work going forward, and I'm wondering if I'm just overfitting on short-vola factors? Short volatility exposure does make sense for a QUALITY factor composite though, in my opinion anyway.

Assuming your definition of quality is independent of price, it would seem as soon as the updated fundamental data hits the market your factor gets at least somewhat priced in by the rest of the market within 1-3 days. I'm curious, since this is a composite, have you looked at the alpha decay for each of the components individually?

Are the spikes in turnover a result of "earnings season" or are you sub-sampling your alpha factor?

Thanks for your comments @Viridian. I haven’t tested the alpha decay for each individual factor but that’s not a bad idea. And yes, the spikes are most likely from the quarterly reports and I agree that at least some of any alpha should get priced in at that time. I’m not subsampling just daily rebalancing. I did have to ‘cheat’ a bit on one of the factors to bring up the average daily turnover to >5% which is a bit unfortunate, but I still see it mostly as a ‘quality’ type factor.

If any of the alpha factors handle alpha decay really well, you could apply an SMA to just those factors and that would smooth out your turnover spikes at least a little bit.

How many different quality factors are in this composite?

Thanks @Viridian. I’m not too concerned about the turnover spikes. To me they are expected, and since the alpha decay is not too steep, it doesn’t matter too much if one doesn’t get fully filled on day 1 or the first few weeks even.

I’d have to check but I think I had quite a few in this one. I was trying to bring up average daily turnover >5% by adding different ones. They tend to all fall into three main categories though: profitability, moat, and balance sheet strength.

My concern is that more factors might == higher likelihood of overfitting? All factors will be fitted on some noise I would think (even purely Fundamentals based ones, but maybe they are less prone?), and when combined maybe it’s all based on noise in the end? What are your thoughts on that? They all held up during my held out cross validation periods during research though, both individually and when combined.

Thanks @Antony, that’s a very good idea! Think I might do that. I’m still waiting for the hold out performance for my Growth one (due to the contest delay). I’m also waiting for the current contest results to figure out which other one to pull to make room for the Quality one. If they all tank during the holdout period I’ll be very suspicious and would think they are quite overfit. However, if they keep pace reasonably well with the AQR or FF corresponding ones during the holdout I’ll be happier. :)

@Joakim, here is a technique I use in my own strategies. Call it CAGR decay compensation. It is given as a suggestion or a slightly different way of looking at the problem but can still have quite an impact.

Using the numbers extracted from your round_trips = True option in your first notebook I made the following chart based on the number of trades, average net profit per trade, … . On the 7th year we have comparable numbers as in your simulation. Then, I extrapolated what would come next and generated the following chart:

We can observe that your CAGR (original CAGR on the chart) is decaying with time. It will probably be more severe going forward since I usually run on the optimistic side. Nonetheless, by the very architecture of your trading script, this CAGR will continue to decay.

An alternative would be to structurally compensate for this decay. The above chart also displays a simple extrapolation of a probable and foreseeable future for some decay compensation.

The top line says that you could compensate to a level such that your CAGR would increase with time. This compensation could be modulated, increased or decreased depending on your other portfolio constraints (mostly by relaxing some of those constraints). But, it would still remain a choice to be made and would require some reengineering to your trading strategy to make it happen.

For instance, a slight nudge on the number of trades to the upside would be sufficient to make the compensated CAGR a flat line, thereby maintaining a constant CAGR over the portfolio's extended life. Slightly over-compensating (depending on point of view) could result in the above chart with its expanding CAGR with time. Should you want it higher, you slightly over-compensate some more.

The advantage of compensating for the expected CAGR decay should be evident from the above chart. Compensating would give your trading script, on average, over a longer-term, year over year, a higher portfolio CAGR level. Thereby defying the Law of diminishing returns from within your trading script.

I covered this in more detail in some of my old articles of which I retained the following which I think was also presented in one of the forums around Sept. 2016: https://alphapowertrading.com/images/divers/Strategy_Enhancers.html

Hi @Guy,

Thanks for this, I appreciate it, especially for analyzing and showing the CAGR decay in the strategy. I'm guessing the decay could be mainly due to 1) deployment of more capital will pretty much always result in lower returns, and 2) alpha discovery by other market participants, resulting in the market becoming more efficient. Maybe other reasons as well (e.g. regime shifts, randomness, etc).

However, for these strategies I didn't intend for them to be traded on their own. I'm more looking for 'pure alpha' factors that can be combined and traded as a combined portfolio, as per the new guidelines. That said, I don't think I'm doing a very good job at this (yet), but I hope I'm at least learning from my mistakes.

I know CAGR on live data is ultimately all that matters, but at this point I'm more looking to minimize risk as well as minimizing overfitting of my models, rather than try to maximize in-sample returns. Compensating for a declining CAGR to me would be introducing more risk, and I'll leave it to others to figure out how to best trade the portfolio.

To critique my own work, I'm worried that all 3 of my factor composites perform a little bit too well both during research and in the backtests (probably due to some training creep, as well as possibly a bit of inadvertent data snooping / p-hacking). I'm therefore more looking to try to simplify, reduce, and generalize my factors, which will most likely result in both lower in-sample CAGR and higher in-sample volatility, but with the hope it will result in more robust models, and OOS performance closer to the in-sample one.

That's my aim at least. For now, I'll leave these running (as they are) in the contest for at least 3 months, to see how they perform on actual live data (I haven't submitted the QUALITY one yet, but I plan to). Again, I'm skeptical, but keep my fingers crossed. :)

Here's the performance stats, from the contest, during the holdout period. Quite disappointing if you ask me, but maybe it's in line with other 'Quality' factors in the last year? Will leave it running in the contest for at least 63 trading days to see how it performs on live data.

@Joakim,
Thanks for all the factor analyses. Very instructive and results mimic my own problems to get Alpha.

Having the 1 year holdout for factset data seems more like what I call a "Do it with one hand behind your back"...very hard...

You're developing a factor that holds up with a (past) year out of sample....with the expectation that it'll extend in predictive power for another year...wow...
Even if it came out well for you, how would you know if you hadn't baked in information from that year implicitly?

If you could afford the factset data up to today, would you use it in your modeling.? I know I would....I've noticed I always fail with one hand behind my back...

That said, varying time period systematic holdout methodologies seem like a good idea to me, yet they are not going to get you any higher alpha, just a more robustly predictive one...maybe...

So my recommendation to you...I'm assuming you would want that given all your postings..if not, that's OK...
1. Figure out why last year's factor went bad for the last year.
2. Fix that if, possible and approximate the year holdout factors (the
ones you can't afford) with ones that can get you a factor valid up to today.
3. Run your quarterly holdouts, if your great graphs hold up...submit/trade.

I believe that's as good as it gets for this methodology, unless you want to forward create data that you can run monte-carlo sims on!

alan

QUALITY seems like the kind of factor that can't work forever. At some point valuations get stretched too far and it will offer diminishing or negative returns -- especially if we hit economic trouble. If you think of it such that at the beginning of your backtest those companies were selling at a discount to their future value, and that's why they subsequently outperformed. Eventually due to that outperformance the discount gap gets closed. And the further valuations get into bubble territory, the harder they'll revert.

Am I thinking about this wrong?

@Alan,

Thanks for your comments and feedback.

Personally I actually really like the holdout periods. I hold out a ton of data myself as well, but in the end I will have looked at all of it and probably trained on it as well. I therefore find it quite nice to have the (forced) holdout period available, as my last validation of the model. I'm also quite careful not to train on the final hold out period, so once I've submitted a strategy to the contest, I'm 'committed' and let it run for at least 63 trading days (assuming it doesn't fail), regardless if it performs well or poorly during the holdout. 'Peek n tweak' on the holdout period is a big no-no for me, as then I'm almost certain to overfit.

That said, just 1 year of holdout (most recent period too) may not be enough to determine if a model is overfit or if it just performs poorly due to recent market behavior. Also, if I train on the most recent year (and hold out earlier periods) there will probably be some recency bias too, so... it's not easy.

@Viridian,

I usually tend to agree with you, but not sure on this one. Factors will go in and out of favor as well I reckon (this is a good thing). In the late '90s, being short QUALITY and long JUNK might have worked quite well, and I'm sure that could happen again, but that doesn't mean it would make any sense (Quality does though).

Also, in a market crash, QUALITY might perform relatively well (relative to other factors) as funds that have to be invested in equities pile in on Quality stocks as a type of 'safe haven', possibly creating a QUALITY bubble due to over-crowding. The fact that all robust factors will under-perform at times is actually a good thing in my view. If they worked solidly all the time, everyone would just start doing those, which would destroy the factor.