Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Higher IC doesn't lead to better backtest performance
Shiv Chawla
posted May 16, 2019
Finance Theory Research Portfolio Optimization Statistics Alphalens

I have been researching multiple alpha factors and with each additional alpha factor, the IC goes up. However, the backtest results don't reflect the improved predictability.

I have two scenarios with combination of around 14 alpha factors each. In the results below, Scenario 1 has better metrics but its backtest performance is far inferior in comparison to Scenario 2.

How do I explain this performance gap?
Is the portfolio construction process sub-optimal? I am currently using Maximize Alpha with vanilla risk exposure and leverage constraints.

Scenario - 1 (Forward 5 Day Returns)
Date Range: Jan 2015 - May 2019
Num. Factors (14)
Linear Combination Coefficient (1.0)
Alpha (0.066)
Beta (-0.237)
Mean IC (0.026)
IC Std (0.139)
Risk Adj. IC (0.189)
t-stat (6.207)
Spread-bps (19.482)
Backtest Performance - Total Return - 34.84%/Sharpe - 0.95/ Max Drawdown - 12.84%

Scenario - 2 (Forward 5 Day Returns)
Date Range: Jan 2015 - May 2019
Num. Factors (14)
Linear Combination Coefficient (1.0)
Alpha (0.063)
Beta (-0.244)
Mean IC (0.023)
IC Std (0.146)
Risk Adj. IC (0.156)
t-stat (5.132)
Spread-bps (16.725)
Backtest Performance - Total Return - 58.96%/Sharpe - 1.30/Max Drawdown - 9.53%

Average correlation between net alpha (simple linear combination of factors) from scenario1 and scenario2 ~ 0.87

5 responses
Jamie Veitch (Azure Hawk)
May 16, 2019

I'm pretty sure that IC only takes into account accuracy, not profitability.

e.g.

Algo 1:
Trade 1: -$5
Trade 2: +$5

Profit: $0
IC: 0.0

Algo 2:
Trade 1: -$5
Trade 2: +$1,000

Profit: $995
IC: 0.0

Shiv Chawla
May 16, 2019

I agree partly.

In context of Alphalens, I guess the definition of IC is correlation of exposure (equal weighted?) and holding period returns. And If it goes higher, it means that predictability improves (or accuracy as suggested above)

Ideally, improved predictability should lead to improved profitability (As optimizer will appropriately chose weights to maximize alpha). And as a researcher, the goal should be improve predictability as suggested by Q in some posts as well (otherwise it will lead to overfitting).

Seems like portfolio construction introduces the performance gap as it has to satisfy constraints that are not taken into account in Alphalens. Could that be or is it something else?

Indigo Monkey
May 16, 2019

Having an IC standard deviation 5 times larger than your IC mean does not make the IC mean that useful, especially, when your mean IC is close to zero.

Shiv Chawla
May 16, 2019

What is the typical Mean IC (st. dev.) one should expect in alpha prediction? Are these IC number meaningless?

Indigo Monkey
May 16, 2019

Each factor should past the IC t-stat test or pval, so that the IC number is not meaningless. Using a noisy factor to predict the future based on past results will lead to varying results good and bad. The mean IC you are getting are typical for 5 day time frame.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.