Introduction to the Quantopian Risk Model in Research¶
In this notebook, we give an introduction to the Quantopian Risk Model and introduce new APIs for interacting with risk model on the Quantopian Research Platform.
What is a Factor Risk Model?¶
Asset prices change for a wide variety of reasons. Some events that affect the price of a stock are specific to just that stock: when a company introduces an innovative new product or suffers a public scandal, for example, the effect on the overall market is mostly limited to changes in the price for that company's stock. In many cases, however, events that affect the price of a stock also affect the prices of other, similar stocks. Many asset prices would change in similar ways if the price of steel tripled overnight or if the US unemployment rate were cut in half over the course of year.
The observation that many assets' prices are influenced by similar external events is one of the motivating ideas behind the use of Factor Risk Models in finance.
A factor risk model attempts to describe the returns of a large number of assets in terms of the returns of a small number of risk factors. The Fama-French model for example, models the returns of each asset in terms of three factors: a "market" factor representing the returns of the market as a whole, a "size" factor representing the returns of large-cap stocks relative to small-cap stocks, and a "value" factor representing the returns of stocks with high book-to-market ratios relative to stocks with low book-to-market ratios.
Generally speaking, a Factor Risk Model consists of two artifacts:
- A set of factor returns, each of which is a timeseries of returns associated with a source of price movement shared by all assets of interest.
- A set of factor loadings that describes, for each asset of interest, how that asset's returns are driven by the factor returns from (1).
In the case of the Fama-French model, the factor returns are computed by taking the returns of portfolios of assets designed to capture the effect of each factor (for example, the market return is calculated from the return of a broad-market long-only portfolio, and the size return is calculated from a long-short portfolio that longs small-cap stocks and shorts large-cap stocks), and each asset's factor loadings are calculated by running a multiple linear regression of the asset's returns against the factor returns.
The Quantopian Risk Model¶
The Quantopian Risk Model defines 16 risk factors: 11 sector factors and 5 style factors.
Sector factors capture the returns associated with the aggregrate performance of each sector of the US economy.
Style factors capture the returns associated with other common drivers of assets returns (e.g. size, value, and volatility).
What Can I Do with The Quantopian Risk Model Today?¶
Factor Risk Models are used for many purposes in quantitative finance. We have many ideas for tools we could build using the risk model, but for this release, we've focused primarily on one important use-case: Performance Attribution via the AlgorithmResult object in research. We've also built experimental support for working with the risk model outputs, both directly in research notebooks and in the Pipeline API.
A complete list of the new API features is as follows:
- Additions to the
AlgorithmResultclass:- New Method:
create_perf_attrib_tear_sheet() - New Field:
attributed_factor_returns - New Field:
factor_exposures
- New Method:
- Additions to the Research API:
quantopian.research.experimental.get_factor_returnsquantopian.research.experimental.get_factor_exposures
- Additions to the Pipeline API:
quantopian.pipeline.experimental.risk_loading_pipelinequantopian.pipeline.experimental.BasicMaterialsquantopian.pipeline.experimental.ConsumerCyclicalquantopian.pipeline.experimental.FinancialServicesquantopian.pipeline.experimental.RealEstatequantopian.pipeline.experimental.ConsumerDefensivequantopian.pipeline.experimental.HealthCarequantopian.pipeline.experimental.Utilitiesquantopian.pipeline.experimental.CommunicationServicesquantopian.pipeline.experimental.Energyquantopian.pipeline.experimental.Industrialsquantopian.pipeline.experimental.Technology
Performance Attribution via the AlgorithmResult Class¶
The biggest change we're releasing today is a suite of enhancements to the AlgorithmResult class that make it easy for Quantopian users to use the risk model to break down and visualize the performance of their algorithms.
What is Performance Attribution?¶
When we're developing an algorithm, it's often helpful to have a sense of what drives the performance of that algorithm. For algorithms with a small number of positions, it can be manageable to simply look at the returns of each individual position. As the number of positions we hold grows, however, it becomes increasingly important to find ways of summarizing the performance of an algorithm in a way that preserves as much information as possible.
Performance attribution in PyFolio happens in three steps:
- Convert Positions to Factor Exposures: Each day, we calculate the algorithm's net factor exposures by multiplying the weight of each asset in the portfolio by that asset's factor loadings and then summing the exposure vectors across all assets.
(For the linear-algebraically inclined, this step performs a matrix multiplication between the (factors x assets) factor loadings matrix by the (assets x 1) column-vector of portfolio weights each day. One potentially-useful way of thinking about this is that the risk model factor loadings define a change of basis that transforms "stock exposure space" to "factor exposure space", and this step applies that change of basis.)
- Convert Factor Exposures to Attributed Returns: The timeseries of factor exposures produced in step
(1)gives us a measure of how "sensitive" the risk model expects our portfolio should be to the returns of each risk factor. The next step is to multiply these factor exposures at each time-step by the associated factor returns value at the same time step, which gives us a measure of the algorithm returns at that time-step that we attribute each factor exposure. - Summarize Attributed Returns: The last step of performance attribution is to aggregate all the attributed factor returns into a single measure of the returns that are "explained" by the risk model. We call this aggregated returns measure your "Common Returns", and we call the remaining portion of your returns "Specific Returns".
Running Performance Attribution: create_perf_attrib_tearsheet¶
The easiest way to run performance attribution is to use the new create_perf_attrib_tear_sheet() method of BacktestResult (the object returned by the built-in get_backtest() function). create_perf_attrib_tear_sheet loads the necessary risk model data and passes it to PyFolio to calculate and plot your algorithm's common and specific returns, risk exposures, and returns attributed to common risk factors.
Let's load up a backtest from an updated version of the Optimize API announcement post.
bt = get_backtest('5a0317326279aa458c825cad')
bt.create_perf_attrib_tear_sheet()
The new performance attribution tearsheet has a few components:
Summary Tables¶
At the top of the tearsheet, a table of summary statistics shows the backtest's annualized total returns, common returns, and specific returns (more on what these metrics mean below), as well as a modified Sharpe Ratio that describes the return/volatility ratio of the backtest's specific returns. For this backtest, our annualized total return was 4%, but the risk model attributes -4% returns to our common factor exposures, which was overcome by an 8% specific return.
The next table provides a summary of our algorithm's factor exposures and attributed factor returns. Most of this algorithm's exposures are pretty small (at least on average), but the risk model thinks we have a moderate bias toward low volatility stocks (expressed by our average volatility exposure of -0.3), as well as a small bias toward large-cap stocks (expressed by our average size exposure of 0.1).
Plots¶
Common vs. Specific Returns¶
The first plot shows a cumulative time series of the backtest's returns, along with the same returns decomposed into common and specific components (this is the output of step (3) in the outline above). The main purpose of this plot is show whether the algorithm's returns are primarily driven by common returns or specific returns. In general:
- If the blue line moves with the red line more than the green line, then the algorithm's returns are primarily driven by common factors.
- If the blue line moves with the green line more than the red line, then the algorithm's returns are primarily driven by specific returns.
In this case, our total return looks like it's mostly driven by specific risk, which is consistent with the low average factor exposures observed in the summary tables.
Attributed Factor Returns¶
The second plot in the tearsheet shows the returns attributed by the risk model to each factor over time (this is the output of step (2) in the outline above). This plot is often fairly noisy, but it can be useful for spotting outliers and for seeing if one factor dominates your attributed returns. It's likely that we'll revise this plot in a future release to make it easier to read.
Factor Exposures¶
The third plot in the tearsheet shows the net factor exposures calculated in step (1) of the outline above. This is usually the best plot for spotting unexpected factor risk concentrations and/or spikes. For this algorithm, this plot confirms our earlier observation that we seem to have a consistent bias toward low-volatility stocks.
Direct Access to Performance Attribution Outputs¶
The performance attribution tearsheet aims to provide a useful default set of visualizations for analyzing an algorithm's risk exposures. While we expect to grow and improve the tearsheet over time, it would be impractical and counter-productive for us to try to include every possible visualization or aggregation of factor exposures and attributed returns, so we've also added new attributes to BacktestResult that allow users to work directly with data used by create_perf_attrib_tear_sheet.
BacktestResult.factor_exposurescontains the data shown in the bottom plot of the tear sheet.BacktestResult.attributed_factor_returnscontains the data shown in the middle plot of the tear sheet.
bt.factor_exposures.tail()
bt.attributed_factor_returns.tail()
We can use this data to build our own custom visualizations of the factor loadings/returns.
The default visualization of the factor exposures shows them as a timeseries. This is useful for seeing how the exposures changes over time, but doesn't help much for seeing the overall distribution of exposures across the span of the algorithm. We can use seaborn to visualize the distribution of the factor exposures.
import pandas as pd
import seaborn as sns
def visualize_exposures_distribution(exposures):
ax = sns.boxplot(data=bt.factor_exposures.dropna(), orient='h');
ax.set_title('Distribution of Daily Factor Exposures')
ax.set_xlabel('Daily Exposure')
ax.set_ylabel('Factor');
return ax
visualize_exposures_distribution(bt.factor_exposures);
This visualization gives us a better sense of the overall spread of our algorithm's exposures, at the cost of losing information about how the exposures evolve over time.
Additions to the Research API¶
If you want to dig deeper into the data that we use for performance, we've added two new methods to the Research API under quantopian.research.experimental:
quantopian.research.experimental.get_factor_returnstakes(start_date, end_date)and returns aDataFramecontaining factor returns for the period between the date bounds.quantopian.research.experimental.get_factor_exposurestakes(start_date, end_date, sids)and returns a MultiIndex-edDataFramecontaining factor exposures for the requested sids between the date bounds. (NOTE: The signature ofget_factor_exposuresis likely to change to(assets, start_date, end_date)in the near future to match the signature of other API methods.
from quantopian.research.experimental import get_factor_loadings, get_factor_returns
Fetch and Plot Factor Returns for 2014¶
rets = get_factor_returns(pd.Timestamp('2014'), pd.Timestamp('2015'))
rets.head()
ax = (((1 + rets).cumprod())
.plot(title='Cumulative Factor Returns', figsize=(12.5, 10), colormap='Set3'));
ax.grid(False)
ax.legend(bbox_to_anchor=(1, 1), loc='upper left');
Fetch and Plot Factor Loadings for AAPL and MCD¶
# NOTE: In a future revision we expect to update the API for `get_factor_exposures` so that
# you can pass strings or Equity objects in addition to sids. For now, however, sids are required.
AAPL_sid = symbols('AAPL').sid # Should be 24.
MCD_sid = symbols('MCD').sid # Should be 4707.
loadings = get_factor_loadings(pd.Timestamp('2015'), pd.Timestamp('2016'), [AAPL_sid, MCD_sid])
Most assets have a sector loading of 0 for sectors outside their own.
loadings.head()
AAPL Exposures over Time¶
nonzero_loadings = loadings.loc[:, (loadings != 0).any()]
(nonzero_loadings
.xs(24, level=1) # Extract just the rows for AAPL.
.plot(grid=False, title='AAPL Risk Exposures') # Plot them.
.legend(bbox_to_anchor=(1, 1), loc='upper left'));
This plot shows that AAPL's largest loading coefficient is to size during this period, but it also carries significant positive exposure to the technology sector and to the momentum style factor, as well as smaller negative exposures to value and volatility.
MCD Exposures Over Time¶
nonzero_loadings = loadings.loc[:, (loadings != 0).any()]
(nonzero_loadings
.xs(4707, level=1) # Extract just the rows for MCD.
.plot(grid=False, title='MCD Risk Exposures') # Plot them.
.legend(bbox_to_anchor=(1, 1), loc='upper left'));
Like AAPL, MCD carries a significant size exposure, but its sector exposure to consumer_cyclical is significantly lower, suggesting that MCD's returns are less sensitive to the overall movement of its sector compared to AAPL.
Working With the Risk Model in Pipeline¶
As part of this release, we've built experimental support for fetching assets' risk loadings as Pipeline API Factors. Each risk factor has a new corresponding Pipeline Factor, which produces the desired risk loadings. These can be used for things like risk-factor-weighting other pipeline expressions.
(NOTE: Pipeline API support for the risk model is currently only available in research. We plan to enable support for risk model pipelines in the backtester in the near future.)
import quantopian.pipeline.experimental as exp
from quantopian.pipeline import Pipeline
from quantopian.research import run_pipeline
from operator import or_
loading_terms = {
'basic_materials': exp.BasicMaterials(),
'consumer_cyclical': exp.ConsumerCyclical(),
'financial_services': exp.FinancialServices(),
'real_estate': exp.RealEstate(),
'consumer_defensive': exp.ConsumerDefensive(),
'health_care': exp.HealthCare(),
'utilities': exp.Utilities(),
'communication_services': exp.CommunicationServices(),
'energy': exp.Energy(),
'industrials': exp.Industrials(),
'technology': exp.Technology(),
'momentum': exp.Momentum(),
'short_term_reversal': exp.ShortTermReversal(),
'size': exp.Size(),
'value': exp.Value(),
'volatility': exp.Volatility(),
}
loading_pipeline = Pipeline(
loading_terms,
# This is a fancy way of saying "keep all the rows where at least one factor loading is non-null."
screen=reduce(or_, (t.notnull() for t in loading_terms.values())),
)
loadings = run_pipeline(loading_pipeline, '2015', '2016')
loadings.head()
We expect the pattern of "get all the risk loadings together in one Pipeline" to be common enough that we've added a helper function to make it easier to do just that.
quantopian.pipeline.experimental.risk_loading_pipeline creates a pipeline that's equivalent to the one we manually defined in the cells above.
from quantopian.pipeline.experimental import risk_loading_pipeline
easier_loading_pipeline = risk_loading_pipeline()
easier_loadings = run_pipeline(easier_loading_pipeline, '2015',' 2016')
easier_loadings.head()
(loadings == easier_loadings).all().all()
Review and Conclusions¶
In this notebook we introduced the Quantopian Risk Model and introduced several new APIs for interacting with the risk model in research.
- We discussed what a
Factor Risk Modelis and developed some intuitions for why factor modelling might be a reasonable way to describe the returns of assets. - We looked at new features of the
AlgorithmResultobject that enable Quantopian users to do analyze and visualize the performance of their algorithms. - We looked at new Research API methods that allow you to fetch and inspect the outputs of the Quantopian Risk Model.
- We briefly looked at how you can fetch risk model data in the Pipeline API.