Part 1 - Investigate the Data with Blaze¶
To start out, let's investigate a partner dataset using Blaze. Blaze allows you to define expressions for selecting and transforming data without loading all of the data into memory. This makes it a nice tool for interacting with large amounts of data in research.
import matplotlib.pyplot as plt
import pandas as pd
# http://blaze.readthedocs.io/en/latest/index.html
import blaze as bz
from zipline.utils.tradingcalendar import get_trading_days
from quantopian.interactive.data.sentdex import sentiment
Interactive datasets are Blaze expressions. Blaze expressions have a similar API to pandas, with some differences.
type(sentiment)
Let's start by looking at a sample of the data in the Sentdex Sentiment Analysis dataset for AAPL.
aapl_sid = symbols('AAPL').sid
# Look at a sample of AAPL sentiment data starting from 2013-12-01.
sentiment[(sentiment.sid == aapl_sid) & (sentiment.asof_date >= '2013-12-01')].peek()
Let's see how many securities are covered by this dataset between 12/2013 and 12/2014.
num_sids = bz.compute(sentiment.sid.distinct().count())
print 'Number of sids in the data: %d' % num_sids
Let's go back to AAPL and let's look at the sentiment signal each day. To do this, we can create a Blaze expression that selects trading days and another for the AAPL sid (24).
# Mask for trading days.
date_mask = sentiment.asof_date.isin(
get_trading_days(pd.Timestamp('2013-12-01'), pd.Timestamp('2014-12-01'))
)
# Mask for AAPL.
stock_mask = (sentiment.sid == aapl_sid)
# Blaze expression for AAPL sentiment on trading days between 12/2013 and 12/2014
sentiment_2014_expr = sentiment[date_mask & stock_mask].sort('asof_date')
Compute the expression. This returns the result in a pandas DataFrame.
sentiment_2014_df = bz.compute(sentiment_2014_expr)
Plot the sentiment signal for AAPL.
sentiment_2014_df.plot(x='asof_date', y='sentiment_signal')
The sentiment signal tends to jump quite a bit. Let's try smoothing it by plotting the 5-day mean using the pandas.rolling_mean function. Note that we set the index of the Dataframe to be the asof_date so that the x-axis would be nicely formatted.
pd.rolling_mean(sentiment_2014_df.set_index('asof_date').sentiment_signal, window=5).plot()
Great! Now let's use this data in a pipeline.
Part 2 - Define Our Factor¶
Now that we have a dataset that we want to use, let's use it in a pipeline. In addition to the sentiment dataset, we will also use the EventVestor Earnings Calendar dataset to avoid trading around earnings announcements, and the EventVestor Mergers & Acquisitions dataset to avoid trading acquisition targets. We will work with the free versions of these datasets.
from quantopian.pipeline import Pipeline
from quantopian.research import run_pipeline
from quantopian.pipeline.factors import SimpleMovingAverage
from quantopian.pipeline.filters.morningstar import Q1500US
# Sentdex Sentiment free from 15 Oct 2012 to 1 month ago.
from quantopian.pipeline.data.sentdex import sentiment
# EventVestor Earnings Calendar free from 01 Feb 2007 to 1 year ago.
from quantopian.pipeline.factors.eventvestor import (
BusinessDaysUntilNextEarnings,
BusinessDaysSincePreviousEarnings,
)
# EventVestor Mergers & Acquisitions free from 01 Feb 2007 to 1 year ago.
from quantopian.pipeline.filters.eventvestor import IsAnnouncedAcqTarget
from quantopian.pipeline.factors import BusinessDaysSincePreviousEvent
def make_pipeline():
# 5-day sentiment moving average factor.
sentiment_factor = SimpleMovingAverage(inputs=[sentiment.sentiment_signal], window_length=5)
# Filter for stocks that are not within 2 days of an earnings announcement.
not_near_earnings_announcement = ~((BusinessDaysUntilNextEarnings() <= 2)
| (BusinessDaysSincePreviousEarnings() <= 2))
# Filter for stocks that are announced acquisition target.
not_announced_acq_target = ~IsAnnouncedAcqTarget()
# Filter for stocks that had their sentiment signal updated in the last day.
new_info = (BusinessDaysSincePreviousEvent(inputs=[sentiment.asof_date.latest]) <= 1)
# Our universe is made up of stocks that have a non-null sentiment signal that was updated in
# the last day, are not within 2 days of an earnings announcement, are not announced acquisition
# targets, and are in the Q1500US.
universe = (Q1500US()
& sentiment_factor.notnull()
& not_near_earnings_announcement
& not_announced_acq_target
& new_info)
# Our pipeline is defined to have the rank of the sentiment_factor as the only column. It is
# screened by our universe filter.
pipe = Pipeline(
columns={
'sentiment': sentiment_factor.rank(mask=universe, method='average'),
},
screen=universe
)
return pipe
result = run_pipeline(make_pipeline(), start_date='2013-12-01', end_date='2014-12-01')
Part 3 - Analyze Our Factor Using Alphalens¶
Now we can analyze our sentiment factor with Alphalens. To do this, we need to get pricing data using get_pricing.
# All assets that were returned in the pipeline result.
assets = result.index.levels[1].unique()
# We need to get a little more pricing data than the length of our factor so we
# can compare forward returns. We'll tack on another month in this example.
pricing = get_pricing(assets, start_date='2013-12-01', end_date='2015-01-01', fields='open_price')
Then we run a factor tearsheet on our factor. We will analyze 3 quantiles, looking at 1, 5, and 10-day lookahead periods.
import alphalens
alphalens.tears.create_factor_tear_sheet(factor=result['sentiment'],
prices=pricing,
quantiles=3,
periods=(1,5,10))
From this it looks like there's a relationship between the top quantile of our factor and positive returns as well as the bottom quantile and negative returns.
Let's try to capitalize on this by implementing a strategy that opens long positions in the top quantile of stocks and short positions in the bottom quantile of stocks. Let's invest half of our portfolio and half short, and equally weight our positions in each direction.
Before moving to the IDE, let's make some small changes to the pipeline we defined earlier. This will make it easier to order stocks based on quantile.
def make_pipeline():
# 5-day sentiment moving average factor.
sentiment_factor = SimpleMovingAverage(inputs=[sentiment.sentiment_signal], window_length=5)
# Filter for stocks that are not within 2 days of an earnings announcement.
not_near_earnings_announcement = ~((BusinessDaysUntilNextEarnings() <= 2)
| (BusinessDaysSincePreviousEarnings() <= 2))
# Filter for stocks that are announced acquisition target.
not_announced_acq_target = ~IsAnnouncedAcqTarget()
# Filter for stocks that had their sentiment signal updated in the last day.
new_info = (BusinessDaysSincePreviousEvent(inputs=[sentiment.asof_date.latest]) <= 1)
# Our universe is made up of stocks that have a non-null sentiment signal that was updated in
# the last day, are not within 2 days of an earnings announcement, are not announced acquisition
# targets, and are in the Q1500US.
universe = (Q1500US()
& sentiment_factor.notnull()
& not_near_earnings_announcement
& not_announced_acq_target
& new_info)
### Now we are using a classifier to separate the stocks into quantiles based on sentiment
### rank.
sentiment_quantiles = sentiment_factor.rank(mask=universe, method='average').quantiles(3)
### In our algorithm, we will go short the stocks in the 0th quantile, and long the stocks
### in the 2nd quantile.
pipe = Pipeline(
columns={
'sentiment': sentiment_quantiles,
'shorts': sentiment_quantiles.eq(0),
'longs': sentiment_quantiles.eq(2),
},
screen=universe
)
return pipe
Part 4 - Implement and Backtest the Strategy in the IDE.¶
See following post for the backtest.
Part 5 - Analyze Our Backtest Using Pyfolio¶
Let's load our backtest result and run it through a tearsheet using pyfolio.
bt = get_backtest('58812b2977ca4c474bbf393f')
bt.create_full_tear_sheet()
And there you have it! Some key takeaways from the tearsheet:
- Long and short exposure are equal.
- Beta is 0.
- Holdings per day is > 500.
- Maximum position concentration is low (< 0.005).
- Trades daily.