from quantopian.pipeline import Pipeline
from quantopian.research import run_pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.factors import SimpleMovingAverage, AverageDollarVolume
Now that we've covered the basic components of the Pipeline API, let's construct a pipeline that we might want to use in an algorithm.
To start, let's first create a filter to narrow down the types of securities coming out of our pipeline. In this example, we will create a filter to select for securities that meet all of the following criteria:
Selecting for primary shares and common stock helps us to select only a single security for each company. In general, primary shares are a good representative asset of a company so we will select for these in our pipeline.
ADRs and GDRs are issuances in the US equity market for stocks that trade on other exchanges. Frequently, there is inherent risk associated with depositary receipts due to currency fluctuations so we exclude them from our pipeline.
OTC, WI, and LP securities are not tradeable with most brokers. As a result, we exclude them from our pipeline.
Let's create a filter for each criterion and combine them together to create a tradeable_stocks filter. First, we need to import the Morningstar DataSet as well as the IsPrimaryShare builtin filter.
from quantopian.pipeline.data import Fundamentals
from quantopian.pipeline.filters.fundamentals import IsPrimaryShare
Now we can define our filters:
# Filter for primary share equities. IsPrimaryShare is a built-in filter.
primary_share = IsPrimaryShare()
# Equities listed as common stock (as opposed to, say, preferred stock).
# 'ST00000001' indicates common stock.
common_stock = Fundamentals.security_type.latest.eq('ST00000001')
# Non-depositary receipts. Recall that the ~ operator inverts filters,
# turning Trues into Falses and vice versa
not_depositary = ~Fundamentals.is_depositary_receipt.latest
# Equities not trading over-the-counter.
not_otc = ~Fundamentals.exchange_id.latest.startswith('OTC')
# Not when-issued equities.
not_wi = ~Fundamentals.symbol.latest.endswith('.WI')
# Equities without LP in their name, .matches does a match using a regular
# expression
not_lp_name = ~Fundamentals.standard_name.latest.matches('.* L[. ]?P.?$')
# Equities with a null value in the limited_partnership Morningstar
# fundamental field.
not_lp_balance_sheet = Fundamentals.limited_partnership.latest.isnull()
# Equities whose most recent Morningstar market cap is not null have
# fundamental data and therefore are not ETFs.
have_market_cap = Fundamentals.market_cap.latest.notnull()
# Filter for stocks that pass all of our previous filters.
tradeable_stocks = (
primary_share
& common_stock
& not_depositary
& not_otc
& not_wi
& not_lp_name
& not_lp_balance_sheet
& have_market_cap
)
Note that when defining our filters, we used several Classifier methods that we haven't yet seen including notnull, startswith, endswith, and matches. Documentation on these methods is available here.
Next, let's create a filter for the top 30% of tradeable stocks by 20-day average dollar volume. We'll call this our base_universe.
base_universe = AverageDollarVolume(window_length=20, mask=tradeable_stocks).percentile_between(70, 100)
We have just defined our own base universe to select 'tradeable' securities with high dollar volume. However, Quantopian has two built-in filters that do something similar. The Q500US and the Q1500US are built-in pipeline filters that select a group of 500 or 1500 tradeable, liquid stocks each day. Constituents of these groups are chosen at the start of each calendar month by selecting the top 'tradeable' stocks by 200-day average dollar volume, capped at 30% of equities allocated to any single sector (more detail on the selection criteria of these filters can be found here).
To simplify our pipeline, let's replace what we've already written for our base_universe with the Q1500US built-in filter. First, we need to import it.
from quantopian.pipeline.filters import Q3000US
Then, let's set our base_universe to the Q1500US.
base_universe = Q3000US()
Now that we have a filter base_universe that we can use to select a subset of securities, let's focus on creating factors for this subset. For this example, let's create a pipeline for a mean reversion strategy. In this strategy, we'll look at the 10-day and 30-day moving averages (close price). Let's plan to open equally weighted long positions in the 75 securities with the least (most negative) percent difference and equally weighted short positions in the 75 with the greatest percent difference. To do this, let's create two moving average factors using our base_universe filter as a mask. Then let's combine them into a factor computing the percent difference.
# 10-day close price average.
mean_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10, mask=base_universe)
# 30-day close price average.
mean_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30, mask=base_universe)
percent_difference = (mean_10 - mean_30) / mean_30
Next, let's create filters for the top 75 and bottom 75 equities by percent_difference.
# Create a filter to select securities to short.
shorts = percent_difference.top(75)
# Create a filter to select securities to long.
longs = percent_difference.bottom(75)
Let's then combine shorts and longs to create a new filter that we can use as the screen of our pipeline:
securities_to_trade = (shorts | longs)
Since our earlier filters were used as masks as we built up to this final filter, when we use securities_to_trade as a screen, the output securities will meet the criteria outlined at the beginning of the lesson (primary shares, non-ETFs, etc.). They will also have high dollar volume.
Finally, let's instantiate our pipeline. Since we are planning on opening equally weighted long and short positions later, the only information that we actually need from our pipeline is which securities we want to trade (the pipeline index) and whether or not to open a long or a short position. Let's add our longs and shorts filters to our pipeline and set our screen to be securities_to_trade.
def make_pipeline():
# Base universe filter.
base_universe = Q3000US()
# 10-day close price average.
mean_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10, mask=base_universe)
# 30-day close price average.
mean_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30, mask=base_universe)
# Percent difference factor.
percent_difference = (mean_10 - mean_30) / mean_30
# Create a filter to select securities to short.
shorts = percent_difference.top(75)
# Create a filter to select securities to long.
longs = percent_difference.bottom(75)
# Filter for the securities that we want to trade.
securities_to_trade = (shorts | longs)
return Pipeline(
columns={
'longs': longs,
'shorts': shorts
},
screen=securities_to_trade
)
Running this pipeline will result in a DataFrame with 150 rows and 2 columns each day. Each day, the columns will contain boolean values that we can use to decide whether we want to open a long or short position in each security.
result = run_pipeline(make_pipeline(), '2015-05-05', '2015-05-05')
result
In the next lesson, we'll add this pipeline to an algorithm.