First Lecture (Sample Algorithm for a Basic Strategy)
For this example, we're going to write a simple momentum script. (When the stock goes up quickly, we're going to buy, when it goes down we're going to sell.)
To run an algorithm in Quantopian, you need two functions:
initialize and handle_data.
def initialize(context):
The initialize function sets any data or variables that you'll use in your algorithm. (For instance, you'll want to define the security (or securities) you want to backtest. You'll also want to define any parameters or values. You're going to use later. It's only called once at the beginning of your algorithm.)
In our example, we're looking at Apple. If you re-type this line you'll see the auto-complete that is available for security.
context.security = symbol('AAPL')
The handle_data function is where the real work is done.
This function is run either EVERY MINUTE (in live trading and minute backtesting mode)
or everyday (in daily backtesting mode).
def handle_data(context, data):
Stock's moving average for the last 50 days and the last 20 days.
VWAP (volume weighted average price) for the last 5 days and the last 1 day.
And the current price.
average_price_50 = data[context.security].mavg(50)
average_price_20 = data[context.security].mavg(20)
vwap_price_5 = data[context.security].vwap(5)
vwap_price_1 = data[context.security].vwap(1)
current_price = data[context.security].price
Another powerful built-in feature of the Quantopian backtester is the portfolio object. The portfolio object tracks your positions, cash, cost basis of specific holdings, and more. In this line, we calculate the current amount of cash in our portfolio.
cash = context.portfolio.cash
Retrieve open orders:
open_orders = get_open_orders()
Algorithm: if the moving average for the past 50 days is higher than the moving average for the past 20 day, then buy shares for 100 percent of the available amount.
Note: This amount fluctuates depending on the PnL
If average_price_50 > average_price_20:
If vwap_price_5 > vwap_price_1:
If context.security not in open_orders:
# Place orders here
order_target_percent(context.security, 1.0)
# Place the buy order (**positive means buy**, negative means sell)
order(context.security, **+**number_of_shares)
log.info("Buying %s" % (context.security.symbol))
Algorithm (cont.): if the moving average for the past 50 days is below the moving average of the past 20 days, then sell shares for 100 percent of the available amount.
#elif average_price_50 < average_price_20: elif vwap_price_5 < vwap_price_1:
if context.security not in open_orders:
order_target_percent(context.security, -0.75)
log.info("Selling %s" % (context.security.symbol))
You can use the record() method to track any custom signal. The record graph tracks up to five different variables. Here we record the Apple stock price.
record(stock_price=data[context.security].price)
Second Lecture (Pipeline Basic Algorithm)
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.factors import CustomFactor, SimpleMovingAverage, AverageDollarVolume
from quantopian.pipeline.data import morningstar
from quantopian.pipeline.filters.morningstar import IsPrimaryShare
from quantopian.pipeline.classifiers.morningstar import Sector
import numpy as np
import pandas as pd
def initialize(context):
"""
Called once at the start of the program. Any one-time
startup logic goes here.
"""
Define context variables that can be accessed in other methods of the algorithm.
context.long_leverage = 0.5
context.short_leverage = -0.5
context.returns_lookback = 5
**Rebalance on the first trading day of each week at 11AM.**
schedule_function(rebalance,
date_rules.week_start(days_offset=0),
time_rules.market_open(hours=1, minutes=30))
**Record tracking variables at the end of each day.**
schedule_function(record_vars,
date_rules.every_day(),
time_rules.market_close(minutes=1))
**Create and attach an empty Pipeline.**
pipe = Pipeline()
pipe = attach_pipeline(pipe, name='my_pipeline')
**Construct Factors. (built-in)**
sma_10 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=10)
sma_30 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=30)
sma_10_30 = sma_10/sma_30
Define high and low sma filters to be the bottom 10% and top 10% of securities.
low_sma = sma_10_30.percentile_between(0,10)
high_sma = sma_10_30.percentile_between(90,100)
Construct a Filter.
prices_under_5 = (sma_10 < 5)
Register outputs.
#pipe.add(sma_10, 'sma_10')
#pipe.add(sma_30, 'sma_30')
pipe.add(low_sma, 'low_sma')
pipe.add(high_sma, 'high_sma')
#pipe.add(securities_to_trade, 'securities_to_trade')
**Remove rows for which the Filter returns False.**
pipe.set_screen(prices_under_5)
def before_trading_start(context, data):
Access results using the name passed to attach_pipeline
.
context.output = pipeline_output('my_pipeline')
Print:
context.output.head(5)
**Store pipeline results for use by the rest of the algorithm.**
context.pipeline_results = context.output
Sets the list of securities we want to long as the securities with a 'True'
# value in the high_sma column.
context.long_secs = context.output[context.output['high_sma']]
# Sets the list of securities we want to short as the securities with a 'True'
# value in the low_sma column.
context.short_secs = context.output[context.output['low_sma']]
**A list of the securities that we want to order today.**
context.security_list = context.long_secs.index.union(context.short_secs.index).tolist()
**A set of the same securities, sets have faster lookup**.
context.security_set = set(context.security_list)
def compute_weights(context):
"""
Compute weights to our long and short target positions.
"""
# Set the allocations to even weights for each long position, and even weights
# for each short position.
long_weight = context.long_leverage / len(context.long_secs)
short_weight = context.short_leverage / len(context.short_secs)
return long_weight, short_weight
def rebalance(context,data):
"""
This rebalancing function is called according to our schedule_function settings.
"""
long_weight, short_weight = compute_weights(context)
**For each security in our universe, order long or short positions according to our context.long_secs and context.short_secs lists.**
for stock in context.security_list:
if data.can_trade(stock):
if stock in context.long_secs.index:
order_target_percent(stock, long_weight)
elif stock in context.short_secs.index:
order_target_percent(stock, short_weight)
Sell all previously held positions not in our new context.security_list.
for stock in context.portfolio.positions:
if stock not in context.security_set and data.can_trade(stock):
order_target_percent(stock, 0)
**Log the long and short orders each week.**
log.info("This week's longs: "+", ".join([long_.symbol for long_ in context.long_secs.index]))
log.info("This week's shorts: " +", ".join([short_.symbol for short_ in context.short_secs.index]))
def record_vars(context, data):
"""
This function is called at the end of each day and plots certain variables.
"""
Check how many long and short positions we have.
longs = shorts = 0
for position in context.portfolio.positions.itervalues():
if position.amount > 0:
longs += 1
if position.amount < 0:
shorts += 1
Record and plot the leverage of our portfolio over time as well as the number of long and short positions. Even in minute mode, only the end-of-dal leverage is plotted.
record(leverage = context.account.leverage, long_count=longs, short_count=shorts)
Third Lecture (Long-Short Pipeline Multifactor)
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.factors import CustomFactor, AverageDollarVolume
from quantopian.pipeline.data import morningstar
import pandas as pd
import numpy as np
class Value(CustomFactor):
inputs = [morningstar.valuation_ratios.book_value_yield,
morningstar.valuation_ratios.sales_yield,
morningstar.valuation_ratios.fcf_yield]
window_length = 1
def compute(self, today, assets, out, book_value, sales, fcf):
value_table = pd.DataFrame(index=assets)
value_table["book_value"] = book_value[-1]
value_table["sales"] = sales[-1]
value_table["fcf"] = fcf[-1]
out[:] = value_table.rank().mean(axis=1)
class Momentum(CustomFactor):
inputs = [USEquityPricing.close]
window_length = 252
def compute(self, today, assets, out, close):
out[:] = close[-20] / close[0]
class Quality(CustomFactor):
inputs = [morningstar.operation_ratios.roe,
morningstar.operation_ratios.roa,
morningstar.operation_ratios.gross_margin]
window_length = 1
def compute(self, today, assets, out, roe, roa, gross_margin):
quality_table = pd.DataFrame(index=assets)
quality_table["roe"] = roe[-1]
quality_table["roa"] = roa[-1]
quality_table["gross_margin"] = gross_margin[-1]
out[:] = quality_table.rank().mean(axis=1)
class Volatility(CustomFactor):
inputs = [USEquityPricing.close]
window_length = 252
def compute(self, today, assets, out, close):
close = pd.DataFrame(data=close, columns=assets)
# Since we are going to rank largest is best we need to invert the sdev.
out[:] = 1 / np.log(close).diff().std()
Compute final rank and assign long and short baskets.
def before_trading_start(context, data):
results = pipeline_output('factors').dropna()
ranks = results.rank().mean(axis=1).order()
context.shorts = 1 / ranks[results["momentum"] < 1].head(200)
context.shorts /= context.shorts.sum()
context.longs = ranks[results["momentum"] > 1].tail(200)
context.longs /= context.longs.sum()
context.security_list = context.longs.index.union(context.shorts.index).tolist()
def initialize(context):
pipe = Pipeline()
pipe = attach_pipeline(pipe, name='factors')
value = Value()
momentum = Momentum()
quality = Quality()
volatility = Volatility()
pipe.add(value, "value")
pipe.add(momentum, "momentum")
pipe.add(quality, "quality")
pipe.add(volatility, "volatility")
Put any initialization logic here. The context object will be passed to
the other methods in your algorithm.
Fourth Lecture (Larkin’s L1 routine with whitelisted “cvxpy”)
import cvxpy as cvx
import numpy as np
def calc_opt_L1_weights(alpha_vector, hist_returns, max_risk, max_position=0.05):
num_stocks = len(alpha_vector)
num_days = len(hist_returns)
A = hist_returns.fillna(0.0).as_matrix()
x = cvx.Variable(num_stocks)
objective = cvx.Maximize(alpha_vector*x)
constraints = [
sum(cvx.abs(A*x))/num_days <= max_risk,
sum(x) == 0,
sum(cvx.abs(x)) <= 1,
x <= max_position,
x >= -max_position
]
prob = cvx.Problem(objective, constraints)
prob.solve(verbose=False)
print prob.value
sol = np.asarray(x.value).flatten()
return sol
Other important info: Issue with open orders & leverage:
If you want to place an order such a placing $1m in Apple, this may take more than a minute to get filled and if it does not, then the algorithm may send another order, so you end up placing multiple orders simultaneously (open orders only get cancelled at the end of each trading day). So you want to make sure the stocks are not in open orders before placing an additional order. A good way to track that you are not overusing leverage is to record your leverage over time:
record(leverage = context.account.leverage)
To check the current open orders:
open_orders = get_open_orders()
Then it is just a question of doing:
if context.appl not in open_orders:
order_percent_target(context.appl,1.0)
We have now taken care of the leverage and ensure it is within a reasonable range. However, we still have an issue with ridiculous transaction amounts of 6m over a single day, i.e. making way too many trades a day. How to deal with this? Schedule functions. Custom functions have to be called to be run unlike default funtions such as initialise or handle data.