Hi @Joe,
Thanks for all your input! This is a tricky area of api design given the tension between ease and flexibility, so I really appreciate the fresh ideas. I do like the concept of a declarative style interface, like you would find in an ORM. Let me clarify our current thinking a bit, and come back to that idea.
In our next release, we'll be introducing two new features. One is universe selection, and the other is batch transforms. Together these are designed to let your algorithm choose the securities to include in the portfolio at runtime, rather than forcing you to decide at coding time. The idea is to split the problem in two, and maximize one part for simplicity (universe selection) and the other for flexibility (batch transforms).
Universe Selection
Our historical database has pricing for approximately 15,000 instruments. They range from esoteric ETFs, to large cap house-hold names, to ADRs, to penny stocks. All of these instruments are not relevant for all algorithms; it would waste your time as a user to stream all the instruments through every algo. We want to provide a way to very easily winnow the securities fed to your algorithm down to a list of securities that are applicable to your algorithm.
The selection process is, as you've suggested, pretty data oriented. You want to be able to choose securities based on properties like market cap or liquidity or fundamentals. At the moment, we have just trade history. Our first implementation will only provide a "dollar volume" screen as described above - you will be able to specify the percentile range for v_value rankings. However, we will try to define the API in such a way as to allow for future expansion.
The current plan is to provide a magic function, set_universe, which takes a subclass of Universe (a new class). Each subclass will accept constructor parameters that define the universe. Since the universe is a set of criteria, we dynamically update the membership as the simulation runs through time - the current plan is to update the universe every quarter during the simulation. In code, choosing a universe looks like this:
def initialize(context):
# choose the top two percentile of DollarValue rankings
set_universe(DollarValueUniverse(98.0, 99.0))
I'd love your feedback on both the DollarValue universe, the api, and the general direction!
Batch Transforms
The universe will give you the ability to define a large number of securities based on criteria, without requiring you to explicitly reference each sid. But, it is extremely unlikely that you will want to make investments in every security in your universe. You will want to explore the entire universe, and find pairs, related groups, or special individual stocks to bet on. To that end, we are adding a new facility that allows you to work with a cross-sectional history. So, instead of getting one day's prices or one minute's prices, you can get the last N days of prices/volumes/etc. All the data in the trailing window for all the securities in your universe will be packaged into a pandas data panel. You can then work with prices or volumes in separate dataframes. The code will look like this:
def initialize(context):
# construct a new batch transform, tuck it into the context
# call the function my_transform every two days, keeping 7 days of history in the data panel
context.a_batch_transform = BatchTransform(func=my_transform, refresh_period=2, delta=timedelta(days=7))
def handle_data(data, context):
# update the transform with the current data events
transform_results = context.a_batch_transform.handle_data(data)
# place orders, rebalance portfolio, etc.
order(transform_results.buys[0], 10)
def my_transform(data_panel):
prices_dataframe = data_panel['price']
# do work on prices...
# return the data your algorithm will need to make buy/sell choices
return transform_results
