Back to Community

Using Fetcher to load Pairs Data

posted

This is an example of a way to use fetch_csv to load more complicated data that you have generated offline and want to feed into your Quantopian algo. Within a csv file you can encode a structure like a dict as a string so it's associated with a single column. The hard part is parsing it back to the original data structure because the JSON libs are not available.

This script uses fetchers 'pre_func' to parse simple dicts/lists so they can be used in your algo. It works for flat dictionaries and lists only where the keys are strings and values are floats.

An example of the data is below. Here's a breakdown of the fields and how they are used

datedate: self explanatory
ranking: A signal strength parameter
relation: A single basket of stocks along with the model params encoded as a single string
longSpreadLimits/shortSpreadLimits: Entry/Exit points for long/short positions on the spread
timeLimit: A tuple of (days, hours, minutes) for the maximum time to wait for a reversion.
volatilty: Used to decide on the leverage applied to the basket

date,ranking,relation,longSpreadLimits,shortSpreadLimits,timeLimit,volatility  
2015-06-01,1,"{'JPM': 2.7, 'GS': -1, 'constant': 25}","(-2.0, -1.2, -0.2)","(1.5, 1.1, 0.2)","(26, 0, 0)",4.5  
2015-06-01,1,"{'BCX': 2.7, 'GS': -3.3, 'MS':1.4, 'constant': -20}","(-1.8, -0.7, 0.2)","(1.5, 1.1, 0.2)","(15, 0, 0)",4.5

Within the algo data contains a 'pair_info' field that will contain this data, you can then map it to the correct symbols and place trades based on it.

This should help those of you who generate your signals offline. It's a pretty specific example, but can be used as a reference and tweaked based on your needs.

3 responses

JOHN CHAN

Using fetcher...
def initialize(context):
fetch_csv(
quandl.build_url('DMDRN/AAPL_ALLFINANCIALRATIOS'),
date_column='Date',
symbol='AAPL' ------------------------> cud be any symbol specified by user not found in quantopian database
)

Will be called on every trade event for the securities you specify.

def handle_data(context, data):
# Implement your algorithm logic here.
if data['AAPL']:
log.info(data['AAPL'])

Most of us in Quantopian use def initialize(context):
context.my_sid = sid(24)

How do I.. subtitute symbols specified by fetcher to context.my_sid = sid(24) Im having hard time implementing it thanks.

David Edwards

Hi John,
Your problem is actually the next thing I'd like to tackle related to the fetcher. Right now you will have to do a string comparison to select the correct sid, for example

if symbol_from_fetcher == context.my_sid.symbol:  
    # Do stuff

Ideally, within the pre_func each symbol would be broken out into a symbol column, that way a 'universe_func' can be used to map the data directly to a sid which would simplify things.

If I find some time I will set up another example that maps the data to sids and makes life easier. There are several ways to do this but they all seem to have edge cases.

David Edwards

I went ahead and put together a different way to organize the data here.

Each pair/basket is given an ID to associate stocks with baskets and there's now 2 files being fetched, one is the stock symbols along with the baskets they belong to along with the stock specific parameters for each basket. The other file is a metadata file containing the other basket parameters.

Doing this allows for a dynamic universe and the model parameters can be added the stocks data object. The metadata file contains the fields for long/short entries, constants, etc. This feels like a better way to access the data within the algo, but it will still take a bit of work.

I had to make flat dicts for each parameter to avoid difficult parsing. Go to the links in the fetch_csv file to see how the data is structured.

You've successfully submitted a support ticket.

Our support team will be in touch soon.