Syncing Fetcher with your Mac

Hi all,
I have a couple algos that I swap securities in and out of fairly often and didn't like having to stop them every time so I set up a script to update them via the fetcher. I thought it was pretty cool, so I thought I'd share. This only works with Macs (sorry pc folks).

There are some drawbacks to this, the biggest one is that your algo will depend on your local machine and internet connection for updates. Also, this sort of thing will most likely not be allowed for managers program algos because it allows for changing algo settings on the fly.

I'll demo the idea with an example, suppose you have an algorithm that rebalances according to fixed weights, like the algobuilder. You will need to have a hosted directory on your computer, I'll use dropbox.

Here's the gist

Keep your current universe and weightings in a file on dropbox (portfolio.csv used here)
In the same directory, keep a file that will be fetched by your algo each day (fetcher_universe.csv here)
At the end of each evening, append the contents of portfolio.csv to fetcher_universe.csv to be used for backtesting and/or live trading.
Automate this process so that it happens every day.

First, create fetcher_universe.csv, I will just add the columns.

FETCHER_FILE = 'path to fetcher_universe.csv'

f = open(FETCHER_FILE, 'w')  
f.write("Date,symbol,weight")  
f.close()

Next, I'll assume you created a portfolio.csv in the same directory with the following format.

symbol,weight  
SPY,0.5  
TLT,0.5

Now, write a script that will append the contents of portfolio.csv to fetcher_universe.csv

"""
update_fetcher.py  
Appends contents of portfolio.csv to fetcher_universe.csv.  
This assumes portfolio.csv and fetcher_universe.csv are located in the same directory as this file.  
"""

import os  
import sys  
import datetime  
import pandas as pd  
from zipline.finance.trading import TradingEnvironment

ENV = TradingEnvironment.instance() # Used for checking if tomorrow is a trading day

ROOT = os.path.dirname(os.path.abspath(__file__))  
FETCHER_FILE = os.sep.join([ROOT, "fetcher_universe.csv"])  
PORTFOLIO_FILE = os.sep.join([ROOT, "portfolio.csv"])

def append_rows():  
    """  
    Appends the contents of PORTFOLIO_FILE  
    to FETCHER_FILE with tomorrows timestamp  
    if it is a trading day.  
    """  
    tomorrow = ENV.normalize_date(datetime.datetime.now()) + datetime.timedelta(days=1)  
    if tomorrow not in ENV.trading_days:  
       return  
    tomorrow = tomorrow.isoformat()  
    data = pd.read_csv(PORTFOLIO_FILE)  
    with open(FETCHER_FILE, 'a') as f:  
        for i in data.index:  
            row = ",".join([tomorrow, data['symbol'][i], str(data['weight'][i])])  
            f.write('\n' + row)  
        f.close()

if __name__ == '__main__':  
    append_rows()  
    sys.exit(0)

Next, you will want to automate this process so you can forget about it if there are no changes to your portfolio.

Apple supports launchd for automating background jobs on a mac. You will have create a new "plist" file in your users "~/Library/LaunchAgents" directory. Apple uses backwards url style filenames, eg. com.dedwards.update_fetcher.plist, not sure why, but I'll go with it.

The plist file is an xml style file that contains the relevant information for running your script. The important pieces are a label, the schedule, and program arguments. If your mac is asleep when the update occurs, it runs when it's woken up next. Because the forum formatting is being a pain about formatting XML, you can see what mine looks like on GitHub. It will call the script located at /Users/dedwards/dropbox/portfolio/update_fetcher.py at 6:45pm every day.

Apple supports launchctl for managing your LaunchAgents, there are some third party apps for managing this stuff, but the command line tools are usually enough. The most common commands are load, unload, start, stop, and list. See the link above for reference.

The following command should start your daily portfolio updates.
launchctl load ~/Library/LaunchAgents/com.dedwards.update_fetcher.plist Now, if everything went according to plan, in order to change your universe and portfolio weightings for the following day, you will only need to update portfolio.csv instead of killing your algo. I will post a reply with an example algo to use this with in the near future.

Hope somebody finds this useful.

Cheers,
David