Automatically exporting Zipline code (multiple files) to Quantopian-compatible code

Back to Community

edited

Dear all,
To design strategies prior going live, I needed a debugging tool and the ability to maintain multiple files.
I'm designing everything under Linux, in Zipline (again, thanks quantopian for this).

Thus, I've elected to follow a specific structure for each strategy design that allows me to automatically export a Zipline strategy involving multiple files, into a Quantopian single-file strategy.

The structure I proposed and accompanying bash script to automatically generate Q from Z is on github:
https://github.com/florentchandelier/zipline2quantopian

In that github repo, I have elected to add an example paired strategy (that I've previously shared to understand the difference betwen Q and Z). The exported Zipline code is provided as backtest hereafter.

HTH

Updated on Post April 12th .... new ATS architecture.

8 responses

John Fawcett

Florent this is really cool! Thanks for sharing your system - this is why we love working with the open source community.
I especially like the idea of a standard for the directory structure. That makes it easier for people to quickly get up to speed on one another's projects.
Have you tried our in-browser debugger? We just recently introduced it, and I think it is a great compliment to your setup. You can activate it in the IDE just by clicking on a line number to set a breakpoint and then clicking build.

thanks,
fawce

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Femto Trader

Thanks Florent for this contribution.
I was also wondering how to manage several shared files with Quantopian and your proposal could fit my needs.

Erick Gomez

Florent, Yes this is great. Do you have plans to do the other way around Quantopian2zipline? It looks like I am no able to use my strategies from Quantopian work out as-is of the box to zipline. I am getting NaN values with the history :( Do you know what could be happening. I am calling it like this in my Linux box

run_algo.py -f spy-tlt.py --data-frequency daily \
--start 2013-01-30 \
--end 2015-02-03 \
--capital_base 5500 \
--output biooutput \
--symbols "TLT","SPY"

Joe Jevnik

There are two notes about history in zipline:

history does not backfill data, you will need to manually lag your algorithm the appropriate amount of bars
Because we do not backfill: dynamically reshaping your history container will not look back for the data and you will get NaN's until it is 'warmed up'.

You should make sure your container will be large enough by using add_history in initialize. Then, in handle_data, keep a counter of bars and use something like:

def initialize(context):  
    context.skipping = 10  # How many bars do you need?  
    add_history(10, '1m', 'price')

def handle_data(context, data):  
    if context.skipping:  
        context.skipping -= 1  
        return  
    hist = history(10, '1m', 'price')

Disclaimer

Erick Gomez

Joe,

Thanks that have solve my problem. However looking into getting the 1minute data history. I am not getting the data. I have check the source code for the zipline and it looks like it is ignoring it.
I have the latest version from the zipline github. Do you know if the minute inverval are supported with Yahoo or I need to contract an extra service?

Regards,
EG

Market Tech

Yahoo historical only support d,w,m.
IB (if you have an account) will give you minutely (60 downloads every 10 minutes).
IQFeed is fairly cheap and will give you FX, futures and equities.
MBT has equity accounts and you can fetch FX, futures(limited) and equities.

Erick Gomez

Market tech,

How do i connect it with my Linux zipline? by default the "source" is yahoo.
EG

Florent Chandelier

Updated architecture: PortfolioManager, OrderManager and StrategyDesign & StrategyPortfolio
The objective was to track instruments' positions per strategies (see the multi-strategy example), even though the strategy are using percent_orders()
The Quantopian file is still automatically generated from multiple zipline files, through an automated script.
Refer to https://github.com/florentchandelier/zipline2quantopian

You've successfully submitted a support ticket.

Our support team will be in touch soon.