Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Is there any way to Speed Up Backtesting

Hi there,

It is taking me more than 10 minutes to perform a backtesting from 2002 to 2014. I am testing with minute data and not tickdata; hence, it is painful.

6 responses

What type of algorithm are you running? Are you using fundamental data or bar data or both? How often do you buy/sell/re-balance? Depending on the answers, you may be able to short-circuit the cycles when you aren't actually doing anything and therefore speed up the algorithm execution time.

You can also choose a shorter back-test time range.

Chosing a shorter back-test time range is not an option. Below is my code.

'''
    This algorithm defines a long-only equal weight portfolio and rebalances it at a user-specified frequency  
    NOTE: This algo is intended to run in minute-mode simulation and is compatible with LIVE TRADING.

'''
# Import the libraries we will use here  
import datetime  
import pytz  
import pandas as pd

def initialize(context):  
    # This initialize function sets any data or variables  
    # that you'll use in your algorithm.  
    # You'll also want to define any parameters or values  
    # you're going to use.

    # In our example, we're looking at 9 sector ETFs.  
    context.secs = symbols('XLY')  # XLY Consumer Discrectionary SPDR Fund  
                          # 'XLU')  # XLU Utilities SPRD Fund

    # Change this variable if you want to rebalance less frequently  
    context.Rebalance_Days = 1

    # These other variables are used in the algorithm for leverage, trade time, etc.  
    context.rebalance_date = None  
    context.weights = 0.99/len(context.secs)  
    context.rebalance_hour_start = 10  
    context.rebalance_hour_end = 15

    # These are the default commission and slippage settings.  Change them to fit your  
    # brokerage fees. These settings only matter for backtesting.  When you trade this  
    # algorithm, they are moot - the brokerage and real market takes over.  
    set_commission(commission.PerTrade(cost=0.03))  
    set_slippage(slippage.VolumeShareSlippage(volume_limit=0.25, price_impact=0.1))

def handle_data(context, data):

    # Get the current exchange time, in the exchange timezone  
    exchange_time = pd.Timestamp(get_datetime()).tz_convert('US/Eastern')

    # If it's a rebalance day (defined in intialize()) then rebalance:  
    if  context.rebalance_date == None or exchange_time > context.rebalance_date + datetime.timedelta(days=context.Rebalance_Days):

        # Do nothing if there are open orders:  
        if has_orders(context):  
            print('has open orders - doing nothing!')  
            return

        rebalance(context, data, exchange_time)  

def rebalance(context, data, exchange_time):  
    # Only rebalance if we are in the user specified rebalance time-of-day window  
    if exchange_time.hour < context.rebalance_hour_start or exchange_time.hour > context.rebalance_hour_end:  
       return

    # Do the rebalance. Loop through each of the stocks and order to the target  
    # percentage.  If already at the target, this command doesn't do anything.  
    # A future improvement could be to set rebalance thresholds.  
    for sec in context.secs:  
        order_target_percent(sec, context.weights, limit_price=None, stop_price=None)

    context.rebalance_date = exchange_time  
    log.info("Rebalanced to target portfolio weights at %s" % str(exchange_time))

def has_orders(context):  
    # Return true if there are pending orders.  
    has_orders = False  
    for sec in context.secs:  
        orders = get_open_orders(sec)  
        if orders:  
            for oo in orders:  
                message = 'Open order for {amount} shares in {stock}'  
                message = message.format(amount=oo.amount, stock=sec)  
                log.info(message)

            has_orders = True  
    return has_orders  

I think your code looks OK. You seem to be avoiding running handle_data on every bar tick, which would be the main way to speed up the algo. A 12 year back-test just takes a while on Quantopian.

The slow down comes from Zipline, then some extra from Quantopian like live chart plotting. Zipline does a lot of computation and store a lot of frames on each bar. Which for optimisation are not really needed. I started to strip zipline down to speed it up so I could run fast optimisations then I found qstk which provides enough for preliminary optimisation. Compared to zipline it is way faster but it only provides basic risk functionality.

Would a stronger wifi connection or faster computer make a difference?

LOL, no. Can't wait until there's a war to get colocation with Quantopian servers ;)