a simplified version of dans algo, using 2x leverage and rebalancing every 5 days

Dear sir,

Could give us a fixed period and money investment example that strategy could let user specify a fixed invest money by each month period.
example like following: Initial total invest amount $10000 dollar , and then continue invest $5000 dollar for each month

I try to modify the algorithm to do systematic(periodic) investment plan for every 30 days , I using following redeposit code , but this seems not work, could help ?

context.portfolio.cash = context.portfolio.cash + context.reinvest_cash  

Hi Devon,

Our backtester doesn't have a great way of simulating in-flows and out-flows currently. Our performance metrics will treat inflows like profits and outflows like losses.

The best example I've seen that sort of does what you're looking for is this algorithm Dan Dunn shared on Dollar Volume Averaging. Note that it does not address the performance and risk measure issues, but it does let you simulate trading with a growing capital base.

I think supporting a more realistic simulation of an account with in/outflows is a great product suggestion and I've added it to our issue queue.

Best, Jess

This is interesting. I'd like to figure out why this algorithm gives (or appears to give) better returns than S&P 500. Here are a few ideas, which might be misguided or based on wrong information etc, so feel free to correct me if you know better (I'm here to learn, not to beat about the bushes):

1. If the transaction costs in the backtest are set too low (underestimated), then one would expect to be able to beat the S&P500, even by tracking the same portfolio, just by rebalancing more often.

2. Slippage might be underestimated, and this could be the cause of the additional returns, in other words: if you were to trade this algorithm live, the returns would be diminished from what you see in the backtest.

3. (I think this is most likely): The weight when rebalancing this portfolio has been chosen to be equal for each sector. However, the S&P500 is market value weighted. This means that each stock in the list of 500 stocks is weighted based on its value at the time of re-balancing. Presumably this does not give equal weights for each sector? If this is true, then consider that it is possible that one (or more) sectors happened to have higher returns over the past 10 years compared with the others (this is guaranteed to happen). If the weight of this sector was lower in the S&P500 (due to a smaller total market capitalization), then the difference in returns between this algo and the S&P500 could be explained by the additional weight (i.e. holdings) for the sector that happened to do better. If this is true, I think it means that the fact that the algo beats the S&P500 in the backtest does not mean that it would necessarily beat it in all future scenarios. It depends if equal weights of sectors beats market cap weighting, and that could change over the years. Perhaps this could be viewed as an additional risk factor that cannot be seen in this backtest.

Hi Gili,

I posted a more in-depth answer just now in a new thread. Short answer is that your #3 is the main driver of outperformance. I'd certainly agree with you that this strategy would not beat the benchmark in "all future scenarios", but actually I think the equal-weighted strategy has a smaller exposure to a size-based risk factor than the SPY does. That's a bit of a semantic point perhaps though :)

Best wishes and thanks for the thoughtful question! Jess

Thanks, Jessica! I read your reply in the other thread. It might be instructive to plot the average return of the overweight sectors vs. the average return of the underweight sectors (as a function of time - daily or minutely). My guess is that you will indeed find that the overweight sectors have a higher return. If so, this is perhaps surprising, since Financials and Technology are in the underweight group, which most people might label is "high return". However, on the other hand, it could be that over this period the returns for Utilities, Materials and Staples were more stable (as one would expect) than Financials and Technology which most likely also suffered more in the crash. Since the holdings for each sector within S&P500 changes over time, it could also be that different sectors are overweight and underweight at different times.

I also wonder if there are significant periods in the backtest where S&P500 actually had higher returns, and in that case it could be that this is difficult to see due to an initial period with higher returns for the equal weighted strategy.

Just to clarify, at each time step, I think one would have to find the market cap percentage for each sector, figure out which are the overweight and underweight sectors (defined as more than 0.5% different from 11%, for example). Then find a separate weighted average for the over and underweighted groups, which are weighted by the difference from 11%. This is complicated by the S&P500's composition changing over time.

Typo guys - and note to those who have cloned this popular algo:

set_commission(commission.PerTrade(cost=0.03))  

should be

set_commission(commission.PerShare(cost=0.03))

If only we could have 3 cents per trade commissions! :) As noted, this does not affect real-time trading, but in backtesting if you rebalance a lot, it is worthy of note.

Hi Ken - thanks for pointing that out.

I think this snuck in because we had the autocomplete set to fill in $0.03 for commissions regardless of whether you chose the PerTrade or PerShare basis - we have since corrected that problem and now autocomplete on PerTrade basis defaults to $1.00.

Since IB's 'flat rate' costs are $1.00 minimum per trade, $0.005 per share thereafter (to a max of 0.5% of the trade value) I think that the $1.00 per trade default is the most realistic option to use unless your strategy will be placing trades of 200 shares or more on a routine basis.

Best wishes, Jess

I agree, in fact allowing you to specifying both parameters and the function pick the max (what IB does) might not be a bad idea :).

Turns out that an equal weighted S&P500 index exists: SPW. It might be worth comparing this algo's performance to it, as well as ETF's that track it, such as Guggenheim Equal Weighted S&P500 ETF. Specifically, you'd have to have higher returns than the ETF for it to be worthwhile to algo trade this. The ETF might have an advantage in lower transaction costs, but perhaps with a smaller portfolio one could snatch some deals that the much bigger ETF cannot (but this might be hard).

    # If it's a rebalance day (defined in intialize()) then rebalance:  
    if  context.rebalance_date == None or exchange_time > context.rebalance_date + datetime.timedelta(days=context.Rebalance_Days):  

Not a big deal but the above line makes the algo weekly reblance to move 1min out every time it rebalances. changing the ">" to ">=" makes sure the rebalance happens on at the same time every week.

    # If it's a rebalance day (defined in intialize()) then rebalance:  
    if  context.rebalance_date == None or exchange_time >= context.rebalance_date + datetime.timedelta(days=context.Rebalance_Days):  

Hi Gili,

Now that we have the ability to swap out the default benchmark (SPY) for any security in the universe you can actually run a backtest of this 9 sector equal ETF strategy and look at returns and risk metrics against something like the Guggenheim Equal Weighted S&P500 ETF you mentioned (ticker: RSP). Note: The cumulative performance of the RSP benchmark doesn't include the 0.4% expense ratio that you'd pay to own that ETF, whereas the algo performance includes $1.00 per trade transaction costs for a fairly aggressive weekly rebalance.

Also, the RSP (or the equal-weighted SP500 index) can have pretty strong sector exposures based on what securities are selected to be included in the SP500 at any given time. So for example if you look at the RSP's sector exposure right now (see table below) you can see that it is overweight financials and consumer discretionary and underweight utilities, materials, and telecom.

Interesting, I didn't realize that RSP is not actually equal weight in sectors. Presumably it is equal weight within the sectors, otherwise nothing about it would be equal weighted...

It's nice that we can now benchmark against other indices, stocks, etc, but could this information somehow appear in the plot? (or maybe it does, but I don't see it?). For example, instead of the word benchmark, it could say SPY or RSP or whatever the benchmark was, or maybe better "Benchmark (SPY)". Otherwise I don't see an easy way to see against what asset the algo was benchmarked against. Even in the code it is not that transparent, since if I understand correctly, the only change (for changing the benchmark), is changing the sid in "set_benchmark = sid(24744)".

Ken,

Per your post:
"I agree, in fact allowing you to specifying both parameters and the function pick the max (what IB does) might not be a bad idea :)."

Note that this functionality already exists. Try the following:
set_commission(commission.PerShare(cost=0.013, min_trade_cost=1.3))

As an aside, note that IB's 'flat rate' costs are actually $1.30 minimum for US API directed orders (always read the fine print (-;). Be sure to click on the arrow for Exceptions (API, Global X ETFs, ...) to view the fees for US API Directed Orders.

In addition, the per share cost is $0.013 for transactions with 500 or fewer shares and $0.008 for more than 500 shares.

All.

I'm really taken by the powerful simplicity of this algo. I've been modifying and live testing it for some time now. I decided it was time to give back to the community and more importantly get the benefit of your review. The most significant change is probably the added ability to weight each sector differently. This allows you to take advantage of short term imbalances among sectors. Doing so also diminishes the mean reversion characteristics of equal sector weighting as noted by Jessica S. in her 4/21/14 post above.

I look forward to your feedback.

Best,
Tom

Jessica, Alisa,

Is there an explanation of virtual machine boot resets/restarts/schedules? This is in relation to production preparation of a strategy that may set various state variables during initialization. I read about the system reset on outages in the Help, and the fact that there won't be a system restart until the next day on an outage event, but what about scheduled down times and restarts?

This applies to the strategy above that the Q is running in production mode as an example. Within this strategy a global state variable "context.rebalance_date = None" is cleared on initialize. As each minute is processed, and the scheduled exchange time finally comes into view a rebalance will occur. If a strategy is restarted then this state variable will be cleared again forcing another rebalance on the next exchange trading window. Restarting this strategy daily would result in a rebalance, daily. (This is obviously not an issue for back test where only one initialize is called.) A rebalance after a reset may not be avoidable, but still, what awareness was considered for this eventuality?

In brief, in production (live trading), when are strategies VM's reset, or the strategies themselves restarted? And what other conditions or issues should a production system be aware of with regards to trading platform constraints, contingencies and such?

@Jessica: you say "The cumulative performance of the RSP benchmark doesn't include the 0.4% expense ratio that you'd pay to own that ETF" (4-21-14). My understanding is that the expense ratio is not a direct charge to the investor but taken out of net assets and reflected in the lower share price or dividend rate. So the RSP ETF DOES include the 0.4% expense ratio, and we don't have to model this expense. If I am wrong let me know so we can model what would be a substantial additional drag to the portfolio !

@Market Tech,

Live trading algorithms are kept logged in every day, and automatically get ready in the morning for the trading day. We run the algorithms on servers through Amazon Web Services, and if one of their servers is scheduled for decommission, we will notify you. We'll schedule a day after market close to move your algorithm to a new server - which we handle in our back end.

All you need to do is choose which day you want to relogin to your IB algorithm, and login after receiving an automated email from us. It's easy!

When this happens, your algorithm is not "restarted", it will pick up from the place it left off. There are a couple rough edges here to be aware of. I would recommend to use "context" on your variables to save state. And also avoid hard-coding a specific day or time. What happens if the algorithm got disconnected (server, login, infrastructure issues to name a few options) during the moment it was supposed to execute a trade? Instead, you should use "smart" logic to trade at or at the next available specific date or time. In the example we posted, you can see this on line 59

exchange_time.hour < context.rebalance_hour_start or exchange_time.hour > context.rebalance_hour_end:  

And now there's the new schedule_function which makes it even easier.

Alisa, thanks for reply.

Is is safe to say then that the Q's "context" is effectively a state object that is persisted for the life of an algo, after initialization, until at what time a quant is notified that the algo may be moved and restarted?

@Paul Yes, I believe you are correct. If you are using the actual ETF, such as RSP, as the benchmark you specify in your backtest then the expense fee is already rolled into the price of the ETF (If I recall how ETF fees work is that a little bit is taken each day of which sums up to the total annual fee over the course of a year).

@Market Tech, yes that's exactly right.