Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Minimum variance using daily vs min backtesting

I have been trying to use minimum variance optimization for a few ETFs. I have the problem of getting completely different results with daily Vs min backtests, even after I control for closing price. I merged the code from a few places - equal weight sample algorithm, David's min variance code using scipy fmin, etc, so you might find it a little inconsistent. I will post the minute based back-testing in the next post. One change here, the weights are not constrained to be positive, which needs to be changed but it still doesnt explain the differences as the log shows exactly the same numbers atleast after first rebalancing

7 responses

Here is the min based backtesting. The log says I do get the identical closing day price as the daily backtesting. But the returns are different

Looks like your log is telling you what price you're trying to order at, but not what price you're clearing at.

Default behavior on Quantopian is that minute mode orders clear at the closing price of the next minute. Daily mode orders clear at the closing price of the next day. Both of course plus default slippage.

Search on the forum for the "trade at the open" slippage model which is designed to approximate trading at the following day's opening price in daily mode.

Thanks Matt, how do I look at the clearing price for my order and print it in the log?

The easiest way to do that would be to run a full backtest and then look at the Transactions tab.

If you really need to see it in the logs instead that seems a little trickier as it doesn't appear this value is stored anywhere explicitly. You can calculate it but you need to save a lot of state variables. It seems like you would have to:

1) Save the Order object that is returned by any of the order methods in an array stored in the context object
2) In handle_data, query your array of orders to see if any are now filled which were not previously filled (so you'll also have to track previous and current states)
3) Then in context.portfolio.positions, look up the corresponding security and back out the clearing price from the weighted average cost basis (which means you will also have to know the number of shares and cost basis from the previous period for each security).

You could shortcut some of that if you knew without doubt that your algorithm was entering/exiting positions entirely (as opposed to accumulating/divesting positions partially over time) by just querying context.portfolio.positions and looking at cost basis / number of shares. But if you are going to add/exit positions partially over multiple time periods that won't work since cost basis is an average (Quantopian doesn't store individual lots right now).

Unless you have an operational need to know the clearing price in real time, using the transaction log from the full backtest seems a lot easier.

Using transaction logs makes more sense for my requirement, thanks.

I am finding it hard to sync up the minute-wise backtesting order execution with the daily backtesting order execution. I am not sure how to specify - "place the order two trading days from now at 4:58PM". Basically I am not sure how to keep track of days in the minute-wise backtesting. Are there any code examples for this use case?

You can create a custom counter to track the number of days, check out the example below.

The logs say:

2015-01-05handle_data:12INFONumber of days is 1  
2015-01-06handle_data:12INFONumber of days is 2  
2015-01-07handle_data:12INFONumber of days is 3  
2015-01-08handle_data:12INFONumber of days is 4  
2015-01-09handle_data:12INFONumber of days is 5  
End of logs.  
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.