Its so stranged, What was wrong for me,I just did Clone Algorithm and then Click "Run Full Backtest",
Why I cannot reproduce the same result!
In my environment , I cannot set the backtest start date from 2002-01-04, after I manual input or click from calendar at 2002-01-04 then it always be set back to 2002-01-03.
Interesting. I have 'felt' this issue before. My time zone is usually EST+5 and sometimes I can't run a backtest to the same end date as a US user i.e. later today they will run a backtest ending 02/19/2014 but I will still be limited to 02/18/2014 for a while. You seem to be seeing the problem at the other end. What is your time zone?
@Dan: has this come up before? I've never mentioned it before.
@AM: Why such a huge difference with a one day shift in start date?
P.
Hi Anony,
I added an additional function (capital_invested) and some plotting--perhaps useful.
Grant
I was curious why changing the backtest date by one day, from Jan 3 to Jan 4 2002, changed the algorithm results from 200% to 600% (!). So I started digging around.
I began to debug by limiting the backtest to the year 2002 and discovered the following:
- The algorithm begins to trade in August 2002 (see the custom charts and logs)
- It trades on a 15-day interval cycle. This set of 15 days changes if you start the backtest on Jan 3 or Jan 4 ( see logs)
- Based on the comment above, it's sensitive to the start date
I noticed interesting activity in November 2002, so decided to investingate further
- The stock driving the November portfolio returns is ALXN
- I investigated the entry and exit points for ALXN depending on the start dates. Based on the cycle mentioned above, it took different positions per date.
- Jan 3 backtest [ALXN details]: on 11/21/2002 BUY 22556 shares and on 12/15/2002 SELL -32003 shares -> Take a long position
- Jan 4 backtest [ALXN details]: on 11/24/2002 SELL -17320 shares and on 12/16/2002 BUY 7423 shares -> Take a short position
- ok, so now we found the major points of discrepancies!
- Jan 3 backtest [ALXN details]: on 11/21/2002 BUY 22556 shares and on 12/15/2002 SELL -32003 shares -> Take a long position
- The stock driving the November portfolio returns is ALXN
Since this algo is based on RSI, I looked at the RSI behavior for ALXN. The RSI is different between the runs, as you can see on the custom graphs
- Implication: RSI for ALXN is radically different depending on the day you start the backtest and which periods you cycle through. Important to note that RSI is a path dependent indicator. This is compounded as the backtest continues to run to 2014.
Feel free to clone the attached backtest and compare the runs on Jan 3 and Jan 4. Note that the custom graph only allows you to record 5 variables at a time, but you can uncomment any of my lines to track the behavior. To see more details on the custom graph, you can click on the variable name to add/remove it from the chart.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
A path dependent indicator means that future calculations depend on the history of the algo and decisions made in the past. It may depend on the portfolio characteristics, current positions, and trade behavior. Thus, if any of these details change, your indicator value will be different. And it may not be consistent from backtest to backtest if you change the dates.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Hello AM,
RSI calculated via TA-Lib as implemented by Quantopian will produce the same value irrespective of the start date. See attached run with different start dates. See also https://www.quantopian.com/posts/ta-lib-rsi-using-something-other-than-close-price and https://www.quantopian.com/posts/ta-dot-macd-values-dont-match-google-or-yahoo-finance-charts
Hello Alisa,
Your algo shows 18.24 as the RSI form Aug 29 to Sep 19 whereas mine shows values from 42.18 through to 64.65. I don't uderstand why.
P.
Hello AM,
I think the argument can be made the Quantopian solution is better for walk-forward testing.
That set aside, what cause the very different results with a one day change in start date?
P.
@Peter, I had cloned an earlier post of the algo, before it was updated by AM. That's likely the source of difference
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.