I always get the closing price of the last trading day one day to late. Like this no real trading is possible.
Today is Monday morning 1 hours before the stock market opens, but the closing price of last Friday is still not available. I use daily prices. How can I get the latest closing price?
Regards Frank
Hi Frank,
In the backtester, we have data from Jan 2002 until yesterday. For example today, on Monday, you can backtest data that includes Friday's prices.
To get yesterday's close price, you'll want to use the history function. Here is an example:
def handle_data(context, data):
yesterday_close = history(2, '1d', 'price').iloc[0]
print yesterday_close The history function will return a pandas dataframe with 2 values: yesterday's daily close price and the current close price of the last bar. We want the first value, we index into the dataframe using ".iloc[0]" to get the price we want.
I suggest to take a look at the history documentation, and if you need more help, post back and we can help you work on your code!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
History() works with both daily and minute mode! So you can either get daily or minute level bars. You're right that it used to work only in minute mode, and we announced the upgrade about a month ago: https://www.quantopian.com/posts/history-is-now-available-in-daily-mode-for-backtesting
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
In fact I found now what is happening. If I do a full backtest, then I get the prices from the last trading day, however if I only build my project, then it will stop 2 days before the last trading day and it is impossible to get price data from the last trading date. This is a little bit strange.
This certainly sounds a bit strange - could you share the algorithm with me and I'll take a look? You can add me by pressing the "Collaborate" button and my email is [email protected].
Perhaps the last day of the backtest is a holiday, or a weekend? Either way, I can take a look to see what's going on!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Try to print the last trading day in debug mode. It does not work. The date is always adjusted 2 trading days ago. However if you run a full backtest it will print the prices of yesterday without a problem.
---------code------------------------
def initialize(context):
context.spy = symbol('SPY')
def handle_data(context, data):
prices = history(bar_count=1, frequency='1d', field='price')
myts = prices[context.spy]
print(myts)
Hi Frank,
When I print 'prices[context.spy]' in the debugger for date 10-22-2014, I get data for all four days: 10-17, 10-20, 10-21, and 10-22.
And in this case, in order to get the last row of data (10-22) you'd have to use '.iloc[-1]'. Here's an example of what I mean:
def initialize(context):
context.spy = symbol('SPY')
def handle_data(context, data):
prices = history(bar_count=4, frequency='1d', field='price')
myts = prices[context.spy].iloc[-1]
print(myts) Let Alisa or I know if you have any problems with this
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Run it with date Okt. 27 as last date in the debugger and then it will only work in the full backtest. Okt. 22 is not the last available close.
Hi Frank,
I'm sorry to hear that you're having problems with this. Can you tell us exactly what you're pasting into the debugger to get your output? I'm using October 27, 2014 as the last date in the debugger and I currently get data for October 22, 23, 24, and the 27th.
Seong
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Set the date to: 10/27/2014 to 10/28/2014 and press the "Build Algorithm" button. After running, the date fields are automatically reset to 10/27/2014 to 10/27/2014 and I am missing the closing price of Oktober 28.
log output----------------------------------------------------------------
2014-10-27PRINT2014-10-27 00:00:00+00:00 196.11
Name: Security(8554 [SPY]), dtype: object
End of logs.
If I set the date to: 10/27/2014 to 10/28/2014 and do a full Backtest, then I have the closing price of October 28.
log output----------------------------------------------------------------
2014-10-27PRINT2014-10-27 00:00:00+00:00 196.11
Name: Security(8554 [SPY]), dtype: object
2014-10-28PRINT2014-10-28 00:00:00+00:00 198.37
Name: Security(8554 [SPY]), dtype: object
End of logs.
Frank
That is indeed very strange. Could you invite me to collab on your algorithm and I can take a look? My email is [email protected]
Thanks,
Seong
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Hi Seong,
I've cloned Frank's algo and run it today (11/2/2014).
Here is the log output:
2014-10-27PRINT2014-10-27 00:00:00+00:00 196.11
Name: Security(8554 [SPY]), dtype: object
2014-10-28PRINT2014-10-28 00:00:00+00:00 198.37
Name: Security(8554 [SPY]), dtype: object
2014-10-29PRINT2014-10-29 00:00:00+00:00 198.14
Name: Security(8554 [SPY]), dtype: object
2014-10-30PRINT2014-10-30 00:00:00+00:00 199.45
Name: Security(8554 [SPY]), dtype: object
End of logs.
Friday is still missing. I've set end date to 10/31/2014, but it was reset to 10/30/2014 as Frank explained. This happens in Full backtest mode too and in both time frames.
Regards,
Ed
Hi,
I found the reason and workaround. Automatic date adjustment for back tests has a bug. It adjusts not only start date, but also end date.
Here is the output of Frank's algo with automatic date adjustment turned off:
2014-10-27PRINT2014-10-27 00:00:00+00:00 196.11
Name: Security(8554 [SPY]), dtype: object
2014-10-28PRINT2014-10-28 00:00:00+00:00 198.37
Name: Security(8554 [SPY]), dtype: object
2014-10-29PRINT2014-10-29 00:00:00+00:00 198.14
Name: Security(8554 [SPY]), dtype: object
2014-10-30PRINT2014-10-30 00:00:00+00:00 199.45
Name: Security(8554 [SPY]), dtype: object
2014-10-31PRINT2014-10-31 00:00:00+00:00 201.66
Name: Security(8554 [SPY]), dtype: object
End of logs.
Could it be fixed?
Regards,
Ed
Hi Quantopian developers,
Can you confirm that this is a bug in backtester or am I missing something obvious here?
Regards,
Ed
Hi Ed,
Is the problem you're experiencing that the backtest dare gets adjusted when you put it too close to the current date?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Hi Seong,
Yes, you can put it this way. This bug makes it impossible to get data for the last trading day without switching off automatic date adjustment.
For example if I run Frank's algorithm right now I can't see data for 11/4/2014 in daily mode. However, I can see it just fine if I switch off date adjustment.
Hi Ed,
I see where your confusion is coming from. From my experience, it takes some time to get the latest data into our backtesting, so often there will be some waiting time (about a day) before you can access the latest trading date (e.g. 11/4/2014 when it's 11/5/2014).
For example, you couldn't access 11/4/2014 before but as of now you can.
I hope that clears up any confusion
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Hi Seong,
I don't think we're on the same page here.
I still couldn't access 11/4/2014 as it's auto adjusted to 11/3/2014.
As I said I can only access data for last trading day if I disable auto adjusting.
Regards,
Ed
Hi Ed, I'm not 100% I understood your problem correctly. I think you are saying that end date adjusts you by one day so you can't run a backtest through the most recent data. Maybe the data from 11/4 wasn't completely merged into the db yet because I am having trouble reproducing the bug now.
I'm not sure if this helps you, but in a backtest, any orders placed do not get processed until the following bar. That means that while you can see the data for the most recent day, you can't place new orders because the next bar does not exist yet. Today is 11/5, so we have data through 11/4 in the database. If you run a minute mode test, you can place orders up until one minute before close on 11/4. However, in daily mode, 11/3 will be the last day that you can place trades, because the bar following 11/4 does not exist yet.
Let us know if you're having this issue? If so and it's not too complicated, could you please give me the steps to reproduce it.
David
Hi,
Maybe the data from 11/4 wasn't completely merged into the db yet because I am having trouble reproducing the bug now.
The data was there as I saw it when I switched off date adjusting.
Here are the steps:
If you still can't reproduce it do the following:
You can see that end date was adjusted to 2014-11-04 in UI and data for the last trading day (2014-11-05) is not present in the log.
So, data for the last trading day(2014-11-15) is available if date adjusting turned off.
I hope it helps.
Regards,
Ed
Hi Ed, thanks for reiterating the problem, I just wanted to make sure I was interpreting the issue correctly. This has been added to the que, in the meantime, turning off the date adjustment will be your best option, I believe the plan is to make it easier to toggle that setting as well. Sorry for the inconvenience.
Hi David,
Thank you for taking care of this!
BTW, do you have any plans to open your issue tracker that we can add and discuss bugs there?
Regards,
Ed
I know the zipline issues are public, but I'm not sure about the UI stuff, I will have to get back to you on that.
so for this function, now that history is deprecated, how do I use data.history to get the price of the last trading day?
Hi Anson, if you want prices for the last trading day for a specific equity, you can do something like this:
data.history(sid(24), 'close', 2, '1d')[-2] sid(24) is for AAPL, so change that to any equity (or a list of equities), and 'close' is for the close price, which you can change to something like 'open' or a list of fields. Our Getting Started Tutorial has more on data.history() in this lesson; I would encourage you to check out the whole tutorial if you haven't already. The tutorial will tell you that we specify 2 and -2 to get the second-to-last data point, since the last one is the current day.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.