I develop corporate event based quantitative strategies in US equities for a hedge fund. I'm reading the Quantopian help section and it looks promising to help back test our strategies. However, it seems from most examples that I see that the backtesting is on a limited "universe" of stocks. Is it possible to do backtesting on the entire universe of listed US stocks?
Also, if I wanted to filter out certain types of stocks (ADRs, ETFs, REITs, for example), can I do that?
In the IDE you can test and trade up to 200 securities at a time. We recently relaxed the limitation from 100 securities, and we're always working to improve this limit for faster backtesting! There are 3 ways to initialize securities in your algorithm using a manual lookup, set_universe for a basket of stocks, or Fetcher to import custom data.
On option is to screen the full list of stocks and whittle the list to 200 before uploading them into your algorithm. If you're interested in this approach, take a look at this thread: https://www.quantopian.com/posts/test-a-daily-trading-strategy-using-history-and-more-than-500-stocks-api-limitations
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Thanks Alisa. Based on my reading of the help section as well as my reading of the post you sent me to, can I confirm that the universe of stocks can be different each day? So if I'm backtesting the impact of dividends, say, on the price of the stock of the issuing firm, if I refresh the universe each day, then I'm limited only if the number of dividend firms exceeds 200 on a particular day? (Assuming that I have a csv that I grab via Fetcher with the dividend tickers and dates).
Yes, the universe of stocks can be different each day. To have a stock leave your universe, you will need to have a zero position in the security.
Keep in mind that if you "refresh the universe" and close out the position, you will not get the dividend for the stock later in the backtest because you don't hold the security. So you'll need to do some careful coding to control the universe entry and exit positions.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
That's exactly the reason - we want you to be able to run fast backtests in a reasonable time and host the computations on our servers. Once the performance of the backtest improves, we will likely raise the cap again to allow for more securities. Right now the backtest can process data for up to 200 securities, every minute, and perform intensive calculations in each bar.
If you'd like to develop offline, you can use Zipline which is the backtesting engine that we open-sourced. The full code is available on Github and there's a Google group for discussions.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Alisa, so the 200 security cap is only when working with minutely data? With daily data, there is no such cap? I saw this distinction from Jessica in the following post:
https://www.quantopian.com/posts/creating-a-universe-of-all-of-the-stocks-in-a-single-index
If you're working with minutely backtest data we cap you at 200 securities I believe
@Derek, If the code is slow and can't process a call of handle_data within ~50 seconds, you will receive a timeout error and the algorithm will stop trading. This is true for both backtesting and live trading. The algorithm will stop to prevent falling behind the market feed.
@Ashish, the 200 security cap exists for both minutely and daily mode backtests. The one possible work-around is you use 10% of a universe using set_universe in daily mode, which is roughly 800 stocks. The one caveat is you can't use this setting for live trading with IB because you're required to run a minute-mode full backtest to simulate the live trading experience. Perhaps you'd like to use this route to test your strategy, and then can reduce the universe size and convert to minute mode backtesting.
If you're developing in daily mode, remember that you're only sampling data once per day, at the last bar. This means that your data has more variation because there can be large price differences at the end of day, depending on the stock.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
