Data Available for 26 Countries
Today, we added the first global equity data to Quantopian. Global equity pricing data and fundamentals for 26 countries are now accessible via the Pipeline API in Research. The table below lists the newly supported countries as well as the corresponding supported exchanges.
EDIT: This table is out of date. Quantopian now supports exchanges in 44 countries. The complete list can be found in the Data Reference.
| Country | ISO Alpha-2 Code | Supported Exchanges |
|---|---|---|
| Austria | AT | Vienna Stock Exchange |
| Australia | AU | Australian Securities Exchange, National Stock Exchange of Australia |
| Belgium | BE | Euronext Brussels |
| Brazil | BR | Sao Paulo Stock Exchange |
| Canada | CA | Toronto Stock Exchange, TSX Venture Exchange, Canadian Securities Exchange |
| China | CN | Shenzhen Stock Exchange, Shanghai Stock Exchange |
| Denmark | DK | NASDAQ OMX Copenhagen |
| Finland | FI | NASDAQ OMX Helsinki |
| France | FR | Euronext Paris |
| Germany | DE | Berlin Stock Exchange, Dusseldorf Stock Exchange, XETRA, Frankfurt Stock Exchange, Hamburg Stock Exchange, Hannover Stock Exchange, Munich Stock Exchange, Stuttgart Stock Exchange, Xetra Indices |
| Great Britain | GB | London Stock Exchange, ICAP Securities & Derivatives Exchange, Cboe Europe Equities CXE |
| Hong Kong | HK | Hong Kong Stock Exchange |
| India | IN | Bombay Stock Exchange, National Stock Exchange of India |
| Ireland | IE | Irish Stock Exchange, Irish Stock Exchange Bonds & Funds |
| Italy | IT | Milan Stock Exchange |
| Japan | JP | Tokyo Stock Exchange, JASDAQ, Osaka Exchange, Nagoya Stock Exchange, Fukuoka Stock Exchange, Sapporo Securities Exchange |
| Netherlands | NL | Euronext Amsterdam |
| New Zealand | NZ | New Zealand Stock Exchange |
| Norway | NO | Oslo Exchange |
| Portugal | PT | Euronext Lisbon |
| Singapore | SG | Singapore Exchange |
| South Korea | KR | Korea Exchange, Korea KONEX |
| Spain | ES | Madrid Stock Exchange/Spanish Markets |
| Sweden | SE | NASDAQ OMX Stockholm, AktieTorget, Nordic Growth Market |
| Switzerland | CH | SIX Swiss Exchange, BX Swiss AG, Swiss Fund Data |
| United States | US | NYSE, NASDAQ, AMEX |
Pricing and fundamentals data for each of these countries is available as far back as 2004. The pricing data is daily, including OHLCV bars. Global fundamentals are sourced from the recently announced FactSet Fundamentals data integration. All global equity data is accessed using a new Pipeline feature, domains, described below and in the attached notebook.
International Pipelines & Domains
Pipeline is a tool that allows you to define computations over a universe of assets and a period of time. In the past, you could only run pipelines on the US equity market. As of today, you can now specify a domain over which a pipeline should be computed. The name "domain" refers to the mathematical concept of the "domain of a function", which is the set of potential inputs to a function. In the context of Pipeline, the domain specifies the set of assets and a corresponding trading calendar over which the expressions of a pipeline should be computed.
Currently, there are 26 domains available in the Pipeline API corresponding to the countries in the table above. The attached notebook explains pipeline domains in more detail.
Example Usage
The following pipeline returns the latest close price, volume, and market cap for all Canadian equities, every day. Make sure to run it in Research.
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data import EquityPricing, factset
from quantopian.pipeline.domain import CA_EQUITIES
from quantopian.research import run_pipeline
pipe = Pipeline(
columns={
'price': EquityPricing.close.latest,
'volume': EquityPricing.volume.latest,
'mcap': factset.Fundamentals.mkt_val.latest,
},
domain=CA_EQUITIES,
)
result = run_pipeline(pipe, '2015-01-15', '2016-01-15')
result.head() To learn more about domains, see the attached notebook. If you're curious, you can read more about the design behind domains in this Zipline issue.
Currency Conversions
One of the challenges in working with international financial data is the fact that prices and price based fields can be denominated in different currencies. Even if you only want to research stocks listed on a single exchange, currencies can present a challenge. For example, a company that is based in the US (like Apple), reports their financials in USD, but lists shares on exchanges around the world. These listings are often denominated in the local currency of the exchange. Having pricing and fundamentals data denominated in different currencies makes it hard to make comparisons between the two datasets.
To solve this problem, FactSet Fundamentals data is denominated in the listing currency of each asset. For example, if you query sales data for the US listing of Apple, you will get data denominated in USD. If you look up fundamentals for the Japanese listing of Apple (listed in Japanese Yen), you will get data from the same US report, but it will be converted to Japanese Yen. The specific conversion techniques are as follows:
Income Statement and Cash Flow Statement
Data from the income statement and cash flow statement of a fundamentals report is converted using the average exchange rate over the fiscal period in question.
Balance Sheet
Data from the balance sheet of a report is converted using the exchange rate from the end of the fiscal period of the report (i.e. the end of Q1 for a Q1 quarterly report).
Other Notes
Contest Eligibility
Unfortunately, international markets are not yet supported in the backtester, and by extension, they are not yet supported in the contest. We don’t currently have a timeline for a contest that supports international markets. We will provide an update to the community when we have more information to share.
Alphalens
Alphalens is currently the best tool for analyzing a factor on an international market. To date, we have suggested that you use get_pricing to get daily pricing data and supply it to Alphalens. International pricing data isn't yet available in get_pricing, which motivated us to revisit this suggestion. After poking around a bit, we put together a wrapper function that makes it easy to write a pipeline factor and run it through alphalens without having to write all of the get_pricing boilerplate code. The notebook attached in the comment below includes the wrapper function evaluate_factor. You can use evaluate_factor to evaluate a factor on any domain.
Holdouts
As described in this post, the most recent year of FactSet Fundamentals data is held out from the community platform, which applies to both US and international data. The same holdout applies to the daily OHLCV data for non-US markets. This means that all research on international factors will need to be conducted on data that is more than a year old. We are hopeful that the holdout will actually help to reduce the risk of overfitting.
Documentation
The notebook in this post is the best reference material on domains at this time. We plan to add documentation and other learning material around domains in the near term.
Multi-Currency Markets
Another challenge related to currencies is the fact that some exchanges don't require stocks to be listed in local currency. For example, the London Stock Exchange only has about 75% of its listings denominated in GBP*. The other 25% are primarily listed in EUR or USD. This can make it hard to make cross sectional comparisons.
To solve this problem, most people rely on currency conversions to bring price-based fields into the same currency. Currently, there's no way to convert pricing and fundamentals data, so everything is denominated in the listing currency (the default). In addition, the listing currency information isn't yet accessible, so you don't have an easy way to determine the currency of a particular data point. Our long-term plan is to solve this problem and make it easy to conduct analyses across assets listed in different currencies.
In the meantime, currency-dependent fundamentals fields are not yet available for assets that are listed in a non-primary currency of their exchange. Pricing data for non-primary currency assets is available, but you should use returns instead of raw prices to build a factor on a non-US domain (probably a good idea anyway!). Working in returns space instead of pricing makes it reasonable to make comparisons across assets listed in different currencies.
*Great Britain, Switzerland, and Ireland all have a significant number of assets listed in non-local currency. Other markets have the vast majority, if not all of their listings in local currency.
Combining Domains
Pipelines can now be defined with new domains, but the idea of 'combining' domains is not yet supported. When working with international equity data, it is a common use case to develop a factor at a regional level (for example, Europe) instead of a country level. We plan to support multi-country domains, but don't yet have a timeline on the delivery of this feature.
Known Bugs
(We will make an effort to update this list as we discover and fix bugs)
- Fixed: Fundamentals data for Sweden, Denmark, and Ireland are not yet available. We expect these to be available soon.* (Sweden and Denmark fundamentals data is now available)
- Fixed: Fundamentals data for Ireland is not yet available.
- Starting a non-US pipeline on a non-trading day (in the local market) raises an exception.
Try building a factor on a new domain and let us know what you think. As always, if you discover any issues that aren't listed above or if you have any questions about the new data, please let us know!
