New Data Available in Algorithms and Research

Quantopian's community platform is shutting down. Please read this post for more information and download your code.

Back to Community

New Data Available in Algorithms and Research

posted Feb 26, 2016

Using Data Q Updates

TL;DR: We've made lots more data available recently for use in algorithms.

Back in the fall, we announced our program to integrate more external data sources in Quantopian. With such a large community of individuals working on Quantopian, we're able to convince these data firms to make their very expensive data available to you for use here on Quantopian.

Over the last few weeks, we've made a lot more data available for use in algorithms. A quick run down of the progress:

-- Estimize crowd-sourced earnings estimates
We recently updated the Estimize consensus estimate data so that you can access it from your algorithms, using the pipeline API. Seong has already published a strategy which uses this data for investing in post-earnings announcement price drifts.

Further, we'll be holding a free webinar next Tuesday with Leigh Drogan, CEO of Estimize, and Vinesh Jha, CEO of ExtractAlpha and formerly of StarMine and PDT Partners. Be sure to register in advance.

-- PsychSignal Trader Mood
Using messages from Twitter and StockTwits, PsychSignal measures trader mood. You can write an algorithm with this data today using one of the 8 variations of the data we've integrated.

-- News Sentiment from Sentdex
One of our community members is Harrison Kinsely. Harrison provides great python tutorials via his pythonprogramming.net project, including tutorials for Quantopian. But he doesn't just do tutorials. Through his company Sentdex, he also provide news sentiment data. Sentdex provides a simple signal that assesses the sentiment of news articles. It can easily be added to an algorithm

-- Expanded volatility data from Quandl and CBOE
The most popular totally free data on Quantopian is the VIX index. As such, we've added 9 more volatility indicators generated by the CBOE and available to us through Quandl. Beyond VIX, we also now offer daily values for VVIX, VXV, RVX, SKEW, VXN, VXD, VXFXI, and VXXLE.

These new data sets are in addition to our existing data from firms like EventVestor and Accern, both of which can be used as signals for your algorithms as well.

Every single one of the premium data sets have free samples that makes it easy to try them out. Further, almost all of them have sample algorithms to clone and try out. You can also explore the data on Quantopian research. And if you want to live trade them, you can purchase a monthly subscription for full access to the data.

Happy coding!
Josh

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

16 responses

Is this data available for live trading yet? I remember in the past most of the data was only available via the pipeline api which did not support live trading.

Yes, it is now fully available in live trading -- in the contest, paper trading and real money with your brokerage.

Note, to live trade with this data, you'll need to subscribe to the premium feed(s) you are using in your algorithm (market data, fundamentals and the Quandl data sets continue to be free).

Thanks
Josh

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Josh,

The VIX historical data is free. But the VXN historical data must be subscribed?

Cheers

Thomas

VXN is completely free, similar to VIX.

https://www.quantopian.com/data/quandl/cboe_vxn

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

gabriel goldszmidt

Hey Josh!

For some reason I am not able to run any of my back tests. It gets frozen once I run the algorithm. I tried it with all my previous codes.

Any clue of whats going on?

Best,
Gabriel

We are tracking a major outage on http://status.quantopian.com/. Your backtests are actually probably kicking off in the background, but the results aren't making it back to your browser. Keep an eye (or subscribe) to the status page for future updates.

Sorry for the inconvenience.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Josh,

I've saw your notebook yesterday. And I have tried your example code at the end of the notebook. But it doesn't work. Could you please have a look at my question here:
https://www.quantopian.com/posts/the-notebook-example-for-vxn-does-work

Cheers

Thomas

I attach my algo here again. It's a copy of your example code.

And one of the Q-colleague Dan Whitnable has helfed me by VIX class. See here
https://www.quantopian.com/posts/how-to-apply-the-rsi-on-the-vix-factor#5891d7dc91895e5fc7dab140

Where he showed me an example for VIX class as follow:
class VIX_close(CustomFactor):
# Default inputs
inputs = [cboe_vix.vix_close]
window_length = 1
# Compute momentum
def compute(self, today, assets, out, vix):
out[:] = vix[-1]

I try to build the similar class for VXN as follow:

class VXN_close(CustomFactor):
# Default inputs
inputs = [cboe_vxn.vxn_close]
window_length = 1
# Compute momentum
def compute(self, today, assets, out, vxn):
out[:] = vxn[-1]

But I got error "Error Runtime exception: AttributeError: type object 'cboe_vxn' has no attribute 'vxn_close'. I wonder what members does the VXN has?

Cheers

Thomas

The naming of the 'cboe_vxn' dataset fields are a little different from the 'cbo_vix' fields. See https://www.quantopian.com/data/quandl/cboe_vxn.
The 'close' field is simply called '.close' and not '.vxn_close'. The following should work.

class VXN_close(CustomFactor):  
    # Default inputs  
    inputs = [cboe_vxn.close]  
    window_length = 1  
    # Compute momentum  
    def compute(self, today, assets, out, vxn):  
        out[:] = vxn[-1]

As an aside, when using the various Q datasets, I find it easiest to first find the dataset of interest on the 'data' page (https://www.quantopian.com/data) then click on the dataset I wish to use (eg "CBOE VXN Index from Quandl" https://www.quantopian.com/data/quandl/cboe_vxn). For each dataset there is a list of available fields to access. Often there are also examples of code.

Hi Dan,

Thanks for the info.

But a question:
In the page of https://www.quantopian.com/data/quandl/cboe_vxn one can see the fields are listed as follow:
asof_date - the timeframe to which this data applies
timestamp - the simulated date upon which this data point is available to a backtest
open - open price for VXN
close - close price for VXN
high - daily high for VXN
low - daily low for VXN

But in fact, if you use the 'cboe_vxn.open', it doesn't work. You have to change this to 'cboe_vxn.open_'. This makes me confused a little bit. :-)

Cheers

Thomas

Hello

sorry if irrelevant but on my note book the instruction

from quantopian.pipeline.data.quandl import cboe_vix

throws a :
"ImportError: No module named quandl"

has it changed?

Jamie McCorriston

Hi Maxime,

Sorry for the trouble. Partner datasets are not currently working in research. We are working on fixing the problem. I will post back here when they are accessible again.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Jamie McCorriston

The issue has been fixed. You can now access datasets in research again. Sorry for the inconvenience.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Just for clarification, it's not possible to trade Accern data in the contest without paying for the data?

That's correct. Some datasets on the site are totally free (like those from PsychSignal, Quandl and fundamentals from Morningstar), some require subscription for the most recent data (like Accern, EventVestor, Zacks, Alpha Vertex etc.) in order to live trade.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

You've successfully submitted a support ticket.

Our support team will be in touch soon.

Need help? Contact support.

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

About Quantopian

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.