Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
How fast can you get market data

Hi,

I am looking to setup an algorithm, I saw that in the example it was saying to get market data every minute. Is this the fastest that we can get stock quotes?

Thanks,

Craig

16 responses

Yes, you get OHLCV minutely bars, time-stamped on the whole minute. --Grant

Thank for the quick response.

I was hoping for a little higher frequency for my trading algorithm. Is there any other 3rd party python applications / libraries outer there that anyone is using to get faster quotes. I have been using ystockquote and the frequency is nice but it is all 15 minute delayed.

If you trade through a broker, most large ones have API's. You can use the API to feed higher resolution data into your algo... However if your code is running on this site it will still only run every minute.

Thanks Ethan, I am working on that with Suretrader, I was just running into a few roadblocks and complexities so I thought I would reconsider quantopian. Looks like I will just have to work through it with them or a different broker.

I'm curious as to which brokers Ethan A. is referring to. Most large brokers offer FIX, but that's no help to the retail market. There's only a few brokers who offer actual programmable APIs, to my knowledge. Does Ethan have a list as I'd like to expand mine. In fact the only two that I know of who offer true APIs are IB and MBT. CQG and REDI offer APIs of sorts, but they're almost impossible to work with. And CQG only trades futures. Bloomberg has an API but who can afford Bloomberg? Of course, if you can deal with FIX, or FFIX even better then the field opens wide up. BATS has some interesting licensing agreements -- anyone know what the going price is to connect to BATS using FastFIX?

IG markets provides REST API (JSON) and Stream API (Lightstreamer). See http://labs.ig.com/

@Craig,

You might give it a try on Quantopian, since it is completely free and all of the nitty-gritty back-end is managed. Keep in mind that the orders to the broker are sent asynchronously, so while the frequency is minutely, the delay is minimized. Note, however, that Quantopian does not provide the individual timestamps of the OHLC minutely values, so when your algo gets a closing value, it could have occurred at any time in the prior minute. There's no way to filter out data that is older than T seconds.

Grant

Thanks for all the input guys, it has been really helpful. I am moving to IB after seeing how good their api is and the ibpy library. Initially I had started trying to get a fix application working with suretrader but ran into a lot of problems and found the learning curve quite steep.

@Femto T., www.ig.com trades Forex and exotics only. And it's UK based which generally means off limits from US trading. But they state that Nadex might be a venue/proxy for routing trades to them. However, Nadex does not appear to offer an API. Nor does Nadex offer security trading; only FX, binaries, indexes and spreads. So, it's not really an option. Good to know though for any European traders who I might run into.

Market Tech, you may be right, as far as broker API's go I only have experience in IB and a little in ToS. I assumed all others were pretty similar. It is still rare to go to a broker and not be able to find some sort of API. What is the difference with FIX? I don't know much about it but a quick search and it seems to provide an avenue for streaming quotes.. How would this not be useful to retail?

A general question is how do these various venues stay in synch with the actual market, and manage latency? Taking IB as an example, suppose they provide a feed with actual market event timestamps (apparently to within 1 second of NIST, per https://www.finra.org/industry/faq-oats-clock-synchronization-faq). Then, those timestamps have to be compared with a local synchronized clock (either on a home or hosted server) to make sure they are fresh, then orders are generated and sent to IB. Eventually, the orders are filled, with timestamps (again, to within 1 second of NIST), and you get feedback to compare how long the whole round-trip process took, and can gather statistics to see if the delays and timing jitter are reasonable. Seems challenging. There could be several seconds of offset, just due to the synchronization of various clocks.

I'm just wondering, in the context of retail trading if timing is managed well enough for there to be any advantage? Or is it less of a timing issue, and simply advantageous to have a more continuous stream of data from which to base trading decisions (that don't have to be executed in split seconds)?

@Ethan A., My experience with FIX, and I believe this is generally true across the board, is that to connect via FIX you will pay $. I believe IB's fix charge is at least $100/mo. with a $500 setup. And as we know, IB is generally very competitive. Every brokerage that we had to connect to using our own software (using QuickFIX) required certification (passing the brokerage's tests). Therein lies the $500 setup. FIX is called a standard but there are many versions and each brokerage uses a different version with their own syntactical sugar injected making this "standard" a set of suggestions rather than steadfast rules. And FastFIX is different still... No, IB's API is a rarity.

Craig:

suppose they provide a feed with actual market event timestamps

If it only were that easy with IB...

IB does not provide market data timestamps on their real time feed.

A recent conversation on the TWSAPI list:

"currently I am only using computer's time as approximated one. want to know if any function can provide these information. "

"Sync your computer to the network time with the Network Time Protocol (NTP). See http://ntp.org. This will make your clock accurate in the millisecond range. In the callback, call system time and you will have a local timestamp for the quote.

If you are asking about when the quote was actually generated at the exchange, unfortunately IB isn't providing that timestamp in the feed. This prevents the ability to measure latency from the exchange to your system, and does mean there is a chance you'll be using a stale quote. If that's something you care about, you need to use a 3rd party feed. I haven't encountered any other feed that omits exchange timestamps."

And before that an inquiry on tick timestamps (trade timestamps) where we had Stergios provide this (which he said he was going to update -- but it's still topical and useful for Mr. Scott):

http://files.meetup.com/1551526/Interactive_Brokers_API.pdf

So, time on data is always a challenge. It looks like Trust and NTP with NIST is one's only recourse.

Last I heard, IB only sent updates every 250ms, so it's definitely not real-time. I have an account with ActiveTick, they are faster, but I still suspect they are using SIP data behind the scenes, so it could be better. What I would really like is a reliable retail data provider which had direct market center feeds and aggregated their own book feeds. Of course, that probably doesn't exist.

I've also heard you get pretty sweet data included with a Lime account, but that isn't really retail anymore...

Simon.

Hmm...seems pretty fishy to me. If a modern electronic market can't apply and deliver timestamps of market events to end users, then something ain't right. "Trust us. We'll handle the timing. No problem."

@Grant, yeah, it's been a head scratcher for quite some time. If you haven't join the TWSAPI group then you should. The Yahoo Groups UI sucks about as bad as a web UI can -- but if you struggle through it you can search and find resources there that have been discussed (for years).