Hi Lou!

Thanks for the kind words, I'm glad you like what we've done so far.

We're definitely looking to support serious, investing traders. I hope that you think Quantopian fits your needs. We don't have a paid support plan yet, but as we move closer to supporting live trading, I'm sure we'll revisit that. We also hope to also expand the set of people who can do this. In that sense, yes, we're trying to find hobbyists, and we want to educate them and give them the tools to do new and great things. Have you had a chance to read the Quantopian Manifesto? It's a high-level, provocative view of what we're trying to do.

We're trying to do everything that you're talking about, and a bit more: a strong platform suitable for a skilled, serious trader; a place for hobbyists to learn; a community; generate a profit for our shareholders; contribute to the open-source community; and change the way the finance is done.

As for documentation, you can find it in our footer or in the IDE, at this link: https://www.quantopian.com/help

Thanks,

Dan

Dan,
Thx for the reply. I'm wondering if Quantopian will evolve into something in between quantstrat (in R) and tradelink (C#) which is kinda like a RedHat built around an open-source trading platform. It would be great if something like that came to Python.

~Lou

I think that Quantopian would benefit from publishing a timeline. What do you hope/plan to do and when?

Dan,
Just as an example, here are some of the features I'd be interested in seeing in a timeline:
--Order types: I thought I only saw market so far. Not much you can do with that. What about stop, limit, stop limit, etc.
--Data feeds: Probably the best retail/semi-pro data feed < $100/mo. is IQFeed. What about it or others?
--Broker connectivity: it sounds like this is TBD but it would be nice to know what quarter you expect to see it; e.g., 1Q12.
--Multiple time series: For instance, my strategies use both 1 day and 1 minute bars in testing and 1 day and real-time data in live trading
--References to DLLs, using other languages: For instance, I use R (I'd like to call via RPy2) and sometimes I use DLLs written in C#

That was just what came to mind immediately. I don't care about canned "indicators" or even the backtest/reporting stuff. For trade reporting and analysis it may be easier to integrate with the R packages like PerformanceAnalytics and blotter that already do this stuff well.

I forgot to add instrument types.
--Instrument types: equities, futures, forex, options
(I happen to trade forex and some futures)

Out of curiosity, if you do offer an actual trading platform, how will you handle things like slippage and partial order fills? will you hire a staff that I can scream at, at 3am because my trades aren't being executed correctly? :)

Vishal, we could go on and on, couldn't we? What about OCO? Or, auto reconnect when your feed drops, etc....

where do you live Lou, I'd like to have a chat with you!

Wow! I love this thread!

Who will be at PyData at the end of October? Would be good to see you in person and review our product timeline.

@Grant, we agree that an open product timeline (subject to change! :)) is right for us. We just need to figure out the best way to harness all this member creativity. At the moment we are very focused on shipping our universe selection and batch transform code. The next big goal is live trading via Interactive Brokers. IB is the most commonly used broker among our current members.

@Lou, I think of Quantopian as a quant finance version of Google App Engine or Heroku. We tie together all the pieces you need, from third-party libraries, to historical data, to broker integration. We want to attract seasoned pros like yourself, as well as greenhorn talent that may otherwise have never had access to good systems. We also want to help people "level up", but letting them focus on the creative aspects, and leaving the platform grind to us. A huge part of our vision for helping members improve their game is connecting them in a community. I think your feedback here on the platform is a fantastic example of why openness will help advance the field. Thank you!

@Vishal, we will enable you to run your algorithms live. However, we will be a technology platform rather than a broker. We will give you push button deployment for your algorithms, but we will be integrating with your broker's API to execute the trades. Of course, you will still be able to scream at us if things don't work :).

fawce,
I think you'd do well to try to integrate with the existing quant finance and trading packages in R. In fact, I think if you just referenced the quant fin/trading packages in R and re-wrote R's quantstrat package in Python you'd make tremendous progress. Or, at the least, it would be a commendable prototype for you to try to beat while growing your own.

I'll be at PyData. I live nearby. Backing up a bit, I don't even know Python yet. I code in C# mostly. But, I'm taking what looks to be a pretty solid Python intro class online at edX/MIT. My hope is that Python plays well with R, via packages such as RPy2, since the "brains" of what I do are in R. R is very useful b/c of the huge function library. But, I don't quite regard R as a programming language suitable for trading; hence, my interest in Python and Quantopian.

vishal, I'm in NYC. skype handle is loubellenyc The next few days are really bad for a call but we could chat/IM briefly.

@Lou - you should definitely check out Pandas, if you haven't already. I think you will find many parallels to the R DataFrame, but you'll be in the python environment. Wes McKinney is doing a few sessions on it at PyData.

I've thought a lot about integrating other languages, especially R. There's seemingly infinite libraries available for R. But, I think the python ecosystem is gaining very rapidly, and python has much greater range as a general purpose language. Do you typically research using R, and then implement your strategies in C#? Or do you run a hybrid in live trading situations?

@fawce,
I've heard of pandas. I didn't think it was intended for systematic trading in particular. There are around a 1/2 dozen R pkgs that are written to help with the prototyping of quant trading strategies.

Currently, I use certain algos written in R and they are called into C# for the strategy implementation. There is a lot of professional quant modeling and strategy prototyping done in R. But, then, at some point, it gets handed to developers who rewrite it in C++, et al. I'm looking to skip that re-work part of the cycle. Ideally, I'd like the algo logic and the strategy to be in Python but also I'd like to be able to integrate R functions liberally in my code.

@Lou, that is exactly our vision: totally eliminate the re-writes, and shrink your cycle time (research, dev, trade) from months to hours.

@fawce,
Well, they're are a lot of systematic traders waiting for such a product so you're in the right space. Honestly, I don't understand why there isn't a full-fledged Python-based platform already out there.

Only one more request :) Does it have to be a browser-based IDE? Why not Eclipse, MSVS, etc?

@Lou - We believe we can create a very powerful IDE in the browser. Take, for example, the rapid feedback from backtesting during coding. We added that functionality because members told us there were code issues that they could only test with real data. Based on the feedback, we knew we wanted to provide a very tight code/backtest/code loop. Creating that loop inside another IDE seems nigh impossible. But, because web apps allow such rapid remixing, we were able to respond to those requests with the new integrated UI. I'm skeptical that a general purpose IDE could match

I also think it is really, really wonderful to offer a full blown financial analysis IDE without a single download or install.

Having said all that, the code validation (compile and unit tests), and the backtester are accessible via REST. That means any IDE could have a plugin that pushes code to the backtester and receives the results. You can imagine embedding a build step to validate your code and even run the backtest. My hope is that the Quantopian community will use REST to integrate with a variety of IDEs, as well as other systems. I'd be happy to provide REST documentation and support to help people make those integrations, but we won't go so far as to create and maintain IDE plugins.

A related request is github integration. Several folks have asked us to pull their algorithm (and supporting libs) from github. This option intrigues me. I don't think it can completely replace a browser IDE, but it seems like a powerful addition.

I look forward to your thoughts on this topic!

thanks,
fawce

fawce,
It's pretty early for you guys. If you think a browser IDE and/or github is best for your market niche then have at it. I'm not familiar with this approach and I'd prefer to have full control on my equipment.

All,

If someone hasn't already touched on it, my recommendation is to consider MATLAB as an example of a flexible environment for research studies. Does anything comparable exist in Python? My sense is that if someone has enough cash to do quantitative trading (as I recall, Ernie Chan recommends at least $100K in his book), he can justify the expense of MATLAB (or an equivalent product). If Quantopian can supply the same level of functionality and access to data for backtesting in a browser for free, that'd be great...but it seems like you have a lot of work ahead of you to achieve a MATLAB-like environment (and supports GPU/parallel processing, etc.).

@fawce,
Regardless of the direction you go in, given that this is an open-source systematic trading product you should really check out the R-sig-finance mailing list or check out their IRC chat room, #R-Finance, on Freenode (just don't tell them you're a Python guy). I've seen a few of these open source projects come and go. R is the only one that's made any real progress with systematic strategy development and that's due to the dedication of about a 1/2 dozen+ quant professionals (mostly working at different firms) who wanted to escape proprietary software like Matlab and are actively engaged in building out the R packages. You'll see a community working everyday to build or improve R packages for strategy development. For instance, an improved package to trade through IB is near launch.

As I see it, the opportunity for another open source strategy development offering is a shorter development to production horizon. It seems like Python would be better suited for this than R. But, the R group is really far along in their build out.. Perhaps, there is some room for adaptation or even collaboration. I'd really like to see you guys succeed.

@Grant,
MATLAB isn't such a good example anymore. The people using R for strategy development are mostly Matlab refugees. Among quants, Ernie Chan is better known for selling consulting, seminars, and books to people who "want-in" on using the computer to make money in the markets.

@Lou thanks for the advice and the pointer. An active community makes all the difference, and R has been very successful in fostering community around the language in many domains. We are trying to do the same around python in finance, so I'm sure there's a lot we can learn from R-sig-finance.

I do think that python makes the transition from research to live trading much simpler. Building an event driven system in python is quite easy, whereas MATLAB, R, and any other "batch" language makes event programming nearly impossible.

@fawce,
I'm sure this is sthg your team has put plenty of thought into but I'm not sure what it takes to make open source work. I've seen a few failures w.r.t open source systematic trading. FWIW, level of funding didn't seem to matter. It almost seems as if you have to recruit some quants at established firms so they can dedicate development time pro bono. Isn't that essentially what happened w/pandas coming out of AQR?

@Lou, you're right about Pandas. Wes McKinney is a force of nature. At some point, he just decided that finance needed python, and that python needed a real dataframe library. He then wrote some of the best statistical software ever - high performance, great api, very well documented. He literally wrote the book on python for data, and ran around the country teaching people how to do data work in python. Wes is a hero.

My opinion is that opensource works best when your users are developers, and when the code is a library, rather than a full blown application (firefox is a noteable exception). The opensource trading systems I've seen are not just applications, they are multipart systems: hard to install, hard to run, very hard to develop against. We are opensourcing Zipline, an event-driven simulation library. It installs via pip, and solves the specific problem of running historical simulations in python. We are doing what we can to keep it focused, easy to install, and easy to import into your projects.

Despite opensourcing the most significant component in our system, I don't see us as being in the opensource software business. We are in the algorithmic investment business, and we use opensource software as part of our strategy because we think it is the right way to make the best backtester.

For Quantopian to be a successful service, we need to make algorithmic investing dead simple. We combine our backtester with other libraries like pandas, operate the physical infrastructure to run your algorithms, maintain connectivity with brokerages, and integrate high quality data into a unified environment.

We do all this so you don't have to. We do it so you can put all your energy into the most valuable part: your algorithms.

@fawce,
It's not clear to me what business Quantopian is in. You say you are in the "algorithmic investment business" and you are a user of OSS as part of your business. So, does that mean that Quantopian or an affiliate is trading either on it's own behalf or on the behalf of others? What other ways does Quantopian plan to make money?

@Lou, We don't trade or invest for ourselves. We are here to help you do that. Our business is providing you with an algorithmic investment platform.

We see many potential revenue models, but they all hinge on building a great community. So, for now we are totally focused on building the platform and delighting the community. As I consider our possible models, I want to keep our backtesting service free. That way anyone can at least learn about algorithmic investing, and try their ideas with real data.

Here is a list of potential revenue models, in no particular order (and I would love to hear other ideas):

• becoming a broker and charging commissions
  
• becoming a broker dealer and taking a commission share
  
• charging a monthly fee for each algorithm that a member uses in live trading
  
• capital introduction: we help you develop and present your track record, then market you and your algorithms to accredited investors
  
• fund formation and back office operations: we establish all corporate infrastructure for a fund for you to operate
  
• trade execution services
  

Another idea would be to have certain gold features (some of which requested in this post) which require a subscription fee. It would still allow a broad community but also give power users a way to keep operations running and play around with advanced stuff.

GLD?

@Vishal: Haha, while I was talking about premium features I do agree that GLD features would be a better name :)

@fawce,
I don't think any of those ideas will work except maybe charging by the algorithm. Your best bet may be some combo of what @Thomas has mentioned and maybe getting paid for order flow by restricting broker access like some of the retail platforms do.

A variation on Thomas' idea might be best. If you give away the product but charge for consulting fees and/or superior features that might be the best way to preserve the integrity of the open source idea. For instance, you could charge for coding, maintaining broker connections, etc., offshore this work and capture a markup w/o having a ton of working capital tied up. One firm has already tried this and failed (that's what I hear anyway) chasing the institutional market (marketcetera) but if you guys stay small enough and don't burn through your cash too quickly you could pull it off.

Our mission is to make finance more open. Idealistic, but I want to do more than make a living at this.

Enterprise software is due for a revolution. Selling a product to an entire organization at once is remarkably difficult and risky, and layering in consulting is even harder. In my opinion, the future for the enterprise is professionals making purchase decisions as individuals. Time will tell if this is a good prediction, and if my extrapolation to the various models above will work.

I'm curious what other people think. Can we make money this way? Other revenue ideas?

Well, fawce, I want you to make a living first so you can continue building out the product :) By consulting of course, I primarily meant putting a markup on coding services. For instance, I can get EU-based CS PhD's for $10/hr. You could put a nice markup on that.

Or those PhDs can join quantopian and take that arbitrage for themselves. If they have the talent, they should get the money for their ideas, not me.

Quantopian might be able to make money by enabling individuals and small organizations to carry out computationally intensive research that they otherwise would not be capable to performing efficiently on a single desktop workstation. Basically, the idea is to rent out "supercomputer" time with the additional benefits of access to the backtesting data set and the software tools for analyzing it. As a potential hobbyist user, I might be willing to pay a modest monthly subscription fee to be able to run more extensive algorithms, beyond your free service. For the lower cost, I'd have to wait for the results--you'd charge professionals more for high-priority time on the system.

@Grant,

Are you suggesting members pay per simulation hour?

I like the idea because it allows us to maintain a free offering. I think that is critical because we need a path for people to try the system without any overhead - no hassle, no cost. At the other end of the spectrum, it is possible to burn through huge amounts of computational resources with backtesting and a person could be exploring and testing for months before putting even a single algorithm into production trading.

Fawce,

At some level, the service could be metered and sophisticated users might prefer this pricing scheme. A three-tiered approach would be:

1. Free. Access to all of the data and analysis tools, but with limited scalability.
   
2. Subscription (monthly/yearly charge). Full scalability, but the user may need to wait (hours/days) for his results.
   
3. Metered (monthly/yearly plus hourly charges). Algorithm is run immediately and efficiently with the results reported in the shortest possible time.

By scalability, I'm talking about some form of supercomputer/cluster/cloud/distributed/GPU computing that would allow for extensive research and screening, with parallel computing capability. This would be consistent with your "power to the people" philosophy, since most folks can't lay out the cash for a personal high performance computer (although my understanding is that GPUs are fairly reasonable), but I figure that institutions are using big honkin' computers.

Grant/fawce,
Are you suggesting that traders upload their code to a cloud controlled by a party that has a potential conflict? I think that might be a difficult proposition to sell. I wouldn't do it.

On the other hand, if you're just selling/leasing a tool that makes access to an independent cloud easier then that would make a lot of sense. I don't see how that's all that different than supplying strategy coding labor and marking it up. Why would you care whether you're supplying a labor/service, software, or equipment input? There's no reason to be high-minded about it.

I'll throw a seemingly silly idea in the hat: Why not subsidize the development of certain free packages or utilities by 'name-brand' quants/firms? There's a bit of an 'Air Jordan' element to open source. "If so-and-so is using it maybe we should too?" Sometimes buzz helps. In other words, pandas doesn't have to be a one-off.

Lou,

All of us at Quantopian have worked in businesses selling technology and also in businesses selling services. Selling services is not the business we want to be in. Yes, it's a personal preference, but it's also much more than that.

We believe that giving people the tools they need to turn their own ideas into reality will empower them far more than asking them to pay us money to do it for them. We believe in giving our users direct access to the technology and data they need to be successful. We believe that fast iteration will enable our users to produce better algorithms more efficiently. We are high-minded about it. We are trying to change the world of finance.

I'm not sure what "potential conflict" you are referring to. Our users own their own intellectual property. We take the privacy and security of our users' data very seriously, and it would be inconceivable for us to steal I.P. from our users. We are not competing with our users; we are enabling them to accomplish their goals.

Advancing our goals by subsidizing third parties to develop open-source functionality we need is certainly an option that we will consider if a situation comes along where that's the right thing to do.

Regards,

jik

@Lou - I found a pretty awesome demo of R from within the ipython notebook: http://nbviewer.ipython.org/urls/raw.github.com/arokem/ipython/1fc3fdb57aa1b3278318eaca9f72f78574fd3e18/docs/examples/notebooks/rmagic_extension.ipynb

Lou raises a good point regarding security. Your help page states "Our databases live in Amazon's cloud infrastructure." You further state that your plan (not yet implemented) is to encrypt the algorithms before storing them. What about running them? Seems like they'd need to be unencrypted at some point? What ensures that only the user can decrypt his algorithms and backtest results?

Is there any precedent for the security scheme you have in mind?

Hi Grant - yes, we need to unencrypt them to run them.

We've known from the beginning that trust and security are going to be important questions for our business. We are working with our members' intellectual property, and when we get live trading going, with their savings. It's very serious stuff. At some point, it becomes a matter of trust. We're going to take a series of public and verifiable steps - open source the backtester, show you our resumes, etc. I think even things like this conversation, where we have candid discussions of our business plan will matter. Our goal with those steps is to give you a reasonable basis to trust us, because in the end we can only provide this service if our members trust us. For some people, that will be unacceptable, and they will build their own trading systems on their own hardware and their own execution methods. We hope to win the trust of most such that they can take advantage of what we're offering: a powerful and well-maintained backtester and trading platform built with a lot of very smart security measures.

We need to write a blog post on this topic and go into depth on it. Maybe that's next week's post.

Dan

@fawce, thx for the link.

@quantopian - I can't see anyone w/a good strategy giving up control over their code even for an instant. It's not a matter of trust, integrity or whatever. Code and secrecy are all a systematic trader has. Sure, I'd run trivial simulations that way. But, I wouldn't run anything truly proprietary on that basis.

Thanks Dan,

I think you need to be open and specific regarding your "very smart security measures." I've gathered that you plan to use the "Amazon cloud infrastructure" to store encrypted data (backtest database, and user algorithms and results). Sounds reasonable, so long as the encryption protocols and access restrictions are strong. Seemingly, this could be equivalent to a trader working off of his personal pc, with encrypted online storage/back-up.

It seems like you may still have a problem with executing the algorithms, since they need to be decrypted. If you are planning to use third-party hosting to run the algorithms, won't you lose control of the unencrypted user data? Or do you plan to host the application, with physical control over the hardware?

Thanks Grant. I agree. We will be more specific. We'll put it into a blog post and FAQ. It's not written yet, tho ;)

Dan

@Dan, Grant,
I was probably being a bit too quick w/my opinion regarding control over code. If there's some way to assure strategy developers that there's no possibility that someone could view their code, such as dealing directly w/ACS, then of course, it totally makes sense.

Hello Lou/others,

What is ACS?

--Grant

Grant, sry, I meant AWS - Amazon Web Services

@fawce
Speaking of Wes, other than open source, how is what you guys are doing different than rapidquant? Also w
ho from your team will be at the Nyc conference?

@Lou,
We are focused on giving the individual quant everything they need to research and trade: tools, data, and community. That's unique in the industry.
I'll be at PyData along with @Thomas, who is presenting, and some of our core developers.

pandas, scipy, numpy, rpy2, ta-lib, are must have libraries.

Hi Pablo - we've got the first three, and ta-lib is actively being looked at. I'll look at rpy2.

Hello Pablo,

In case you are interested in other modules, there is a list of supported ones on the help page (under the heading "Module Import"). As of 2/16/2013, it is:

bisect  
cmath  
collections  
datetime  
functools  
heapq  
itertools  
math  
numpy  
pandas  
pytz  
Queue  
random  
re  
scipy  
statsmodels  
sklearn  
time  
zipline

Grant

Awesome! Thanks!