@Peter,
Have been monitoring on the Slack channel, and you've been making good progress there!
Hope to contribute soon, as I now have a potential use-case.
alan

Hi Peter -

It seems like the end game would be to get zipline-live up and running on a hosted server, with integration with IB, Robinhood, etc. and data feeds (i.e. what Quantopian was doing). Quantopian already knows how to do this, so is there a lot of duplication of effort here? Or does "on premise" mean getting it running on a local server (e.g. in one's basement)? If the latter, this would seem like a step backwards.

Also, I'm wondering how much interest there could be in this? My understanding is that Quantopian only had a few hundred real-money traders. Even if they all came back, it wouldn't amount to much participation in the grand scheme of things.

Perhaps you could elaborate on where you see this going?

I'm sure there is actually certainly a lot of interest & a poll could probably confirm that. I would guess a lot more interest to live trade than to participate in the contest. Ever since they stopped live trading, I've wondered why anyone would even want to pay for any of the premium bundles?

Can you outline a bit more as to how the zipline-live solution will look? Will you need to deploy it on a cloud server & it automatically hooks up to a centralized source of data feeds or the exchange? I.e. if you want to use fundamental data, where would you get it from?

I'd be interested to follow along at least, what is the slack channel called?

FYI, QuantRocket are looking at doing something similar - https://www.quantrocket.com/

Hi Peter,

I am Director of the Quantitative Finance Club at Texas Tech University. The Quantopian platform has been an invaluable learning tool. We have a few programmers who are interested in a functioning live server such as zipline-live. We are familiar with the open source zipline repository and host backtesting locally. My contact information is [email protected].

Hi All,

Someone has already mentioned QuantRocket and while they seem to be moving forward with development, their business model is to charge fees to use platform based on size of account traded. The larger the account the higher the fees. While I know some software companies use this software licensing model, I will never use a platform to with such a licensing model.

Another software that is also zipline based and is working specifically on Cryptos is Enigma's Catalyst. They are using another open source software ccxt which is used for all the various Crypto exchange integrations. They are providing as open-source and while I am looking at using this software they don't yet have pipeline functionality built and it does not connect to any equity brokers like IB.

I want a tool that I can build automated algos for all markets; Crypto, Equities, and Futures. I have looked at the zipline-live repository and it seemed that the momentum of dev has slowed. If we can all work towards an effective open source solution based on zipline, ccxt, and an effective IB connector where we control the intellectual property of our models and can run the workflow from a dedicated server .... This would benefit all!

I'm In! :)

thanks, Alan, our idea are alike! Are you member of the Slack channel? In would love to discuss the roadmap further with interested people.

@Peter

Kudos to you for this effort! I’m not sure I have the coding skills you’re looking for but would can offer assistance with testing and general feedback.

@Dan, Peter
I second Dan‘s offer

My general feeling about Zipline-live is that while it could be better, it does work and given the circumstances that is great.

Explaining the circumstances

• Many of the people writing code for trading cannot share their code. There are a number of reasons why they cannot share it: company
  policy, an asshole boss, compliance issues, etc. This is understandable.
  
• Many of the huge OSS projects were done by employees of a company. Python has been supported by Google for many years, IBM has supported many Apache projects, etc. While Zipline and Quantopian are humongous contributions to the open source community, they were done so with a financial incentive, and they were written by employees of a company benefiting from the use of this software. The takeaway here is: don't feel bad that the project with 4-5 part time contributors looks slow and shitty compared to one run by 40 full time employees.
  

Some ideas I had for getting more contributions to Zipline-live
0. We could put some effort into highlighting contributors, this will incentivize developers who want their work to be seen. (I wanted to translate my book from Python 2 to Python 3 and I was too busy to do it myself so I asked the community to do it with the promise of recognizing the translator and it was finished quickly.)
1. GSOC - applying for a grant from Google to support an intern for the summer to do something big.
2. Asking some of the more forward-thinking hedge hedge funds to contribute. Traditionally hedge funds have been a stiff-ass place to work where it is against the rules to wear headphones or jeans, etc. There are some large and small ones that behave more like a software company than a bank. We could approach these and ask that they contribute, while explaining benefits.

Remember fawce (when they killed brokerage integrations)

Moving forward, we want to help people find other ways to trade their own money algorithmically. One possible option is the zipline-live open source project that is derived from our open source backtester, zipline. This project has the potential to be an alternative for personal trading. We've talked to the project leaders, and we've agreed to support the future development of zipline-live. Interactive Brokers has agreed to provide some assistance as well. We support more than a dozen different open source projects already and zipline-live fits our mission. We are also soliciting additional corporate sponsors to help support the project. If you're interested in using this project for your own personal trading, or in helping to build this project please visit http://www.zipline-live.io/.

I haven't been extensively following the zipline-live project until recently. Did any of these three ever actually happen?

I'm not sure if there's any validity to it, but I've noticed a tendency for Quantopian to be less enthusiastic about sharing lately. To my knowledge, the code for the Optimization API, the risk model, and the new SimpleBeta has not been shared yet. Regarding the Quantopian support for the zipline-live project, I would have expected an announcement that they'd published the details of their integration with IB and Robinhood, but I haven't seen anything (maybe this is a red herring, if the details and code would be irrelevant). Regarding the IB support mentioned by Fawce, has Quantopian provided an IB point of contact? Perhaps he/she could be invited to comment here. Any candidate corporate sponsors could be invited, as well.

Another potential opportunity for Quantopian would be to provide a zipline-live update and facilitate discussion at the upcoming QuantCon (perhaps inviting IB and candidate corporate sponsors to meet face-to-face). This would be a substantive way of showing support.

On a separate note, I'm wondering if someone could take a crack at articulating a business case here, from the perspective of Quantopian, IB, Robinhood, corporate sponsors, etc.? It kinda smells like throwing good money after bad, given Quantopian's effort to make it fly and having it crash with limited engagement.

noticed a tendency for Quantopian to be less enthusiastic about sharing lately

Natural as humans given the many quite-less-than-civil words that are all-too-frequently dished out in their direction perhaps. Even with interesting contributions happening, it's a tough realm here.
Or could be from lessened optimism. One might think of that as a bad thing and yet on the other hand for example the 3% downturn could be a good thing if it brings about adjustments toward 30% profit per year instead of being stuck perhaps around +5%. Hoping some will see my point.
So much of the stock market is a battle against our own human foibles, that's why I love algorithms, even though we still have to struggle with ourselves in creating them. Meanwhile it is true in any endeavor to some degree also, the battle against our own human qualities. The accomplishments here are astounding but those who have made them possible really need to almost sort of hold their breath staving off pride for when the bank account numbers go up, then breathe a sigh of relief. Part of avoiding pride is to really consider that some of the considerable brain-power and insights out here in the little people can be practically manna from heaven. I have been sidelined on something I thought would be beneficial and that's no fun, I still wish it would be considered.
We can all assist toward maximizing the big picture returns by taming our words to some degree I think, perhaps. Remember the animated character in the movie Inside-Out representing anger, with flames coming out of the top of his head. When things reach that point it may be time for a personal email to feedback or direct only to an author/member involved. Personal attacks publicly harm this entire thing. But then there's the historic death-by-a-thousand-cuts phenomenon, persistent digs that can wear one down. I don't have the answers, it just feels to me that there could be some current trusted paths worth revisiting, double-questioning, and maybe some overlooked worth reviving. Personally I like some of GK's suggestions above and elsewhere, but would replace the word crash with suffer. Another contest with no restrictions might be a bit over the top but generally thinking way outside the box might be something to consider. I believe I might even contract many outsiders for mere days at a time each for their objective viewpoint and say let us have it, criticize everything you can in our methodology, don't hold back, just be cordial.

Sign me up

The case for zipline-live is that there are very, very few people who care about it, but for those that do it is very, very important.

Looking from Quantopian perspective I understand that they have almost no incentive to help the project - as it is taking any good algos away from their own fund .

But even with all features, zipline-live will still not be able to mimic historical, real-time and fundamental data available for free at Quantopian. So in order to replicate it, you really have to do it in a serious way - forming a startup probably. Which then hits the problem of a relatively small target audience. I estimate a few hundred people ready to pay ~$50/month for it which is just too small of a market.

With respect to data vendors, my impression is that Quantopian has been able to get favorable terms by showing that they could prevent wholesale data theft. In this fashion, they could offer free retail trading (with the idea that the retail traders would also write hedge-fund-viable algos). The shortest put would be for Quantopian to revisit their decision, to see of there would be any way to offer a development sandbox for zipline-live that would allow users to hook up to their basic data feeds. The problem comes in, however, that to convince the data vendors that they aren't getting ripped off, they'd need to host everything end-to-end, or have some sort of spin-off business entity that would manage things. Everything comes back to the potential size of the market. My sense is it probably needs to be something like 10,000 active users, but I recall that Quantopian only had a few hundred. Frankly, I think Quantopian made the right decision (although the execution was kinda poor, giving only a 1-month notice), given the information I've gleaned. Presumably, they still have an offer on the table of $250M from Point 72 and should do whatever they can to execute on the hedge fund business model.

Looking from Quantopian perspective I understand that they have almost no incentive to help the project - as it is taking any good algos away from their own fund .

Apparently the userbases do not overlap, that's why they killed brokerage integrations. So I'm not really sure how it would take away from Quantopian to support zipline-live's development, because the algos that Quantopian is using and that zipline-live users are using are probably fairly different.

I think this discussion quickly derailed. I'm well past caring about the end of the brokerage integrations or what made sense for who and why. Still not happy about it and never will be, but life goes on.

My question is that there was a stated commitment by Quantopian to support zipline-live in a serious fashion. Does anyone know if they actually followed through in any meaningful sense at all?

From what I saw some members from Q's engineering team listened in on the Zipline-live Slack channel and provided guidance if we got stuck. They were there to answer questions as well. The engineers at Q have always been quick to respond to me, and helpful.

The code for SimpleBeta has been published, go to the Zipline Github repo and search for SimpleBeta.
The code for the optimizer hasn't been published, from what I understand it is not that involved. I actually would like to take a crack at writing my own version, to understand it better.
Some of the other code like the broker integrations with IB may not make sense for them to share.

SimpleBeta source: https://github.com/quantopian/zipline/blob/master/zipline/pipeline/factors/statistical.py Thanks

Regarding the optimizer, I’d think it would be a welcome addition to zipline. Using the existing Q code would make sense, modifying where necessary.

Regarding business model, the goal is individual profit by combining community quant and coding skills. In the context of often asserted low capacity winning algos of interest only to smaller accounts, I am willing to give my coding time in exchange for implementing an algo in my personal account that will compensate me for that time. I expect our community has members who would take the opposite side of this trade, sharing algo knowledge in exchange for an execution platform. Together, we build out the platform for public use and private benefit.

I am open to joining a club of quants and coders with such a mission statement.

Regarding data, it would be nice to be able to import zipline into the Q research platform and run backtests there. Is this supported?

I have read the recent posts. I am now thinking that the tool just might get mature more quickly if IB were convinced that this tool could generate significant commissions in the long run. In reality, they really need a tool that supports algo development and automated trading.

I would think a conversation with the right person at IB and they could be convinced to be a major competitor. After all, they are broker that most would want to trade through given their access to markets and fee structure.

Has their been discussions with IB?

I'm sorry , hoping for a white knight is not a strategy. Chasing a white knight is a fruitless exercise, because of the relatively tiny target market. Instead of trying to disprove these statements, I advocate joining in an effort that benefits each individual member financially for their efforts as outlined in my initial post to this thread.

Thanks for continuing to work on zipline-live. I wish I could be of some use, but when it comes to python and css, I know just enough to get by.

I think it's important to not get discouraged by the idea that only a couple hundred people were actually using Quantopian's live brokerage trading feature at the time it was discontinued. The number of people who were using the platform working towards live trading is easily multiples of that -- people who were still working on trying to develop a solid algorithm or still watching their paper trades, confirming out-of-sample performance before moving ahead with real money. That part of the process takes a lot of time, effort, and includes endless potential setbacks. Though those people weren't actively using the live trade feature, they were on the platform to use it, and shouldn't be discounted.

I routinely check up on my paper trades here on Quantopian to see how they're continuing to hold up out-of-sample, and after the effort that went into developing them it's such a shame they're only making theoretical money. If there were a drop-in replacement, with Robinhood support, that works, I'd happily pay to use it (so long as it doesn't cut too much into my PvR). Though my experience live-trading with Quantopian was that it did not behave like the backtester and thus required lots of debugging/modification of my algo, until it stopped being able to sync altogether, which was the end of that. So even with Quantopian's infrastructure it wasn't smooth sailing (more like drowning). Without Quantopian's centralized infrastructure and data streams, it all seems even more far fetched.

@Peter

I would be happy to help out with the project. Where are u in the timeline of this project? I'm strong in Python and CSS and have used Zipline in many of my algorithms while running them locally. I also have dabbled in Django and could be of some use there as well.

While I was travelling (snowboarding in Japan) I followed this thread and I'm actually quite heartened by the responses and I had quite a few in my inbox as well. Lots of ideas to chase still. The gist of my thinking is that starting a funded startup and put some good development power behind it would be the most fruitful avenue. Zipline and Zipline-live can become a very important part of the financial ecosystem but in my opinion, Quantopian has taken an US-centric institutional route, I would like to take a tiered approach by developing and contributing to zipline and zipline-live and make it such that it's easy to keep an algo's code for yourself but let other people trade with it and they will pay a contribution. The crux in my view is data, multi-asset capabilities (trade on multiple of exchanges (IB, Robinhood, cryptos, etc) at the same time: enable multiple currencies, enable European and Asian exchanges, etc, etc ). Then the hard part: a server farm that is stable where one can move an algo to without disclosing the code and being able to rent out your work. Take zipline, Quandl , C2, Quantconnect and a proper reputation and verified trade system, hustle and mix and try to do this in a proper way.

So lots of work that will need funding.

I have spoken to several people who would like to be involved and would like to provide funds and I'm exploring several avenues, from self-funding (by me and a founding team), VC funding, ICO offering that would power the ecosystem and some other ideas that got shot down (robbing banks and old ladies)

Let me take a week and nail some stuff down and I'll come back. Zipline is too beautiful for it just to be a hedgefund tool. Hedgefund power to the people is a more wholesome framework... and hedge and other funds can join the party as they want.

Thanks, Peter

Peter,

If you can answer the Heilmeier questions, you'll be off to a good start.

I'd be interested in helping, if we can get some clarity on the overall enterprise.

Hi Peter,

As background, I spent a lot of time trying to get crypto data working with zipline.
I eventually found the solution, forget about it and use backtrader!
I have also contributed code to ccxt.
So I'll definitely be watching this project with interest.

Hi Peter:

Some of this code may be of use to you as well:

https://github.com/ScottFreeLLC/AlphaPy

Regards,

MC

Thanks Conal, CCxt integration seems a good way, Enigma/Catalyst managed so zipline-live can, but I would love to discuss the gotcha's with you

Thanks Mark, I'll definitely play with your framework and see where and how we can use it

Hi,

Is this an active effort? Update?

Hi Grant, I'm sure you're aware of https://groups.google.com/forum/#!forum/zipline. The use of a csv directories seem most useful to local builders. I'd say collaborative efforts were abandoned with the loss of quantopian's support but the community is still alive.

I'm wondering more about a more-or-less one-to-one replacement for the Q broker integrations, more than zipline, which is solely a backtester (a piece of the puzzle, but not the whole enchilada...mixed metaphors...oops).

Hi Grant and All,

This is indeed still a very active activity. I'm in the process of raising finance for a zipline based live trading environment and it looks good so far. The team leading it now is Erick Gomez and myself and Tibor. I have published in Trello/SLack a proposed roadmap and I would love to get feedback.

What we are looking at now is create an ecosystem for free trading for algo developers and algo devs can rent out their exhaust to others earning some extra income. Most income of the project will be used to buy/process data, run the infra and improve zipline via bounty/ community-based systems. That would help the algo developers and we would put extra manpower in developing zipline further.

proposed roadmap

and I would love to have feedback on it.

Hi Peter

The roadmap looks great. Where can potential investors get more information ?

Also quantrocket.com has been making a lot of progress, I just had a demo of their product last week and there are 1-2 months away from trading with zipline.

Does one have have a Trello account to view the roadmap? I'm not seeing anything that looks roadmap-ish.

The cards are an item Grant

Tried Quantrocket just for testing, had a lot of issues downloading and maintaining data which seem to be more Docker-related than anything else.

I'm currently using FriarTuck for live-trading (no backtesting, just trading), it required a bit of tweaking to my algos and I've added some local state-saving and database code, but the core FriarTuck package itself has been working just fine with minute data for me and as an added bonus works with Robinhood. I'm currently slowly trying to pick up some conversational Rails and develop a web dashboard for it.

If zipline-live continues to progress I'll happily review it and even donate (though I'm just a working stiff, my resources are limited). I've had a lot of issues downloading historical data with both Zipline-Live and Quantrocket; fortunately they didn't kill backtesting or paper trading on Quantopian so that's still serving me well.

Hi Peter -

What you've posted on Trello seems to be more of a bottom up technical feature/task list. Is there anything that describes the project from more of a top-down, architectural approach (perhaps graphically)? Beyond the purely technical, are you proposing a business? If so, what sort of business? How would users of the software fit in? Etc.

Hi Peter & all -

In my opinion, the end result will flow out of the business model for any implementation of zipline-live. If the idea is to set up a for-profit enterprise and return profits to a set of deep-pocketed investors, I fear it will be a train wreck as we have seen for a certain Boston-based start-up. It will just be subsumed and corrupted by the powers-that-be. The lesson I've learned is that unless there is some fundamental structure in place to enforce a vision, the idea of a platform for the people will not be realized. As the expression goes, "Fool me once, shame on you; fool me twice..." Before I would contribute a lot of time and effort, I'd want to see that the business fundamentals are aligned with my interests and those of the crowd.

Grant I don't think there is a need for such tone. Quantopian has been of tremendous service to the entire quant community, and still provides almost all of its tools not only for free but open source as well. So all credits for that and let's not forget that they don't owe us anything and that they are a business and not an open source project.

What I think you are alluding to and what many of us think should exist is a full-featured open source quant platform. Sort of a WordPress of the quant world. A true way to democratize quant trading. A business then can still live on top of that platform (like wordpress.com on top of wordpress.org to reuse the analogy). That is the course I would like to see zipline-live take and would be happy to support.

@ Vladimir -

I'll try to keep my disillusionment to myself. Thanks for the Wordpress concept. I'll take a look.

Hi Grant,

You know me already for several years. ;) I hear your pain. I am with you on making sure that we have a brokerage part of the process and that's what Zipline-live is intended to be.

btw – Do you use any other solution for your live trading?

This is indeed still a very active activity. I'm in the process of raising finance for a zipline based live trading environment and it looks good so far. The team leading it now is Erick Gomez and myself and Tibor. I have published in Trello/SLack a proposed roadmap and I would love to get feedback.

As Peter mentioned we are working also to be able to raise capital to maintain the infrastructure. I like Vladimir's analogy of WordPress. In order to run a Wiki or Wordpress alike services org, it requires infrastructure and $$$$$. Also, we are cooking away for a reward system to feed the ecosystem. Ping me on Slack my user is "Gomez" if you want to chat further.

Regards,
Erick Gomez

Hi Erick -

Good to hear from you. I'll try to catch up with you on Slack this weekend.

I've just been dinking around on Quantopian; no live trading. My approach has been to start from zero, not putting in any of my own, hard-earned capital. I'm up $35 in the contest, so I have a way to go. It's just a fun thing for me, but I'd be glad to share what insights I might have.

I'm just cutting my teeth on Quantopian and zipline, but I am a python programmer and would LOVE to lend a hand, time permitting. How do I sign up for the Slack channel?

Been thinking about business models since you were talking about creating a hosted Zipline-live service.

I'm under the impression that brokerages can be convinced to sponsor platforms and help offset some of the infrastructure cost. I think quantconnect has some such deal with IB. Which brings up the question -- what would a hosted zipline-live service bring to the table that isn't already covered by QuantConnect? Doesn't QuantConnect do pretty much everything?

As far as the WordPress analogy goes, you're taking about a freemium business model then? That's pretty much what QuantConnect does.

RE: the idea mentioned earlier, I'm not convinced that it's legal for non-certified algo coders to rent out "algo exhaust" to non-accredited investors. I know there is a site that does this, but I don't see how they get away with it. There's got to be a reason why Quantopian didn't go this route. (Retail equals if not outnumbers institutional capital on the stock market.) I love the idea, but obviously investors could be easily get burnt by unscrupulous, unskilled, or unlucky coders. Anybody have any insight into the legalities?

An alternative would be to release a crowd-sourced algo ETF. If it generates enough alpha it could justify fees on the high end of the spectrum, which could be divyed up between algo coders while also paying infrastructure costs as well. Just look at the (undeserved) popularity of AIEQ -- this could be hot. That would be a cool way to both make money and make algo trading accessible to your average retail investor on a large scale.

I'm with Grant. After what happened with Quantopian after outside money got involved, any new service would have an uphill battle to gain trust of the community. I think people would want assurances that the community won't just get thrown uber the bus, and I think promises alone wouldn't be enough.

I had the same thought about potential legal snags with peer-to-peer leasing of "algo exhaust" streams. My hunch is that if it gained any coherence and volume, it would catch the attention of the regulatory Eye of Sauron. Basically, one would set up something like the Q data feeds except that the feeds would come from individuals rather than institutions. I'd think that there is a fine distinction between a data feed and a portfolio allocation feed. The former could be construed as providing a stream of market intelligence from which trading decisions would be derived by the user, whereas the latter would amount to direct investment advice or offering an investment product (i.e. a portfolio weight vector versus time). So it seems that there is a precedent for selling market intelligence data streams without legal woes, but I'd think that if the streams were offered as retail buy/sell signals, it'd be a non-starter.

Even if peer-to-peer data streams are packaged as market intelligence, there is a personal liability issue. If I were to engage in this sort of activity, I'd definitely consult a lawyer first, and consider doing it under some sort of separate business umbrella to protect my personal assets, should I get sued. It would be an expensive, time-consuming pain...

The crowd-sourced algo ETF idea would be basically the same thing Q is trying to pull off for the hedge fund market, except by creating an ETF. If this idea actually has legs, why not just engage Quantopian in developing a plan and doing it on their platform directly? I'd guess that going after a retail market would be even more challenging from a regulatory standpoint than a closed hedge fund, but I could be wrong.

One over-riding issue is that it takes a special kind of masochistic technogeek to do any of this stuff in the first place. There are only so many people who will dive into an API like Q's and have the technical capability and patience to see it through over many years of work. My sense is that the big market for retail trading is basically a kind of entertainment. You have a lot of people dabbling in market speculation, in the same way that some people enjoy various other forms of gambling. For example, just have a look at the Robinhood site--a bunch of images of cell phones. Just click away...no systematic strategy R&D, followed by out-of-sample testing, etc. (2 years later, you might be ready to inject capital). It would be interesting to hear from some industry insiders about how the retail trading market is viewed...it is a big, lucrative market, and my sense is that it exists because it is a legalized form of gambling, with the house taking a huge share of the profits, just like any casino. It is largely an entertainment venue.

On a technical note, I have to think that technologies such as block chain, smart contracts, crypto-currency, homomorphic encryption, Tor, distributed computing (e.g. BOINC), etc. should be considered as part of any infrastructure. There are also physical infrastructure considerations, such as air gaps, data diodes, unidirectional networks, etc. that could be brought to bear, as part of an overall security plan. The technology should support the individual quant in every way possible, I submit. The impediments would seem not to be technical, but inertia in shifting "Wall Street" power to the individual.

As far as trading infrastructure goes...come on...the Internet now supports multiple simultaneous streams of HD video to the home (granted, there is buffering and it's not real-time, and if it fails, nobody loses money...but the bandwidth is tremendous). Handling trade data should be free, in the end (just like phone calls are basically free, since negligible bandwidth is required).

Thanks Grant, Blockchain is indeed a good tech to reliably do some things like bridging trust between untrusted parties and in this case this is something we are looking into . Untrusted parties could be devs and data firms, devs and other devs, etc

Hey guys, i just saw your post. if you still need help, i would be up to support the project. I know python, django, css, html.
If there is something i can contribute let me know.

Hi Marcel, We welcome any help. please ping me on our slack channel then we can discuss things

Hi guys,

I don't want to rain on Peter's parade. I think it's great to support this undertaking. I will probably join the zipline live community soon but I was in the pressing need of getting something up and running live fast, as my QP projects were live trading for about 1 year prior to termination. Six months ago it did not seem as if the project would move very fast. I do want to point out that after trying several different directions I finally settled on QuantConnect (QC). I've had some difficulties in porting the code from QP over to QuantConnect - but with patience and perseverence I have succeeded. Got my third program trading live now. Connection to IB works pretty well. Have not yet lost any connections for more than a few seconds, and then these quickly reconnected. Supposedly works with 5 other brokers but I have only tested IB . I like the fact that QC supports not only stocks but also futures and options, and offer sample programs in each.

They make their money from hosting the app, - $20 per algorithm per month (no multiple algos at the same time or extra fees). RIA's can do multihosting for $200/month. I have found their support to be quite good. Be prepared for a wait of about a day on your qustions though, Their heart is definitely in the right place. but then they're a much smaller team that Quantopian. They do make available their source code, which you can host on your own with your own servers or one of the cloud servers.For someone versed in the tribulations of version changes, library compatibilities, etc,, it might be worth hosting your own. I calculate your could run a couple 20-30 algos on a cloud server for around $200 a month with Google, depending on their database crunching requirements. In order to save myself from vendor lock in, I probably will take that complicated path. But meanwhile, I'm able to trade live now.

If someone is not a skilled python programmer , but wants to get up and running quickly, I am available for a low fee to port the app over to quantconnect. PM me on the fees. I'm hardly a python genius - (I speak much better English...) but I'm a persistent sob, and so far have never failed at achieving what I needed to in Python.

Good luck to us all.

Interesting that QuantConnect can offer hosting at $20 per algorithm per month. Quantopian said this wouldn't even come close:

Some people ventured estimates of the size of the personal trading business. We only have a few hundred people trading real money. If we started charging $20/month (suggested earlier in the thread), and every single person agreed to pay, it would only be a fraction of the total cost. I just shared a longer version of this answer here.

So why would QuantConnect be able to charge $20 per algorithm per month, but it was not viable on Quantopian? Is it explained by the fixed costs and the limited participation rate on Quantopian? Are we sure that the only way that QuantConnect makes a profit is by charging for hosting?

The reason I'm bringing this up is that retail trading costs seem to be hard to pin down, and it germane to the zipline-live effort. For example, for an individual to do algorithmic trading on a set of N U.S. stocks every M days with turnover X, with $Y of capital, there ought to be a going market rate (direct and indirect, e.g. slippage, etc.). It should be like looking up the cost of electricity (a weak analogy, I know, but you get my point...by the way, a Google search yields 12 cents per kilowatt-hour, which in the end, should be relevant to algo trading, since cloud computing is a commodity, and it is really just the cost of electricity to keep it all running). For institutional trading of gazillions of dollars, I trust that the powers-that-be have this all worked out, but since there isn't a coherent market on the retail side of algo trading, it isn't a commodity yet as it should be (i.e. retail traders are getting ripped off).

They do make available their source code, which you can host on your own with your own servers or one of the cloud servers.For someone versed in the tribulations of version changes, library compatibilities, etc,, it might be worth hosting your own.

Another important point. The paradigm should be more like firmware not software (think router firmware versus desktop pc OS, with automatic updates), perhaps with a mirrored test instance to confirm changes, leaving it up to the individual quant whether to update or not, and when (perhaps still providing an "auto update" option). Running an algo with real money on a platform that is in perpetual beta, with no control over the changes--not where zipline-live should land, should it be implemented as the backbone to a hosted trading system.

Hi @Serge and @Grant,

$20 a month per algo seems quite expensive for only the computing hours. The cost could be much lower. I have found that the main cost would be the professional grade data. Without proper data any backtest is useless but if your systems use data to make decisions you need to know it's reliable. And while I traded with QC and have found it cumbersome, I think the path we are taking with zipline-live is one where we all get get better. Quantopian has Free live trading which was brave, but if a product is free, most likely you are the product. Anyway, I'm working hard with Erick to make ZIpline-live a complementary environment to Quantopian and live trading will be the core. I'll be at Quantcon and I'm happy to discuss your needs and your contributions.

Cheers, Peter

I have found that the main cost would be the professional grade data.

Well, let's assume that Q would allow upload and comparison of their data sets to ones used by zipline-live, per their terms of use, then perhaps the cost could be kept in check. I'd inquired whether this might be an issue with regard to their license agreements, but did not get an answer (I'm not sure how they would preclude it, if both the uploaded and proprietary data are both available on the Q research platform). As a starting point, getting the Q and zipline-live US stock OHLCV daily bar historical data in decent agreement would be a good idea. Perhaps there is a cheapo/free data set that could be uploaded, and scrubbed using the professional grade Q data? Seems like a great use case for the new API under development.

One curiosity is why "professional grade data" would be expensive in the first place? What makes it expensive? And I'd think that brokers such as IB would want to distribute it so that traders make good trades with them. More good trades means more profits means more trading and more traders. What am I missing? Also, from a regulatory standpoint, if retail traders don't have access to good data, they are more likely to get ripped off (but then maybe the regulatory bodies are more about protecting the incumbent institutional interests).

@Grant I'm not worried about the "professional grade date" There several providers that can offer the Data Re-Distribution.

Another important point. The paradigm should be more like firmware not software (think router firmware versus desktop pc OS, with automatic updates), perhaps with a mirrored test instance to confirm changes, leaving it up to the individual quant whether to update or not, and when (perhaps still providing an "auto update" option). Running an algo with real money on a platform that is in perpetual beta, with no control over the changes--not where zipline-live should land, should it be implemented as the backbone to a hosted trading system.

Are you suggesting to invest more on-premises vs cloud? What we have noticed is that it requires several months for a novice none IT skilled to ramp-up on the on-premise solution. My inclination is going to the cloud instead of an on-premise solution. Imagine all those # o hours of time setting up your OS, your Python, trying to hook all of the plumbing and paying for the fees of the OHLCV quotes data x [number of quants]. When the quant could just focus on their algos and dont worry on the electricity/commoddity (as you pointed out )

On-premise doesn’t really make sense for a variety of reasons. Hosted algos would be the way to go. The problem is with code changes that can’t be managed by the user. In the end it is like 24/7/365 infrastructure requiring a firmware paradigm where if possible users have plenty of time to evaluate and deploy version updates (critical security updates that affect all users may be an exception). With Q the API is kinda in perpetual beta, which is scary when somebody else’s money is on the line (not so much if we are talking Netflix...just read a book instead). A hosted system doesn’t mean API software is not controlled by users.

@Grant My understanding is that premium datasets are expensive because the people who offer or own the data know they can get away with charging an arm and a leg. It's not that it's difficult/expensive to source or distribute the data, rather the providers know how valuable it is to their customers. Real-time exchange data for example is expensive because the exchanges see it as a way to increase their income, and they have a monopoly on their respective exchange's data, so they can.

From reading the QuantConnect forum, I don't know if I was reading too much into it, but it seemed like QC have a deal with IB where IB helps subsidise some costs in exchange for integrating with IB.

@Peter If you think about it, $20/month covers not just spinning up the AWS for the algo (and offsetting server costs for all the free users) but also development and maintenance of the platform. Depending on how your value your time, if it saves you roughly a half hour a month of having to maintain the servers and framework/platform codebase, then it's not expensive. In addition, it's hopefully just a drop in the bucket
compared to the profits/alpha your algo is churning out.

I would also like to mention another issue in the hosted vs open source topic -- trust. This is huge. If an algorithm gets stolen by an unscrupulous SasS host who in turn exploits it and maxes out its capacity, it could leave the algorithm's author empty-handed for all their hard work and research that went into developing it. We've trusted Quantopian, but why would we trust another service?

One use case for zipline-live I've been thinking about would be to run a contest/fund algo completely offline (i.e. hosted on another platform), and just upload the buy/sell signals to Q, once the new My Data API is developed and released to production. We have a set of requirements and in terms of data, a starting point would be daily OHLCV bars and fundamentals, limited to the QTradableStocksUS (QTU). So, for a hosted zipline-live instance, to run an algo for this use case, what should it cost per month, back-of-the-envelope?

It is worth noting that if one benchmarks at ~ 0.1% institutional expense ratios, a $20/month cost implies ~$250K minimum in capital just to overcome the hosting cost. So, if a use case is to do long-term investing at low expenses, it is important to benchmark against expense ratios one would obtain by just buying a basket of ETFs/mutual funds and re-balancing manually on a monthly/quarterly basis.

@Viridian. I'm an RIA with a couple of big clients and a couple of "little fish". For the former, there's no problem with the $20/month, On 500k that may be 5 basis points or so a year, and 25 bpts for 100k accounts. This is less that what most mutual funds charge - and comparable with Vanguard funds. But for smaller accounts, it's much more significant, with a greater impact on performance. My "conservative" algo shoots for pre-fee returns of 13% CAGR and my "aggressive" algo targets about a 50% return ("I know , I know, that's outrageous, but it is what it is"), so those numbers are tolerable. But I would like to be able to assemble a couple 100 small users into a single group, trade them as a group, for a reasonable monthly fee. I can do that with IB's trade baskets, but Quantconnect would charge $200 / month for that possibility. If I host my own using their Lean version, I can do that.

I also agree with you that the biggest concern for anybody doing this professionally - or personally with real intent to profit - is to not be suddenly stranded as we were with QP, and having to start from scratch. Of course I had all my source code backed up, but porting to another venue took months! "Never again", quoth the raven.

The latter reason is why I think the most successful long -term open-source or collaberative solutions should do the following. This would also address Grant's version controls, yet speak to the issue of software / hardware maintenance complications.

• Develop for a given cloud platform that can be reasonable expected to survive the next decade (AWS, AZURE, Google cloud)
  
• Develop one set of instructions for installation (configure hardware choices, load needed libraries, link to TWS, link to shared datasources)
  

• Develop another set of libraries for the zipline code itself. These could be versioned, perhaps similarly to what IB does with TWS . TWS latest and TWS beta, representing different degrees of experimentation. All of these would be run on the cloud.

  The idea here is that the new users of ZIPLINE could experiment and startout running on the community AWS zipline platform. This keeps the learning curve as shallow as possible. But at any time those individual users of ZIPLINE could each set up their own accounts with AWS, and easily be up and running. This would address all the issues of data security, privacy and resource monopolization.

Rereading my post, it seems I've suggested the only custodian/broker to work with should be Interactive Brokers. This is not my intent. I love IB, but I don't want to depend on any one outfit. Also, I think a link to the crypto-currencies is essential, though most of those will soon die a quick and very painful death, taking many investors with them. But one thing at a time. Starting off with IB is fine.

Regarding the security issue, that's a biggie, and one has to decide what's necessary and the cost and inconvenience that can be tolerated. It would be interesting to hear how hedge funds manage this (including Q's 1337 Street Fund, which I guess uses cloud computing for algos?). Would they actually trust cloud platforms (e.g. AWS, AZURE, Google cloud) with their "secret sauce" code, from a security standpoint? If the code actually provides some "edge" it would be tremendously valuable; I'd think they'd want to keep it under tight wraps from a security standpoint. Putting it out on a distributed cloud platform and not controlling the hardware would seem scary, but I'm no expert.

Hello!

I follow this thread with big interest. Personally, I think there is a need for a marketplace of quantitative strategies which are written in python and work with different assets, so I would like to know what is the situation now.

Maybe some comments:

I have experience with QuantConnect and I can say that QuantConnect is not good for cross-sectional algorithms (algorithms with bigger stock universe). It is recommended to use max 100 stocks otherwise it is very slow. it doesn’t support pipelines and so on. Its python compiler is not very friendly, it is difficult to find bugs. It was improved recently so it is easier now but python documentation is still pure and there are still bugs in the environment.

In general, cross-sectional algorithms are more suitable for institutional investors but could be good for smaller investors as well. I see from some of my clients that a need for pipelines is quite high and I think support for pipelines (and fundamental data) in zipline-live should be the first priority. In such marketplace institutional investor should have some ways to connect with quants and contribute for that.

Collective2 is an example of marketplace which works so all questions regarding legalizations are I believe answered there. Why not use then Collective2 to publish your strategy, I don’t like their fee policy and there is delay when you copy a strategy. So, it is better to use original strategy than just to copy results and maybe use more reasonable fee policy for smaller investors.

I agree that a trust to security of algorithms sources is very important and will be difficult to reach without institutionalisation. In this case most probably some kind of fee policy should be introduced. For popularity I suggest that only profitable algorithms are billed and they are build based on profit (maybe only charged to subscribers). So, in development phase you don’t have any cost, this will give advantage too some other initiative like QuantRocket.

It is not so obvious to me that algos need to be hosted, or if they are, in what fashion. They could be run "on-premise" (either from a home-based server or a private cloud instance) and still take in data and output trading signals. For example, Quantopian doesn't host their data vendors, yet presumably some of them are making a return on their signal streams. If they can be stand-alone, from a technical standpoint, I don't see why individual algos putting out signal streams couldn't do the same (in fact, they could just plug into Q's existing infrastructure for data vendors, and Q could revive their broker integration).

Erick Gomez and I are at QuantCon to extend the network of contributors and talk to you guys about what next for zipline-live. So make sure to say hi, we will be at a table near the lunch place. Come and say Hi!

Peter