I'm also interested in this.

Is it possible to make calls to APIs on other servers in order to pass the algorithm other data sources?

Your code needs to reside on the Quantopian server. They won't steal it, since their business would go up in flames if they started doing this. See the help page for some additional security measures you can apply to your Quantopian account.

I suppose you could consider using fetcher to feed in signals overnight...but it sorta defeats the purpose of the whole platform.

--Grant

Your code needs to reside on the Quantopian server.

You didn't answer why. If there is no intent on stealing it I can imagine multiple scenarios where the code doesn't need to be stored in their servers, they would win the trust of their users and they would also protect themselves of being robbed by external hackers.

Maybe this will help:
https://www.quantopian.com/faq

If your heart is set on using your own development environment, take a look at Zipline. Zipline is our backtester, and we have open sourced the code. If you want, you can connect Zipline to a data source and use your own development environment.

I think that would keep you self-contained except for any data source.

Thanks garyha ! I think that's exactly what I was searching for!

If all you're using Quantopian for is to fetch and execute external signals, then as Grant pointed out, that defeats the whole purpose of the Quantopian platform, so why bother? You'd be better off writing your own code to talk to one of the numerous available broker APIs to place your orders.

If you're using Quantopian the way it's intended to be used, i.e., your algorithm's logic is implemented in Python and executed by Quantopian, then at some point Quantopian has to actually have the source code in order to execute it. If we were going to steal your code -- which we're not -- then we'd be able to do it at that point. So allowing users to store their code elsewhere doesn't really provide any additional protection for it. Mind you, there are other reasons why we might want to support it -- for example, we hope to do a Github integration at some point to allow users to gain all the benefits that would accrue from being able to version-control their code -- but security isn't the major motivator for such features.

We've communicated with our user base numerous times on this topic (e.g., http://blog.quantopian.com/on-trust/, https://www.quantopian.com/faq, https://www.quantopian.com/security, https://www.quantopian.com/posts/updated-terms-of-use-and-privacy-policy). We've committed not to look at the algorithm code of anyone who doesn't give us permission to do so, and we will honor that commitment. We are all ethical people who have no desire or intent to enrich ourselves by stealing others' intellectual property. Aside from that, as Grant pointed out, our entire business model is predicated on having the trust of our user base, and squandering that trust by violating it would be harmful to our interests, so there is no incentive for us to do so.

Jonathan, I haven't used Quantopian yet so maybe I am not getting what is the main goal of the tool. I was searching for an easy API that I could use with python to test my financial algorithms with historical data, paper trade and finally live trade. Quantopian seems to have all that so it's perfect in the sense that I don't need to waste time starting from scratch. I understand that you have commitment not to look at people's code those honorable commitments are nice but they are not used since the middle ages, specially when there are ways to ensure those commitments and until now no one here was able to say WHY does the code need to be in your server. You will never lose the trust of your users because your users will never be sure that their code was stolen or not.

Bottom line:
- Why does the code needs to be in your server?
- If the answer is: it doesn't have to be you can use zipline and have all the features (without the comfort of a nice IDE of course). Then it's perfect I will use the tool and gladly pay a fee or something.
- If the answer is: because we have a commitment and we are going to honor it so don't worry. Then this is just an evasive answer that makes me lose trust in what you're doing here.

Jonathan, I haven't used Quantopian yet so maybe I am not getting what is the main goal of the tool.

Indeed, you are not.

Quantopian is not merely "an easy API," nor is that what it is intended to be. Quantopian is a full platform for the development, testing, and execution of algorithmic trading algorithms. We provide an integrated development environment, a debugger, more than 13 years of minute-bar historical stock pricing data, Morningstar fundamentals data, and a complete, ready-to-use framework for backtesting and live-trading algorithms. Over time, we will add more data sources and features to keep Quantopian at the vanguard of algorithmic trading in your browser.

Quantopian is also a crowd-sourced hedge fund, enabling our users who create successful trading algorithms to manage money in our fund, reaping the profits from doing so without any down-side risk.

Our goal in creating Quantopian was to make algorithmic trading as accessible as possible to as many people as possible. Providing nothing but an API for people to use would not have achieved that goal. The API is only a very small part of what we provide to our users so they don't have to build it themselves.

Why does the code needs to be in your server?

The code needs to be on our platform so that it can be edited within our IDE.

The code needs to be on our platform so that our users can take advantage of our collaboration features to work on their algorithms with other users and, sometimes, with our support representatives.

The code needs to be on our platform so that we can execute it.

The code needs to be on our platform so that it has access to the data we provide, whose licensing restrictions prohibit us from allowing them to be exported out of the platform.

those honorable commitments are nice but they are not used since the middle ages

I am really not sure what you mean.

Pretty much every single web site you visit which collects any sort of data from or about you has a Terms of Use page and/or Privacy Policy which spells out exactly what they will and won't do with those data. These are legally enforceable, and indeed, have been enforced through legal action in the past.

You will never lose the trust of your users because your users will never be sure that their code was stolen or not.

If we were to make a habit of stealing people's successful trading algorithms, then you're right that the first user whose algorithm we stole, and the second, and the third, ..., probably wouldn't notice. But eventually someone would notice, because the only way Quantopian is going to be financially viable is if we run a substantial amount of money through the algorithms we trade, and eventually somebody will pick up on the fact that some other mysterious entity is placing all the same trades as their algorithm, but in a lot higher volume, and disrupting the market sufficiently that it's reducing their profits.

When that happens, we'll get caught, and we'll be out of business, and quite possibly some of our principles will end paying huge fines or spending some time in jail. We're really not interested in taking that kind of risk; the risk of working for a revolutionary startup is more than enough, thank you very much.

But let's be clear... That's not a risk we need to take, because the business model we've created allows both us and our users to be successful without us ever taking unauthorized advantage of their intellectual property. Indeed, our business model only works if we give users adequate financial incentive to develop successful algorithms and let us license them for the fund. If we steal people's intellectual property rather than paying them fairly for it, then we will fail; it's just that simple.

until now no one here was able to say WHY does the code need to be in your server

As you have said, you haven't used Quantopian yet and you are new here. So please, allow me to point out, for your benefit and the benefit of other new users who might be reading this thread, that the question you are asking -- why does your code need to reside on our server -- has in fact been addressed many times, explicitly and implicitly, in this forum, in our documentation, and on our blog. It might be beneficial to you to read what has been said before, before falsely asserting that the questions you are asking have not been asked or answered.

Then this is just an evasive answer that makes me lose trust in what you're doing here.

If you take some time to read our forum, blog, and about page, you will observe the following:

1. We are as open and transparent about our business as any company in existence, and more so than most. Take a look, for example, at the post-mortems we've published on our blog about the (minor) security breaches we've experienced in the past.

2. Nearly every Quantopian employee is active in the forum. Our CEO posts here regularly. We are also active on Reddit, HN, etc., where we respond as openly and transparently as possible to questions about our platform, always, of course, disclosing our affiliation when doing so.

3. Every single Quantopian employee is identified on our about page, with links to their social media, LinkedIn, etc. You, or any other user who has concerns about whether we can be trusted, is free to dig as deeply as you want into our backgrounds and evaluate whether you believe you can trust us. A simple Google search, for example, will take you to my blog, a quick perusal of which will lead you to the conclusion that in addition to being snarky, righteously indignant, and judgmental -- hardly uncommon on the internet! -- I suffer from what my wife sometimes calls being "ethical to a fault"; it will be clear to anyone who spends any time reading my blog that I wouldn't be caught within a thousand miles of Quantopian if I there were even a whiff of unethical conduct here. We are staking our personal reputations not only on the success of Quantopian, but on the ethical success of Quantopian.

You won't find anyone else, in any industry, being more open and transparent than we have been. To accuse us of being "evasive" in any way is simply false.

I encourage you to take the time to fully understand what Quantopian is, what value we provide to our users, and why our financial interests are aligned with respecting and honoring your intellectual property rights. Having done so, if you still prefer not to utilize our platform, then we fully respect that decision, and of course you are free to use Zipline to create your own algorithm execution platform to use instead of ours.

Hi,

woah, everything got a bit heated ;-)

I don't mind keeping my main code on the server, but there are still external sources of information which could be pertinent to my algorithm.
Is there then no way to make calls to other APIs in order to provide additional data to my algorithm? Is this something that would be considered for the future? Not all of the variance in stocks can be explained by past behaviour, so if we are trying to create the most effective algorithm it seems obvious that at some point more data sources will be needed. In addition some types of machine learning algorithms are much easier to program on e.g. matlab/octave than on python.
A simple procedure to call external APIs seems logical since it would be reasonably easy to integrate with the existing platform.

Any comments appreciated.
Ralph

Is there then no way to make calls to other APIs in order to provide additional data to my algorithm?

You can use fetcher to import data into your algorithm from the outside world. Backtests import fetcher URLs once, at the beginning of the backtest. Live algorithms import fetcher when you first launch the algorithm, and then subsequently each morning before market open (when, exactly, varies from day to day, so you need to make sure the URL you specify to fetcher is available all the time, or at least when you launch the algorithm and subsequently between midnight and 9:45am).

We've got a project on our roadmap to enable intra-day fetcher, so that you can get to external data more frequently than once per day, but we don't yet have an ETA for when that functionality will be available.

Ah I hadn't found that, thanks for the info. Looking forward to adding external sources by the minute if that becomes available.

I recommend that Jonathan be put on as a full-time Quantopian help desk first-responder! His talent for emphatic, impassioned feedback should be featured more regularly!

For new folks, one of the magical things here is that Q has convinced a number of data vendors to supply institutional-grade data sets to the masses, and has systematically covered the risk of those data sets being stolen in bulk. So, to take advantage of the data, you need to use the online platform. There is no offline API (other than zipline, if you want to call it that), because there would be no free data magic. If the Q vendors said "No problem. Give away all of the data you want. We're tired of making money on our data sets." then Q would be different, and better is some respects. But that's not reality.

Regarding adding external sources by the minute, my hunch is that it won't happen any time soon. Jonathan can elaborate (although it might be Q IP territory that he doesn't want to discuss), but my sense is that any additional feed would have to be folded into their real-time minute bar feed, derived from Nanex Nxcore (see help & FAQs). They compute minute bars on the fly from tic data (I think). So, to have everything work seamlessly with another feed, it would have to be synched up. I could be wrong, but I don't see an immediate path for a user to provide, for example, some IP address from which Q could pull real-time data via some arbitrary XYZ protocol, if that's what you are looking for. It is more likely that they add additional curated real-time data feeds, as they are today for the Nanex Nxcore feed.

"Every single Quantopian employee is identified on our about page, with links to their social media, LinkedIn, etc. You, or any other user who has concerns about whether we can be trusted, is free to dig as deeply as you want into our backgrounds and evaluate whether you believe you can trust us."

Come on, this is just silly. Apparently trustworthy people commit fraud or other financial crimes all the time. People trusted Madoff for decades. The problem with your argument that you wouldn't have a business if you ripped off other's code is that only makes a difference to the owners of the business. It's not much of an argument for low-level employees who have far less skin in the game than the principals. A low-level employee stands to gain a great deal by filching code (they can sell it to the highest bidder or trade it on their own account - either way earning far more than Quantopian pays them if the code is good).

Correct me if I am wrong, but is Quantopian really offering no guarantees or protections against employee snooping for code stored on your servers?

It would not be advisable for Quantopian to share the nitty-gritty details of how they manage security; I wouldn't expect a step-by-step technical response. That said, conforming to industry standards/best-practices and having an independent audit would be things that would provide some assurance. Also, as they take on other people's money, I'd guess that they will be subject to information security compliance requirements.

What's the alternative? Unless you run algos on your own hardware, under your physical control, and have air gaps/unidirectional links, you're gonna have the same kinda risk, regardless of who hosts your algos in the cloud. If anything, Quantopian has more skin in the game, versus a vendor who is just doing retail algo hosting.

Grant: "Unless you run algos on your own hardware, under your physical control, and have air gaps/unidirectional links, you're gonna have the same kinda risk, regardless of who hosts your algos in the cloud."

I don't agree. It's a big gap between running everything on your own isolated hardware and allowing third parties to see your code in plain text without your knowledge. There are a number of measures Quantopian could employ that would at least ameliorate the risk of code theft, but I can't see anything in the terms of use or privacy policy that addresses this. Perhaps because they want to minimize their liability in case it does happen.

Hi Bill,

On https://www.quantopian.com/policies/terms, it states:

Unless you choose to share your Content, your Content will remain private. Quantopian will not review, share, or otherwise make use of private Content except as specifically provided in our Privacy Policy or this Agreement.

I haven't looked for exceptions buried in their policies, but I've been with Quantopian since 2012, and they've been very consistent on this point.

Regarding your comment about "a number of measures Quantopian could employ that would at least ameliorate the risk of code theft" you might contact them directly to see if they could give you some assurance, and would be willing to share a few details. Is it common practice for cloud services to publish technical details of how they manage information security? I'm guessing that it is more typical to publish policies, and maybe say that they've passed an audit and such.

Overall, I share your concern, however, given the number of high-profile information theft cases in the news (which is probably only a small fraction of the incidents).