I am a newbie but am surprised to see the returns to be so low. If I understand correctly, per-day returned dollars would be to multiply the return (say 0.0045) with the total capital invested.

For example if I sum up Grant Kiehne's real money returns till now, they show -0.10; Does this mean his algo had a 10% loss thus far? Thanks for the clarifications

Sa

Yes, his algo (and mine) were both stopped and liquidated when they hit $90k. Mine was subsequently restarted, since they decided it had been a mistake to stop it, but has since done poorly as well.

The take-away is that writing profitable trading algorithms is not easy, and from personal experience, it's worse if you cut corners during development that lead to backtest overfitting.

Simon.

Hello Sa,

I posted my algo:

https://www.quantopian.com/posts/winning-algo-drops-below-$90k

Aside from being high beta, long-only, there was probably some over-fitting combined with bias in the fixed universe of securities. The rules and administration of the contest have evolved, so this sort of thing is less likely to succeed down the road. I knew that the algo could do poorly in a flat or down market, so it was a bet that the market would continue to go up, and that the securities picked would continue to out-perform. Since I had nothing to lose, it seemed like a good bet.

Note that you can clone my algo, and see how it has performed to-date by running a backtest. You can also change the stock universe, and play around with the parameters, as well.

I would agree with Simon's statement "that writing profitable trading algorithms is not easy" and the Quantopian team has gotten much better at filtering out corner-cutting. On the other hand, they continue to up their game in terms of the tools provided for algo development, so everything seems pointed in the right direction. It's still gonna take a lot of work and luck to find a consistently profitable strategy that will scale to $5M-$10M.

Grant

Quantopian open winners Real Money
https://drive.google.com/file/d/0B10uueaFeSYvdURGU1daVDNMV3M/view?usp=sharing

This is what you are pointing to. I don't know who produced that. Someone pls explain how the average could be beneath the other values.

Ahh Garyha, the non-intuitive nature of compounded returns and geometric averages. For example, consider two days of returns; 100% on day one (I doubled my money!) and 50% loss on day 2 (I lost half my money!). The simple average is a 25% gain, but the compounded return is 0%. Or consider the reverse with a 50% gain followed by 100% loss; even though your average returns were only -25% you have no money left.

That is what is happening to the Q-fund. Even though the individual returns are now -10.2%, -8%, -7.9%, -8.3%, 3.1% and 0.2% (in order of funding) averaging about -5.2%, the overall portfolio is -9.2% if you compound the daily returns together; mostly due to the fact that the first four winners suffered large draw downs.

A couple of words of experience. Q can claim that they started a fund with a fixed amount of money (say $1,000,000) and that each $100,000 contest winner was given an allocation of the pool. This allows Q to claim that the overall performance of the entire fund is much better. This scheme works when you are losing money, the un-allocated capital brings the overall negative performance up. Of course, in a winning fund the un-allocated capital dilutes the performance and no ethical manager reports performance sans cash. Also, actual money management is constrained by investors unwilling to pay a manager for holding cash and the periodic subscriptions and redemptions every fund faces. Q did not, AFAIK, announce a set pool of contest winnings and so the only treatment that is fair (and real-world) is that new winners capital is treated as a subscription around the time of investment.

As an aside, the most critical decision a fund manager faces is their first few investments. It is hard to dig yourself out of a hole with compounding returns. Also, greenhousing performance, where you start multiple funds and then market based on the winning funds, is a big NO-NO.

YMMV

The contest is a fun promotion to encourage our community to develop high quality algorithms; it is not a fund. We are not making any claims about the contest winners. Claims are not necessary because we are disclosing complete data.

When I look at the contest, my primary concern is whether our selection has improved over time. Are we engaging the community to produce higher quality algorithms, and are we ranking them in a way that surfaces the highest quality algos? I'd love your help scrutinizing that.

The contest is a laboratory for Quantopian's research team to iterate and improve our evaluation tools (pyfolio, also opensource!) and our selection criteria for the first fund product. Overall, the algos winning for the past few months are better constructed (hedged, clear and intentional position building/reducing, tightly managed leverage, universe screening with fundamentals), and now we can all watch to see how they perform with real money.

I've paid for this learning by taking the trading losses incurred along the way. Because we are exclusively using Quantopian's money, we can approach the problem of finding high quality algorithms scientifically, and with a long view. The fund business is plagued by short-term goals that promote short-term thinking. The contest is a series of experiments that seeks to optimize long-term algorithm production rather than short-term returns.

We will continue to share our complete data on contest winner performance, so you may conduct your own analyses, and draw your own conclusions. I'd love to see your analysis, especially if it is in a research notebook :).

thanks,
fawce

Hi Fawce,

Hmm? I have to wonder that if all of the contest winners' algos had performed splendidly, you'd say something else. "The winners represent the early stages of our crowd-sourced hedge fund. As their 6-month real-money out-of-sample trading wraps up, we will increase their capital levels and roll their strategies into our fund." In fact, you are implying that the recent batch of winners is just that: potentially fund-worthy algos funded at a seed level. So, I think that Sally has a good point. Until you actually put money into other algos as part of the fund, the contest is your fund. And the aggregate performance has been relatively poor. You can't just say it has been a learning exercise.

Grant

FYI the CSV files of the winner returns are totally broken (downloaded 2016/Sept/01). The rows all have different random numbers of columns, many cells are an empty quoted string instead of a number, one cell in the middle of the 3rd file even contains a URL (just weird).