They're not possible. If they were the algo writers would be hired by the Citadels of the world and be making 7 digits.

But, if they were capable of even 30% of their recorded performance, Quantopian would be foolish to sit on the sidelines with so much potential profit leaking away every month. 650 algos? And they pick ONE every month. Why bother? Really. Why bother? They should at least scale it: 1 the first month, 2 the second...

I agree. Buying a lottery ticket is cheaper in terms of time and effort :)

When the contest got kicked off, there was a long discussion regarding ways of biasing a backtest. Frankly, not biasing a backtest for the contest would be a sign of a not-so-clever quant. And pretty much anything can happen in a month, after only a handful of trades. So, yes, perhaps a lottery ticket would be a better choice, but then you'd have to pay for Lumosity to keep your mind sharp, instead of coding algos for free! : )

Another approach would be write a reasonable algo that might work for the hedge fund, submit it to the contest, and send it to them, code and all. Ask Q to read the code, understand it, run it themselves, and provide you with feedback. As I understand, they'd be perfectly fine with this approach, since it is your IP and you can do as you please with it. I could be wrong, but my sense is that they'll never get to $10B under management with black-box algos anyway. And if they tell you to get lost, or don't give any feedback, well, then give up.

Good idea Grant. I also think Q should give points to people who actually trade their own money with the algorithm as you would never put a gamed algo live. I'm one of those idiots who does not enter an algo in the contest if I would not be a willing to put my own money behind it. Now people seem to game the contest plus this beta demand... MAkes me a lot less interested in the contest. Especially when one of my algos is still ranked in the bottom because the calculations of Q are bugged.

I was kidding about lottery ticket else I would not be here. But frankly, even if I tried hard to bias the backtest, I cannot make 200% returns with double digit sharpe. I just wonder how they do it.

Beginner - It's the beginning of the month, and the leaderboard is full of algos with only 1 or 2 days of paper trading. The results are very volatile at first. Yesterday's leader is #19 today, and who knows where today's leader will be tomorrow. Not only do longer paper trading records mean a more stable paper trading score, but the consistency factor ramps up each day. A biased backtest can't maintain consistency between paper and backtesting, and they drop down the list.

Market - One a month for now, dozens per month in the fund.

Peter - I thought we got that fixed, but evidently not. Will check on it again.

I would love the current winner to come discuss her algorithm a little bit.

Perhaps just a little insight ... A magician never reveals his secrets as I understand it.

But the results were astonishing. Truly boarding unbelievable. A nudge in the right direction would be plenty.

Yup my post was actually a tribute to current winner. How did they do that?

@Dan, "dozens per month in fund" is exciting.

@beginner,

Yeah, if you are interested in this stuff, hang in there. I've been tinkering around on Quantopian since 2012, and part of the fun is that you get to participate in the sausage making of product development. For the contest, Q got to a point where the lawyers had blessed it and they just pushed it out. And now they are iterating. As long as nobody loses an eye, it is a great model. There are a lot of questions they'd never have answered by keeping it in the lab. How many algos will we get? To what extent will participants cheat? And if they do, how might we manage it? Will the real money results correlate at all with the backtest and paper trade results? Might we be able to identify hedge-fund-ready algos with backtest and paper trading alone? "Oh, we never thought of that. Hmm?"

The $100K of capital with monthly winners is a small incentive to the crowd to help in developing the tool, which is the Quantopian Algo Harvesting Algo (or AHA for short). My read is that it Q would want AHA to scale up to their 35,000+ users, and could be run on a rolling basis. Every day, they'd like to walk into the office and have a new batch of algos to consider for the hedge fund ("AHA, it has done it again! A true gold mine! " ).

One potential problem I see, though, is that it still appears difficult to sort out if one's algo is decent or not. It is a reasonable assumption that some of the backtests are biased in some fashion, either intentionally or unintentionally. So, for a given month, participants have to rely on just one month of paper trading results to gauge the goodness of their algos with respect to the crowd. It is important that algo writers get accurate feedback with respect to the pool of algos that would even be considered for the hedge fund. So, it seems like there is some room for improvement in this feedback loop.

Overall, I think Q has strong incentives to get all of this sorted out quickly, so that the best algos bubble up to the top. They gotta succeed, or they won't be able to pay the light bill.

Hi! Nice post, Grant! So do you think that there is the possibility that some of the algos are gamed? How to disqualify them without looking at the code? Please check out my post here: https://www.quantopian.com/posts/gamed-slash-backtest-overfitting-algos-in-the-contest

Yea the best back test returns I have been able to return staying in the new beta constraints is 30% with a 1-2 Sharpe. I saw one guy with a 550% return 15 sharpe and a .01 beta to me that seems border-lining on unbelievable. Just my 2 cents.

-Spencer

If you click on the name of the entrant in the leaderboard you can see how many backtests they've run which is somewhat of a proxy to how back-fitted their algo may be, or at least how much time they've invested. I'm in the 800's so clearly guilty, but I feel like I'm in good company with Grant and the other winners. The most amazing thing about the May winner was that she managed to only produce 11 algorithms and only backtested each of them on average twice but still come up with those results, as well as a respectable 26th and 38th place algorithm. This might be the type of prodigy Quantopian is looking for and the rest of us are a bunch of posers!

How often are the leader boards updated?

I have am ago that I submitted yesterday, and it hasn't shown up on the leader board.

Any idea why ?

@Adrian,

The process isn't perfect, but as I mentioned above, Q has incentives to improve it. And as I suggested to beginner above, if anyone has an algo that is a real humdinger and can back it up, well, make sure Q looks at it (or at least the results). Just contact them directly and tell them you want a few minutes to make a case that you deserve capital. You won't get marked down for lack of passion.

An* algo*

On a side note,

I really want the ability to edit posts.

I often browse the forums from my phone, and don't want to spam with corrective posts, no more than I want to leave my mistakes public lol

@James, the leaderboard is updated daily after market close. Typically between 5 and 8. There's a novelty twitter account (@quant_cosell) that tweets out updates.

Hey Josh,
I was excited to see where my new algo has ranked, but only my old one appears on the leaderboard.

Is there a chance I just missed some sort of cut off last night ?