One issue I consider with many Quant factor strategies (and why they mightn't achieve alpha consistent with backtests) is that they assume they are first actors, but on the same signals that many others are playing. Not only are they often not in the trade at the right time, but a second issue arises with the systemic risks associated with loading up on the same factors others are in as well. Think back to 2008 when all the L/S funds inverted when they saw their shorts shoot up because they were getting covered by others who had the same positions, and having to do this while selling off their longs (to other sellers of the same positions)... Those highly crowded trades got burned, bad.

I'd be curious if there are datasets that include the actual strategies (or a sample of them) that are run in the Quantopian simulation universe.

If we're talking about fun algorithms to build here, it'd be worthwhile to try something like the first derivative of the data challenge Quantopian has framed.

I'm new to the forum, but when they run a challenge, is it based on the same historical dataset and just pluck a winner, or do they let it run in semi-real time against eachother? ..And/or historically (but incorporating the other participant's actions/algorithms in that historical simulation)?

That'd be the way to find the truly better algorithm -- the one that ate up the others liquidity of the named positions at the right time.. adding some timing and finesse could yield a superior algorithm and performance in the market. Each time someone made a tweak to some of their rules (perhaps instead of buying an asset whenever it was in the top decile, you instead added a window for a limit buy order below market at a range dependent on it's historical stdev), that tweak/change to their code could be re-run against all the other algorithms in the database (preferably with L2 liquidity information on the field), and see which back test will be the best. Whose cuisine would reign supreme.

This method creates actually new problems with simulated prices in the backtest however.. Because if you are pitting all of these actors as employing their algo-strategies against one another, the ones that actually execute and have larger allocations and capital deployment will alter supply/demand for those assets and create bubbles that are unlike the market data that the system is being re-calibrated on at each point, there is a path-dependence that is lost.

I don't know if the system can calibrate to really tell the success of an algorithm without knowing something about the field in which it is competing.

...but I suppose that's where the "art" component comes in, feeling and sensing the market, and mixing the quantitative with the qualitative, the human element.. it's like that Star Trek where Picard get's kinda assimilated by the Borg.. Is he Robot/Collective, or is he Picard? Can he come back? When does he start kicking ass again? Is it the efficiency of his network brain of computerisms or is it the individualistic nature that outsmarted it? {insert cheesy reference to boldly going where no 'man' has gone before}