Hi Jay,

To get a bit more info, could you run it past the go live date (for true Out of Sample data), with the following options when creating the tear sheet: (live_start_date='YYYY-MM-DD', round_trips = True)

The old contests were using a cost model VolumeShareSlipage that consistently underestimated actual slippage in live trading. So that cost model is no longer under use. Rerunning with current backtester's default transaction/slippage cost model is recommended if not already done so and for a wider time interval of 10+ years.
Try entering the algorithm in the new contest and see how it is faring. The new contest is more in line with the allocation criteria.

@Leo M,

So by implication, all the 23 members that already received allocations were evaluated using the old Slippage Model which you say "consistently underestimated actual slippage in live trading". Umm, wonder how they are doing now?

@Jay aka Apricot Parrot,

With new rules / thresholds / format, many algos written under old contest will not hold . Like Leo said, try to adapt your algo to the format of the new contest.

@James, it appears to me that Q4-2017 or later allocations might have been made using the new slippage model.

@Leo,

New Slippage Model was announced December 18, 2017 changes-coming-to-the-default-slippage-model
And was applied to Contest 38 which is still on going... contest-38-rules-changes
So, I believe no new allocations were yet made using the new slippage models via contests.

@Joakim,

Even if Jay ran a more detailed live round trips notebooks, it will not give accurate information about the results of the algo. For one, the algo executes orders on open and tries to close all one hour before closing, an intraday strategy that avoids overnight risks. The notebook only covers at minimum, daily frequency strategies. Always pay attention to warnings in the notebook, such as:

UserWarning: This algorithm has relatively high turnover of its positions. As a result, performance attribution might not be fully accurate.

Performance attribution is calculated based on end-of-day holdings and does not account for intraday activity. Algorithms that derive a high percentage of returns from buying and selling within the same day may receive inaccurate performance attribution.

Rightio.

@James, https://www.quantopian.com/posts/changes-coming-to-the-default-slippage-model "The new model is more accurate and more consistent with the tools we use internally to evaluate algorithms for capital allocations." Jess's statement indicates they already shifted to the fixed basis slippage and that they probably have a means in their evaluation routine to override the algorithm's slippage/transaction costs to any model they want.

As to the number of allocations, in https://www.quantopian.com/posts/a-new-contest-is-coming-more-winners-and-a-new-scoring-system Dan Dunn states that "In September we shared that the number of allocations was 14". One can make an educated guess from that and recently quoted numbers.

@James, Bingo - this algo avoids overnight risk... which is not a bad thing.

"Performance attribution is calculated based on end-of-day holdings and does not account for intraday activity."

This is a problem with Quantopian... this algo isn't gaming the system or gaming the contest. The algo was designed to be extremely selective and to avoid overnight risk. Intraday algos shouldn't be entirely dismissed at Quantopian... they're literally some of the most successful algos being deployed at HFs. Maybe a different risk assessment should be developed and applied to intraday algos.

I did end up converting this algo to use optimize ordering, and developed later versions that actually outperformed this one and traded more frequently. I'll try to post some more tear sheets soon when I get a chance.

@Jay,

I hear you, I placed third on this contest you are referring to. I had much higher returns and stability than you did but you beat me on lower volatility and drawdown. Guess what, I had the same intraday strategy you did that avoided overnight risks.

Bottomline is after many years, Q is still a work-in-progress, hopefully heading into the right direction and getting better. It is understandable that along the way, they will need to make adjustments, change rules, format, etc. based on their actual trading experience and objectives. Unfortunately for us, the crowd , we suffer collateral damage as a result of these changes. Their game, their rules, we play!

Hi @Jay,

If you're concerned with version control, perhaps copy your contest code into a new algo, and run a longer backtest with more OOS data (from when you submitted it to the old contest), and run a new tear sheet with those two options enabled?

As for overnight risk, wouldn't this effectively be cancelled out in a Long/Short strategy? If you find that you're consistently losing money on both the Longs and the Shorts overnight, then that to me sounds like you should be able to make money by reversing the positions overnight?

@James, That's an interesting bit of information! Good luck on the other contests.

@Joakim, Even a well paired long/short portfolio would be subjected to market risk/event risk if held overnight. Eliminating overnight risk is one of the only free lunches left in the market.