I did notice it as well on two of my entries (both quite new ones so not affected by the rolling 63-day rolling window). Not significant, but definitely different. One was slightly higher for me, and the other one, which had had the same score for three days in a row (which I thought was very odd), just marginally changed but no longer have the same scores on those consecutive days. Not sure what's going on.

New default slippage model maybe? I'm getting slightly worse backtest results too, with the exact same code...

Daniel & Joakim:

There are a few different things that could change the score of a contest submission in such a way that the score from yesterday's graph looks different than the values that was actually published yesterday. I'll list a few of the more common ones below.

1. A submission has now been in the contest for more than 63 trading days. This is relevant because the score of a contest submission comes from a trailing window of up to 63 trading days. Once it's been in the contest for longer, it's earlier days in the contest stop contributing to the score. This is explained a little more succinctly on the contest page.
   
2. An algorithm is non-deterministic. If an algorithm is non-deterministic (e.g. has un-seeded random functionality), the simulation from night to night will be different. We run a new backtest for each contest algorithm every night, so if an algorithm is non-deterministic, the historical scores could look different.
   
3. We make an API update that changes the behavior of the algorithm. This is very similar to 2) in that a software change can make the historical results on a new backtest look different. This can happen from time to time but I can't think of any major changes that were made recently that would have impacted the results.
   
4. We update historical values of a dataset. Also similar to 2) in that a historical data change can change the 'path' of a backtest. This did happen recently with FactSet Fundamentals. In the announcement post, we mentioned that some of the data in Sep. and Oct. 2018 might change causing changes in simulations. Here's the relevant text from the post:

We are expecting to apply a minor update to this dataset in the next couple of weeks. Specifically, there are some data points in September/October 2018 that are surfaced a day late. When the update is applied, some of your simulation results might change, but it should only be for the September/October 2018 period. Since this data is in the holdout period, you would only notice this if you use FactSet Fundamental data in a contest algorithm.

Do you think any of these might explain what you're seeing?

Hi Jamie,

In regards to the above, the only option I can't eliminate is #2 only because I am not sure if it applies to mine.

The only instance where this occurred was between the 5th and 6th and looking backwards. For example if I select the 5th and look back to my score on Oct 31 it is consistent with the scoring if i chose Nov 2 and looked back to Oct 31. However, if i start from Nov 6 and look back each day prior my score from Oct 31 is a lot different. I wouldn't expect the score from Oct 31 to change just because of a new day.

Thanks Jamie!

1 and 2 I can rule out for my two affected entries. Both were submitted on or after Oct 22, and both should be deterministic. One uses exclusively FactSet data, but the other one doesn't use FactSet at all (doesn't import it), so it's a bit weird that that one would have been affected.

Joakim, could you send an email to [email protected] with the name of your affected entry? (Daniel, I believe you already did this, thank you).

Done!

Also, I forgot to thank you guys for allowing an additional contest strategy with the FactSet Fundamentals roll out. I think this is a very cool and generous way of celebrating a big achievement with the Q Community!

Done
Just to piggy back off Joakim thumbs up to the team!!