Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Why consistency score is flawed

I tried to reproduce consistency score using gaussian kernel density. I have two series. One with a mean of 20% and another with a mean of -20% with same variance. Guess what the consistency score is? 90%. This is clearly flawed. I am clearly not an expert with this but you can play with the random generators and notice that sometimes different distributions get a high consistency score.

1 response

Without disrespect to anyone, here is an algorithm in contest with rank 4 today and consistency score of 83%. Now see the difference between paper trading and back test:

annual returns: 0.47% vs 87.44% (really?)
sharpe: -1.58 vs 25.49

I won't go through the other numbers.