I would recommend not being super focused on up/down accuracy in isolation. You can have a directionally correct accuracy of 25%, but if the stocks move in that direction 4x more than when they move in the opposite direction than that could be desirable to other models with higher accuracy and lower directional movement.

I'd also be sure to work 2008 into any model that you can get data for as a good stress test. Many of my L/S models have accuracy rates in the 60%+ range and are not something I get excited about. Some are pretty interesting with 45% accuracy rates. You just need to be sure you are taking in the whole picture.

Thanks for your reply Spencer.
Something here I don't understand, If your model is 25% right about direction then it means you are 75% wrong. So 75% of times you think you should short or long will be wrong (Also you can reverse your model prediction to be 75% right in this situation, but your strategy seems to be different from this). How can you profit from such models? I thought the most important part is to have a ranking system which can be right more than 50%

Okay pretty simple concept.

For example we can compare two models with different accuracy rates.

The first one correctly predicts direction 45% of the time and makes +$0.06 on average vs losing 55% -$0.02 on average. So with this model if you do 1000 trades you will have made +$16.

On the other hand you could have a model that is correct directionally 70% of the time, but only makes +$0.02 vs losing -$0.06 30% of the time. In that case you end up with -$4.00 after 1000 trades.

Directionally the second model is more accurate, but it still is not the preferable model in terms of +EV. Not saying a high win rate is bad, but in many cases it is just one small piece of the puzzle.

I would add one more thing.
- How does it compare to the market. For a strategy to be good, its return should beat the market consistently enough. Sometimes it beats, sometimes it doesn't is not a strong sign.

Thanks for your insight.