Here is the same test with pb_ratio. This was the ratio used in the original paper by piotroski. It was also the best performing metric according to the quant investing link above.

125% drawdown.......... Momentum crashes are a thing. Maybe I should coin a term called the Piotroski crash...

I'm 100% sure something is wrong with my method though. Will follow up when I find something.

Great work. 2009 crashes are expected as quant strategies suffered at that time, and both of these were used by folks. One problem with Piotroski... you get a few stocks and they might not be liquid enough. Really nice to see that the algo found stocks that I remember having good Piotroski scores back in 2003.

@Johnny Wu,

Thanks for your posts. Please continue to do so because I am learning a lot about fundamentals from you.

Best regards,
Beginner

Kai: Thank you for taking the time to respond and I find your real life experience to be valuable in finding a solution to this problem.
I do have a few counter-arguments though.
The portfolio value is not so high that slippage from poor liquidity should explain these crashes. I ran the test again for market cap over 10 billion, and it still has a 90% drawdown. I decreased the starting capital to 10k and ran it right before a crash. The algo literally had to buy 1k worth of stock of giants such as Astrazenica. These dips were confirmed by checking individual stock performance, and indeed they do have poor returns. It is not an artifact.

You're right in that it could be due to low number of stocks that pass the filter during these times. Perhaps it could be explained by a concentration of risk? I don't really think this explains it either because there are other times when number of stocks are pretty close, but do not experience this crash.

Also, I don't think that enough people use the piotroski scores that it should affect these stocks specifically. Even if a large number of people did use it, Piotroski is a value-based indicator, shouldn't value investors know not to sell during dips if they are confident in their fundamentals?

If one is imaginative and does not care for evidence-based explanations, you could dream of a reason like... these high quality, liquid, cheap stocks are cheap for a reason, the market hates them, and when the market needs stock to dump during a crash, these are the first sold due to the bad press surrounding them. Reasonable explanation, but that's not what I'm looking for because it cannot be proven. I'm also pretty sure something is wrong with my code and not the strategy as every other backtester from other sources have had success with piotroski.

And thanks Beginner :)
We're all learning together

Stian Andreassen gave me this great idea on the other thread that I should use a stop loss order. Unfortunately using a simple order_target_percent with target zero percent will sometimes produce pending sells even after another procedure has already sold from the portfolio, effectively shorting the stock.

I hacked together a stop loss procedure for post #1.
Basically, every month I purchase the top ranked stocks according to value and piotroski and instead of holding for the entire month through thick and thin, I exit if the stock dips x% and move it to cash for the remainder of the month.
This backtest uses x=5%

As you can see, it doesn't do very much. Not a worthwhile strategy.

In biology, we would say that we need a "negative" control to see if the effect on the results were due to the intended treatment and not a confounding factor. So here is the x=15% version.

Certain metrics are better. Sharpe ratio is better for instance.

Time to give up on Piotroski+value? Unlike pure value, it almost always outperforms every year, but the drawdowns are just too much for one's health.

I really like the idea of piotroski, though. I think I will use it to filter out really bad stocks instead of using it as a main criteria.

Please ignore this entire post, I found out that leverage steadily increased. No wonder the drawdowns were insane.

Great thread! I'm curious to know what the performance looks like if the leverage bug is fixed?

I think that anyone who visits this thread gets an email every time I post. I apologize if I'm spamming your email, but unfortunately I can only post one backtest per post :(

I fixed the bug that caused the leverage. The bug is actually part of the template that was in the tutorial code. This shows how unintuitive the order system is. I hope that there will be either
1. An improvement in the system so we don't have to keep tracking leverage
or
2. An explanation of how the order system works and how it can cause leverage bugs along with common errors

Anyhoo, here is the original piotroski (not modified). I reverted to the original piotroski because it's easier to get a grasp of what the score means in the context of out of sample scores from outside Quantopian. Although I won't spam the results, I found that the enhanced version produces better results.
This looks more normal. I like it.
How do I know that Piotroski scores are improving the results? I ran the program with middle scores (4-6) and results were meh. I also ran <= 3, and the results were just atrocious. I suppose I could code a piotroski neutral version as it would be a better negative control, but I have to clean the house before the Mrs. returns.

Now this is the reason why I'm so excited: the stop loss
It works really well! For the most part, it is able to avoid large drawdowns, even during the value mini-crash of 2012. Here it is at 10% stoploss.

By the way, the program needs to run for a year before it can start trading. This is the reason why it matches the benchmark for the first year. That is intentional to ensure that they are visually comparable (I called this priming in my code). I also changed the benchmark to IWM (Russell 2000) because I think it's a better representative of my investable universe.

Here is the 20% stoploss. Less isn't always more apparently. Better results but lower Sharpe. If found to be robust, I'd favour the 10% over the 20% any day.

There is an error in the code. Current_data long term debt to equity ratio should be lower than old_data long term debt to equity ratio.

Johnny, I am new to algos and am looking to adapt your source code. I understand that this is a few years old. When I clone and go to run this as is, it breaks with the fundamental_df. Does that have to do with some changes quantopian has made? Thanks