Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
How to programmatically detect stock patterns, what algorithms well-known screener are using ?

In Finviz screener they can detect pattern like Head and Shoulders, Trendline Support, Wedge,... and When I click on a ticker a chart shows up with pattern's lines in there as well. Like this

I've been searching for a way to do the same thing for awhile but there are no clear clues yet.
I don't think Finviz doing this manually since there are too many stocks.
I'm not asking for exact algorithms but If anyone knows their approaches to this matter please share your thought

15 responses

Looking at a stock curve as humans we recognize local highs and lows instantly. Anyone know how that is done in code?

@Olive Thank for the response. Did you come up with this approach or is it a common way ? I've heard about techniques like machine learning (image recognition). Good thought though!

You definitely have to look at this https://www.quantopian.com/posts/an-empirical-algorithmic-evaluation-of-technical-analysis

There you'll find the code to detect any technical patter that can be defined as a sequence of local maximum and minimum. Then, if additional constraints are required, you can refine your patter matching algorithm on top of the identified patterns.

Also, make sure to have a look at the already existing technical patterns in pipeline: https://github.com/quantopian/zipline/blob/master/zipline/pipeline/factors/technical.py

I recently wrote an algorithm that uses an extremely dumb method to detect horizontal support and resistance levels within a short (90 minute) window, which confirmed that this is a very reliable signal when used with medium-volatility, low-volume stocks (penny stocks). Encouraged by those results, I tried to build on the concept with a new script that on the fly keeps track of multiple levels of horizontal support while not limiting the time window, hoping it would work for stocks with slower momentum. Unfortunately, that version of the script seemed to almost exclusively enter losing trades so I gave up on it. :(

What I would do to find non-horizontal support and resistance trendlines I would loop through the history window you want to take into consideration, keeping track of each peak and valley and their corresponding time offset keys (so you collect sets of x-y pairs for each). To identify the peaks I simply compare -- is the current price greater than most recent entry for the peak? if so update it. If it dips below some threshold, add a new entry and start keeping track of a new max value. Visa versa for the valleys. Then run those x-y pairs through numpy's linear regression library (np.linalg.lstsq) and you've got your trendlines.

The problem with trendlines is that they're super open to interpretation. Looking at that image you posted, you could draw those lines in a dozen different ways that would make equally as much sense (and probably all be equally as unhelpful).

I took a quick shot at writing a method to detect a very simple head and shoulders pattern, but probably I made some mistake in my logic or wasn't using a large enough window or correct thresholds since none of the stocks I'm throwing at it get through. Let me know if you get this to produce anything useful. Perhaps it would be better to use this logic in a pipeline screen... I just don't know my way around pipeline well enough yet.

def head_n_shoulders(hist):  
    support_threshold = 0.004  
    shoulder_threshold = 0.01  
    head_threshold = 0.004  
    support = hist[-1]  
    shoulder1 = 0  
    support2 = 0  
    head = 0  
    support3 = 0  
    shoulder2 = 0  
    support4 = 0  
    # Work backwards to identify head and shoulders pattern  
    for d in range(0,len(hist)):  
        price = hist[-1-d]  
        # Breaks pattern if price dips below theoretical support  
        if price < support * (1-support_threshold):  
            return False  
        # Start by finding first shoulder level,  
        # must exceed specified threshold above support level  
        elif not support2 \  
        and price > support * (1+shoulder_threshold) \  
        and price > shoulder1:  
            shoulder1 = price  
        # Descending from first shoulder, retests support level  
        # (record lowest level reached within threshold)  
        elif shoulder1 and not head \  
        and price < support * (1+support_threshold) \  
        and price > support * (1-support_threshold) \  
        and (not support2 or price < support2):  
            support2 = price  
        # Ascending again past height of previous shoulder,  
        # record highest head level  
        elif support2 and not support3 \  
        and price > shoulder1 * (1+head_threshold) \  
        and price > head:  
            head = price  
        # Retest support level again  
        elif head and not shoulder2 \  
        and price < support * (1+support_threshold) \  
        and price > support * (1-support_threshold) \  
        and (not support3 or price < support3):  
            support3 = price  
        # Breaks pattern if next shoulder exceeds head  
        elif support3 and not support4 \  
        and price > head:  
            return False  
        # Instead it just needs to exceed threshold above support level again  
        elif support3 and not support4 \  
        and price > support * (1+shoulder_threshold) \  
        and price > shoulder2:  
            shoulder2 = price  
        # Pattern started by entering in from below support level  
        elif shoulder2 \  
        and price < support * (1-head_threshold):  
            return support - (head - price) # Price target?  
        # End of the line, nothing found...  
        elif d == len(hist):  
            return False  

I took a crack at coding up the trendlines idea I described earlier...

Basically, since you can't look back in time, the line you see in the graph is the end point at that moment in time of the constantly evolving trend lines. It turned out way sloppier than I thought it would. The logic for identifying and keeping track of peaks and valleys definitely could be improved. Then there's the problem that often the support is above resistance, which probably indicates something is really wrong with the code.

Anyways, I used the trend lines to employ a very simple strategy: buy when at support, sell when at resistance.

Thank all for sharing your knowledge !
It's gonna take me sometime to go though these suggestions
I will definitely come back to share the result
Have a nice weekend!

You might also start by checking out Talib. It has candlestick pattern detection and a whole slew of indicators typically used by stock charting software.

@Viridian Hawk thanks for the advice.
I tried that one. It's very good one except for candle's patterns go with market trend like Hammer, Shooting Star... I can fix that by defining market down/up trend and then combine with Talib's result or rewrite the whole thing.
Do you have any suggestion for defining market trend ?
By the way I'm currently working to build and test a commercial screener so I'm looking for fast and accurate solutions.
I'll start testing those solution next Monday!

I added a pipeline to the trendlines algorithm and just fed the Q500 into it without using a filter, not sure what type of filter would work best with this algorithm.

@Eric, cool. I just coded this up just to see what would happen. Didn't have much faith in it. I'm not sure how to trade off of trendlines anyways. Perhaps in conjunction with other signals? Also, probably my logic is probably deeply flawed. Maybe it could be tweaked into something useful.

Anyways, there were a few things that needed to be fixed: I fixed it to keep track of different highs and lows for each stock instead of one context array holding the points for all of them. Also, made sure the context.x-axis (time axis) only gets incremented once a day, instead of with each stock per day. I also removed the last peak and valley points since they are still in process of being established. Interesting thing is that none of those fixes seem to have had much of any effect on the outcome of the backtest. If anything it's worse.

Basically I figured that adding the pipeline to this algorithm might allow it to generate alpha with multiple stocks because that worked with the other algorithm. I cut and pasted a few lines from the other algorithm and got the pipeline version to buy and sell multiple stocks, although the custom graph was messed up. I didn't understand the code well enough to make the other fixes. I'm also surprised that the updated version isn't performing better. Will test the updated version and see if I can get it to generate more alpha.

Found one source of alpha although I don't get why it works. In the rebalance function the x-axis is incremented once per day using context.x += 1. I ran backtests with larger numbers over the same timeframe and the algorithm was generating 0.02-0.03 alpha.

The profit per dollar on the EN version is over twice the VH ver and both are higher than their chart because they used only 29 and 61 percent of the initial capital respectively. EN is 22.5% in profit per dollar invested instead of 6.5%, with a leverage max of .63.

@Blue yeah, I think that says something when an algorithm that is doing something totally random produces better results than the properly configured version. :) I just wrote it up to test the idea, and posted it as a starting point in case anybody would find it useful. I think Olive Coyote's advice about determining pivot points is probably a better route than my method of using linear regression of peaks and valleys -- but maybe it could be improved and made into something useful.
Since starting working with Quantopian though I've become a hardcore skeptic of technical indicators. I think they're generally a load of hogwash. For every cherry-picked example of a chart where it works as advertised, look anywhere else on the chart and it falls apart.

It depends on the period for the backtest.
I just copied and you can see the result. But I am very interested in an improved solution.