This may help.

Wonderful effort Q. I applaud the vast array of interesting papers you guys have published.

Despite my deep scepticism I will certainly contribute to this once I have the time.

I believe an Event Study could give us much more information regarding those technical patterns. So I added an event study at the bottom of your NB. With this information it should be easier to build an algorithm as we can better decide when to enter/exit our positions once we detect a pattern.

P.s. Andrew, your posts are always interesting, thanks for your work.

I was puzzling yesterday about currency conversion and back testing on patterns.

On trend following I have decided the "correct" way to deal with a multi currency Portfolio is to convert all time series to the base currency prior to running my daily orders (and of course for any backtesting). In my view there is little point in declaring an entry into the yen version if the Nikkei if you are a dollar investor - you need to take the currency into account to check the trade still makes sense I'm dollars.

Many backtesting programs work with native currencies and only convert the P&L back to the base currency.

While my method is fine for long term trends, I suspect it would create havoc with other types of pattern such as those being considered here.

you need to take the currency into account to check the trade still makes sense I'm dollars.

This is offtopic but you could also hedge the position on FX market so then index returns would then basically be in USD. If you trade yen based nikkei equity without hedging you are actually trading both the equity and yen at the same time (this might not be a problem at all but it's worth noting).

Mikko M
Agreed but in trend following that is exactly what I think one should do - trade the trend in the Nikkei as that trend is modified by the currency. There are two other alternatives - take signals just on the Nikkei trend and ignore the currency except in calculating the daily P&L, or hedge as you suggest.

There are very few hedged ETFs around - most trade in one currency (eg the USD) with an un-hedged investment in yen assets. The market ensures the USD price remains in line both with the movement of the Nikkei and the USD/JPY.

I don't have much difficulty with any of these approaches but in other forms of pattern recognition, I suspect logically a dollar investor would be best to base his signals on the unconverted JPY time series. There again I have done no testing whatsoever and am just raising a "note to myself" for when I come to look at this later.

If there is any validity to a "head and shoulders" (if it has any predictive power) I imagine that may disappear if you convert the time series to USD.

In trend following I don't mind doing that. I'm an investors rather than a trader and if an up-trend in the Nikkei is outweighed by a down trend in the yen, I don't want to take the trade.

Incidentally I don't hedge. I want the diversification.

@Luca That event study is a great addition to this analysis. Thank you for sharing!

Some takeaways:

1) I'd want the algo to trade on each of these technical signals. A larger "N" is going to lead to smaller error bars on our overall forward return.

2) Most of these signals are significantly decayed after 6 days. (I'm still concerned that the kernel regression has introduced lookahead bias). I'd be interested to see how many of these signals would have been missed if we provided only {1,2,3,4,5,6,7} days of trailing price data after the last local maximums identified with the full range of available data.

3) Inverse Head-and-Shoulders appears to be a particularly useful signal. It occurs often and has a large, slow decaying forward return.

Do you notice anything in particular?

@Andrew, here is an algorithm that uses those patterns.

There is a pipeline factor (PatternFactor) that detect those patterns and return a code (an integer) any time a pattern is found.

You can see the factor has a class member "indentification_lag" that is a limit on how many days ago a pattern can be recognized: ideally we would like to recognize a pattern the day after his last local minimum/maximum is found, but in practice we need more days to detect the last minimum/maximum. The "indentification_lag" parameter is there to let us test different configuration.

After changing "indentification_lag" we are suppose to update the number of days a position is held accordingly: the latter we detect a pattern the less days we can hold a position. The number of holding days per event can be configured in before_trading_start function.

This backtest used an indentification_lag of 4 days and a holding period 2 days (as Andrew correctly reported "Most of these signals are significantly decayed after 6 days" )

I'm glad we looked at this in an algo. Identifying the last maxima/minima in a pattern is clearly a key limitation. It'd be interesting to run these tests on various kernel regressor bandwidth parameter values to see if there is an identification lag/smoothness combination that produces superior returns.

I wonder if Kalman filter smoothing would look any different? I'll try swapping one into the notebook.

Here is a factor tear sheet of PatterFactor (the pipeline factor used in the previous algorithms to identify the technical patterns). There is a tear sheet for identification lags of 1, 2 and 3 days. I expect the results to be similar to what we found in the event study (even though in this NB we analyse a shorter time period 2013 - 2016, for technical limitation of research environment)

Edit: updated NB

Summarizing what we have done up to know....

• The event study shows the technical patterns have some signal that we can exploit

• The signal decays after ~6 days, though Andrew is concern that the kernel regression introduces a lookahead bias: indeed we cannot predict a local minimum or maximum without having collected some data(daily close price) AFTER the local minimum/maximum. This means we cannot use the full 6 days signal but possibly less days of it and would be interested to know how many.

• Created a pipeline factor, PatternFactor, that detects the same patterns analysed in the event study. PatternFactor is configurable in the number of identification lag days to detect a local minimum/maximum. Obviously the less days we configure the less local minimum/maximum it finds and also, less days could mean detect false positive.

• Created an algorithm with PatternFactor and experimented with identification lag of 1,2,3,4 and 5 days (more is pointless as the signal decays after 6 days)

• The algorithm is not profitable, this can be due to bugs in the code :) or Andrew's concern is well founded. To understand the problem we created a factor tear sheet for PatternFactor

• Studied PatternFactor with the Factor Tear Sheet and discovered that if we don't give enough identification lag days to the factor it is not able to detect
  the same patterns we studied in the event study. Meaning it detect local maximum/minimum that are false positive (they wouldn't have been detected as local maximum/minimum if we had provided more data)

The last conclusion was not clear simply looking at the last two factor tear sheets posted above. So here is a new NB that shows PatternFactor with 1,2,3 and 10 days identification lag (I never tried 10 days lag before ). In this last NB I also filter out stocks whose price is less than 5 dollars because the signal returns are very low and the transaction cost would make those stocks untradable.

My opinion is that with identification lag of 10 days the factor is able to detect patterns very similar to the ones seen in the event study. There are little discrepancy but we have to keep in mind that we are not using the same universe in the tear sheet and in the event study (as the pipeline recalculate its stock universe every day while the event study has a fixed universe for the whole time period examined).

If we look at the results of PatternFactor with 1,2 or 3 days identification lag we can see the recognized patterns are not similar to the ones seen in the
event study. In particular, the more lag days we have the more similar the patterns are to the ones in the event study. In my opinion that should corroborate Andrew's thesis/fear of lookahead bias.

It would be nice to have my conclusion reviewed by someone else, as it is very likely that I missed something (and we can never exclude bugs presence in the code).

Another thing I noticed is that PatternFactor with identification lag of 1 day, while detecting different patterns than the event study, generates a signal that last 1 day. Is not very strong but we could try exploiting it.

The algorithm has few shortcomings.... but just to show the signal generated from identification lag of 1 day.

Thanks all for contributing to this post but I have one question:
What look ahead bias is being referred to? That is, Isn't the period used for the kernel regression plus lag to observe the signal roughly equal to the time that a trader would take to identify the various patterns?

The look ahead bias is in the Andres'NB and in the event study NB (not in the algorithm and the factor tear sheet) and it is due to the kernel regression being run for the full time period in one shot. In reality you would have to run the regression every day instead, like a rolling kernel regression on the past 30 days of data or so. Why this matter? Because the kernel regression doesn't give you good smoothed values at the end of a time series, there isn't just enough data to do it. This means that in reality you cannot trust kernel regression in the proximity of the current date, as you run a regression every day, and the last local minimum/maximum needed to identify a pattern might become evident only some days later. By the time a pattern is identified there might still be some alpha left or not, but to understand that we should rewrite the NB code so that it performs the pattern identification using a rolling kernel regression to simulate live trading behaviour, differently we cannot trust the results as they are biased.

Thanks Luca....I guess I assumed that the NB was replicating the paper exactly. So since the authors were using rolling regressions, they had no look ahead bias as Andrew implied at the beginning of this post:

As I discuss in the notebook, one of my concerns with the author's methodology is the introduction of lookahead bias via the kernel regression...

Well, In his NB Andrew takes into account the detection lag when he calculates the returns. He assumes a 4 days identification lag that decrease the hypothetical returns. But still the NB might be considering patterns that wouldn't have been recognized in real trading, with a rolling kernel regression
It's not that hard to modify the code so that it uses a rolling kernel regression though.

For all the experts on this forum, is the paper by Lo on this subject dated? (do not mean disrespect for the paper here). Are there there any best practices/standard libraries in identifying these basic technical patterns since it appears that most of the financial investment platforms offer screening for these patterns such as finviz.com and interactive broker(?). Thanks.

Hi Andrew, I am not able to run the notebook and it has problems at cell # 9. The error is:

raise ValueError('Cannot index with multidimensional key')

KeyErrorTraceback (most recent call last)
in ()
25
26 plot_window(msft_prices, smooth_prices, local_max_dt, local_min_dt, price_local_max_dt,
---> 27 price_local_min_dt, pd.datetime(2005,2,19), pd.datetime(2006,2,26))
28 plot_window(msft_prices, smooth_prices, local_max_dt, local_min_dt, price_local_max_dt,
29 price_local_min_dt,pd.datetime(2007,9,1), pd.datetime(2008,9,1))

in plot_window(prices, smooth_prices, smooth_maxima_dt, smooth_minima_dt, price_maxima_dt, price_minima_dt, start, end, ax)
13 smooth_prices_.plot(ax=ax)
14
---> 15 smooth_max = smooth_prices_.loc[smooth_maxima_dt]
16 smooth_min = smooth_prices_.loc[smooth_minima_dt]
17 price_max = prices_.loc[price_maxima_dt]

/usr/local/lib/python2.7/dist-packages/pandas/core/indexing.pyc in getitem(self, key) 1294 return self._getitem_tuple(key)
1295 else:
-> 1296 return self._getitem_axis(key, axis=0)
1297
1298 def _getitem_axis(self, key, axis=0):

/usr/local/lib/python2.7/dist-packages/pandas/core/indexing.pyc in _getitem_axis(self, key, axis) 1454 raise ValueError('Cannot index with multidimensional key')
1455
-> 1456 return self._getitem_iterable(key, axis=axis)
1457
1458 # nested tuple slicing

/usr/local/lib/python2.7/dist-packages/pandas/core/indexing.pyc in _getitem_iterable(self, key, axis) 1020 def _getitem_iterable(self, key, axis=0):
1021 if self._should_validate_iterable(axis):
-> 1022 self._has_valid_type(key, axis)
1023
1024 labels = self.obj._get_axis(axis)

/usr/local/lib/python2.7/dist-packages/pandas/core/indexing.pyc in _has_valid_type(self, key, axis) 1377
1378 raise KeyError("None of [%s] are in the [%s]" %
-> 1379 (key, self.obj._get_axis_name(axis)))
1380
1381 return True

KeyError: "None of [['2003-01-13T00:00:00.000000000' '2003-02-20T00:00:00.000000000'\n '2003-03-20T00:00:00.000000000' '2003-04-04T00:00:00.000000000'\n '2003-05-08T00:00:00.000000000' '2003-06-02T00:00:00.000000000'\n '2003-06-19T00:00:00.000000000' '2003-07-09T00:00:00.000000000'\n '2003-07-28T00:00:00.000000000' '2003-09-08T00:00:00.000000000'\n '2003-09-19T00:00:00.000000000' '2003-10-07T00:00:00.000000000'\n '2003-10-17T00:00:00.000000000' '2003-12-19T0

Amir Vahid,
Change the code

local_max_dt = smooth_prices.iloc[local_max].index.values  
local_min_dt = smooth_prices.iloc[local_min].index.values  

to

local_max_dt = smooth_prices.iloc[local_max].index  
local_min_dt = smooth_prices.iloc[local_min].index  

Here is the Alphalens event study NB

EDIT: cleaned up the NB a bit

Luca: This is excellent! A couple of observations:

• Really cool to also see what the stock is doing before the event which gives one a visual on what the technical analysis is latching onto
  
• Holy cow, some of these seem to actually do something meaningful
  
• Interesting to see how some that are bearish (RTOP and TTOP) seem to have the right negative short-term alpha but then long-term positive alpha (thanks for including 11D as one can see that it's where a lot of long-term decays again).

Have you attempted any type of signal combination?

@Thomas, there is my old algorithm few posts above but please note that it's not straightforward to decide how to best trade all those signals together. Depending on the detected pattern we want to hold the securities for different number of days. Also, since some of the patterns perform best after more than one day, when we detect some signals we don't know if it is better to trade all available cash on those or to keep some cash for following days. Either we risk to have an underleveraged portfolio or we don't trade all the signals because we used all the cash. The trading logic is quite complex.

Please install the fastdtw on the research notebook.

Out of curiosity, I tried the TBOT signal alone

Guys, thank you so much for your effort.

Did you test ta-lib candlestick patterns as well?

Thank you all for so much effort. Such an interresting work.

@Luca
I am trying to play arround with your NB(cleaned up version) , Just tryng to analyze the alphalens with 4 quantiles and change the universe to 'QTradableStocksUS'. But the algo seems to be unending!!

I ran it for more than 4 hours and tried to do so 5 times but i can t see an end of it.
Was it taking that long when you ran it ??

NB: kept the other variable Identical:
factor_name = 'factor'

start_date = '2014-08-01'
end_date = '2016-08-01'
top_liquid = 'QTradableStocksUS'
filter_universe = True # very slow, filter out untradable stocks
show_sector_plots = False # very slow to load the sector column in pipeline

alphalens specific

periods = (1, 2, 3, 4, 5, 6, 10)
quantiles = 4
bins = None
avgretplot = (10, 25) # use None to avoid plotting or (days_before, days_after)
filter_zscore = None
long_short = False

prices_cache = None # this saves lots of time when running tear sheet multiple times

@Moussa,

You may be running into too much ‘memory utilisation’?

Try clearing memory by killing all active NBs. Then maybe try a shorter time period, and/or fewer periods (eg only 1, 5, 10 perhaps before honing in?).

@Joakim

Thank you a lot for taking the time to help.

I did kill all active NB, it was the only one running . However i will try for fewer periods but @ luca did succeed for 10 periods so i do not understand.
Even when i try to run the NB with no changes, the same thing happens.

Did you succeed with running it yourself?

Having the same problems. It will run for about 1% of total backtest, and then it just loads forever and never gets anywhere. Based on the time that is took for it to get to 1% completion, I can only imagine that it must've taken several days for @Luca to compile the algorithm.

Wondering if anyone has been able to put the patterns into Custom Factors?

Thank you very much for your all contribution. It seems like the long-strategies are generally more profitable than the shorting one. I wonder if this is because, in the long-term, the stock market would generally go up?

Guys, there is an error on HS and IHS rules:

(abs(e2 - e4) <= 0.03*np.mean([e1,e5]))

should be

(abs(e2 - e4) <= 0.03*np.mean([e2,e4]))

!!!

How do you parameterize the relative magnitude of the patterns?

Mannn, Im going through this place in 2020 and lot of things have changed

Change this

kr = KernelReg([prices_.values], [prices_.index.values], var_type='c', bw=[1.8,1])  

to
kr = KernelReg([prices_.values], [prices_.index.values], var_type='c', bw=np.full((1, 1), 1.8))

because new pandas don't support the old one. It was hard time for finding this fix, will update if i find something more

You can refer the updated version of this here
https://github.com/rushic24/quantopian-offline-2020/blob/master/An%20Empirical%20Algorithmic%20Evaluation%20of%20Technical%20Analysis.ipynb

Hello guys, I have updated the notebook to python 3, here is the attachment.