Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Detecting high and low programmatically

My compilation of indicators (then percentized/normalized 0-100) correlelates ok with some ideal buy/sell points. The next step is trying to decide when high and low points have probably been reached (a trailing stop helps on sell).

Peaks/valleys are simple for us visually because we're seeing the future on the chart however cannot know hi/lo at the time it happens in code, seems difficult at best if not impossible.
So the idea would be to minimize the time gone by in looking back and that's ok.

Has anyone tackled this problem with a fair bit of success?

7 responses

This is the million dollar question. If you use a level trigger for an "oscillator", you'll be red-lining all the way against a trend, if you use a fast-slow crossover, you'll be a few weeks late for every reversal and get chopped up. These sorts of indicators always seem so promising to the eyeball, since they are so clearly coincident with major turning points, but are (in my opinion) a major rabbit hole to try to automate, because essentially they contain zero to very little actual predictive value.

This is one of the angles that I have been thinking about but haven't had a lot of chance to get it working yet. My way of thinking is to catch slope changes. For instance, a stock goes up from $40 to $42 and then down to $41, then you have a slope of +2 followed by a -1. At $41 that's when you sell, and yes it lags a little bit, but hopefully you're out not too far from the peak. Of course, you'll have to tweak the time periods for slope calculations as a single period slope will give you a whole bunch of noises. Hope this helps.

How would your algorithm work to detect a long-term maximum before the price has already gone down too far? Remember that in real life you don't know the data from the future.

Remember that in any given period of time you'll see a bunch of minor fluctuations up and down, and you have to filter those out. They are also fairly random.

Thank you for the replies.
@Simon: By "fast-slow crossover" were you talking about what I'm addressing below or does it have the potential to be different?
@Felix: Below I'm going to paste in a slope calculation in case you or someone might want to play around with it.
@JS: I'd say once you have a lot of experience, those things go without saying.

I have a lot of code that goes into the compilation of indicators (in an 1800 line algo overall).
The naysayer side of me says it is way too complex and simple is better, code flies when you're having fun.

Considering again that the indicator is my own combined collection of other indicators,
a question now, rather than counting on thresholds...

What if I were to reproduce that indicator in the chart with lookback window numbers slightly off from the original, and then use their two curves (compilations) in MACD style (two lookback window time periods), I'm wondering if anyone has any prior experience with that as a principle (aside from MACD) and whether it can usually be counted on. (Seems to me for example two simple moving averages, SMA, like 10 and 11 will make for slightly shifted curves and hi/lo--**ish** crossovers independent of thresholds if I'm not mistaken).

And do you think there is a reasonable chance then (enough to try it) that my custom indicator crossover points might be fairly close to the existing peaks and valleys seen in the chart above?

Thanks

Here's the slope calculation I mentioned,
I don't currently use it a whole lot
and don't recall what I meant by the "??? Fix this" note.
Can be improved surely.

def slope_calc(in_list):  
    ''' Linear Regression to obtain slope  
          (straight line ideal representation of a collection of data points)

         In: List of numbers (can be as strings)  
        Out: Number, slope  
        A slope of 1 is 45 degrees, -1 is -45 degrees? [True?]  
            however 90 degrees is infinity. [I might have made changes since that note was made.]  
        It is rise over run.

        x - x axis (one each day, minute or whatever)  
        y - y axis (macd values or prices etc)  
        m - slope of the line  
        a - point on the y axis where the line intercects it (not used here)

        http://easycalculation.com/statistics/learn-regression.php  
        Regression Equation(y) = a + (b * x)  
        Slope     m = ((n * sum(x * y)) - (sum(x) * sum(y))) /  
                            (n * sum(x^2) - (sum(x))^2)  
        Intercept a = (sum(y) - (m * (sum(x)))) / n  
    '''  
    if len(in_list) == 1:  
        return 0  
    n       = len(in_list)         # like 10  
    sum_x   = 0  
    sum_y   = 0  
    sum_x_y = 0  
    sum_x_2 = 0

    # Collect sums  
    for i in range(n):      # 0-based, 0-9 for list of length 10  
        x        = i + 1    # 1-based  
        y        = float(in_list[i])  
        sum_x   += x  
        sum_y   += y  
        sum_x_y += x * y  
        sum_x_2 += x**2

    # Calculate slope  
    that = ( n * sum_x_2 - (sum_x**2) ) # ??? Fix this, tmp avoid divide by zero  
    if that != 0:  
        slope = ((n * sum_x_y) - (sum_x * sum_y)) / that  
    else:  
        slope = 0

    return slope * 100  # pretend these are like angles

Given the image that you've posted, I can't think of any way to convert that to trading signals near the points that you would like.

EDIT: though, if you are going to attempt some sort of MACD signal based on it, I'd first try it on the un-normalized version of the indicator, rather than the 0-100 one, which suffers from being pegged at the extremes while the range estimator catches up to the signal.

I apply indicators to prices as usual, then add the results of them, then normalize that sum.
The effect might be thought of as sort of a voting system, when they all agree, that's when the hi/lo points show up.
In the process can also easily give one or more of them more weight with simple multipliers before the sumation.

Gary - using regression is actually a good idea. Just dug out my code and here's part of it.

    m7_bar = price_history[context.stock][-7]  
    m4_bar = price_history[context.stock][-4]  
    m3_bar = price_history[context.stock][-3]  
    m2_bar = price_history[context.stock][-2]  
    m1_bar = price_history[context.stock][-1]  
    down_slope = (m7_bar - m3_bar) / 4  
    up_slope = (m1_bar - m3_bar) / 2  
    buy_signal = (down_slope > 0) & (up_slope > 0) & (down_slope * 2.0 < up_slope)  

It essentially uses a few fixed points (-1, -3, -7 ticks) to calculate 2 straight line slopes - if it works it's perfect, but if something crazy happens between these ticks it may have some unintended consequences. And the trigger I used here is that if the slope of the bounce is more than 2 times the downward slope, then it's a buy.

On whether the MACD style slope calculation (I'm guessing that's what you meant), you might just want to try it out and see how it goes.