Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Advance-Decline Statistics

Hi all!

I'm trying to model and advance-decline line for a universe of stocks. As far as I can tell, there are two ways to do this. One would be setting my universe to say, the top 100 stocks in the NYSE and pulling each individually to see if it was an advance or decline. On the other hand, I could also use fetcher and pull in a CSV that contains a daily NYSE advance-decline spread. That said, I'd rather do it all programatically if possible. Is there a function somewhere which already calculates this information? Or is it better to do it the quick and dirty way of just pulling from outside sources?

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

11 responses

Kathryn,
I tossed together an example, it sounds like this is doable without using the fetcher. I used the dollarVolumeUniverse, but you might want to keep the universe static for this sort of thing (not really sure). I used wikipedia's definition of advance-decline, I hope that's the same thing you're referring to.

David

Perfect! Thank you! I hadn't thought to do them all simultaneously as array math. I'm actually trying to build the McClellan Summation (summed McClellan Oscillators), which needs the A-D as one of the underlying pieces. I thought I had it working, but it breaks on the line

RANA = (advances - declines) / (advances + declines)

with a ZeroDivisionError. Given that advances and declines should never both be zero, and when I print them never are, I'm baffled as to how this is happening.

can declines be negative?

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

You can set a breakpoint by clicking in the left hand gutter, and then inspect the variables to get to the bottom of it.

Thanks and Kathryn & Dave.

incCount = float(len(advances))  
decCount = float(len(declines))  
RANA = (incCount - decCount) / (incCount + decCount)  
import talib  
import numpy

PeriodsToConsider = 63

# Put any initialization logic here.  The context object will be passed to  
# the other methods in your algorithm.  
def initialize(context):  
    set_universe(universe.DollarVolumeUniverse(floor_percentile=99.0, ceiling_percentile=100.0))  
    context.target = symbol('SPY')  
    context.ADs = []  
    context.RollingADs = []  
    context.lookback = 60  
    schedule_function(func = HandleDataScheduled, date_rule=date_rules.week_start(days_offset=0))  
# Will be called on every trade event for the securities you specify.  
def handle_data(context, data):  
    prices = history(2, '1d', 'price')  
    change = prices.diff().iloc[-1]  
    advances = change[change > 0]  
    declines = change[change < 0]  
    context.ADs.append(float(len(advances) - len(declines)))  
    context.ADs = context.ADs[-PeriodsToConsider:]  
    ad = sum(context.ADs)  
    context.RollingADs.append(ad)  
    context.RollingADs = context.RollingADs[-PeriodsToConsider:]  
def HandleDataScheduled(context, data):  
    ad = context.RollingADs[-1]  
    adMA1 = talib.EMA(numpy.asarray(context.RollingADs), timeperiod=PeriodsToConsider/3)[-1]  
    adMA2 = talib.EMA(numpy.asarray(context.RollingADs), timeperiod=PeriodsToConsider)[-1]  
    if (adMA1 > adMA2):  
        order_target_percent(context.target, 1.0)  
        #for stock in data:  
        #    order_target_percent(stock, 1.0/float(len(data)))  
    else:  
        order_target_percent(context.target, 0)  
        #for stock in data:  
        #    order_target_percent(stock, 0.0)  
    record(AD=ad, ADMA1=adMA1, ADMA2=adMA2)  

Well that's pretty nifty. I tried integrating that little snippet to make it a float. No dice. If I run it without the "RANA= " part, it prints out advances and declines starting at 39, and never hits the if statement. As far as I can tell, there aren't any actual zeros. And certainly none that should cause build errors, rather than runtime errors.

Here's the part up until where it breaks.

# Will be called on every trade event for the securities you specify.  
def handle_data(context, data):  
    prices = history(2, '1d', 'price')  
    change = prices.diff().iloc[-1]  
    advances = change[change > 0]  
    declines = change[change < 0]  
    incCount = float(len(advances))  
    decCount = float(len(declines))  

    print "A: %s" % incCount  
    print "D: %s" % decCount  
    if (incCount + decCount) == 0:  
        print "Zero"  
    context.ADs.append(incCount - decCount)  
    #Ratio Adjusted Net Advances (RANA): (Advances - Declines)/(Advances + Declines)  
    #RANA = (advances - declines) / (advances + declines)  
    RANA = (incCount - decCount) / (incCount + decCount)  

Adding a simple test can protect against /0. Although 0 should never be possible, I guess the Python compiler felt differently.

def handle_data(context, data):  
    prices = history(2, '1d', 'price')  
    change = prices.diff().iloc[-1]  
    advances = change[change > 0]  
    declines = change[change < 0]  
    incCount = float(len(advances))  
    decCount = float(len(declines))  
    if (incCount + decCount) == 0:  
        print "Zero"  
    context.ADs.append(incCount - decCount)  
    #Ratio Adjusted Net Advances (RANA): (Advances - Declines)/(Advances + Declines)  
    if ((incCount + decCount) != 0):  
        RANA = (incCount - decCount) / (incCount + decCount)  
    else:  
        RANA = 0.0  
    record(RANA=RANA)  

Figured it out! There is apparently a two day period when AAPL was the same. In my code, I only had conditions for increasing or decreasing, not counting on anything ever staying the same.

advances = change[change > 0]  
declines = change[change < 0]  

If I fix it to include that case, it runs without error. I'll be posting the whole thing soon!

advances = change[change > 0]  
declines = change[change <= 0]  

@Kathryn,

I'm not sure you're getting what you think you are getting. If you examine the advances (a 'pandas.core.series.Series') and the declines (another series) and then subtracting those two series from one another, you get yet another series containing all NaNs.

I'm pretty sure that it's the count of the advances and the count of the declines that you want to work with. The actual delta values within each series are of no concern. Of course, if you want to actually use the values you'd want to create either % returns or log returns and sum those. That might be an interesting treatment of the A/D but would be considerably different than using just the counts. See the contents of these series (by debugging your code) below:

advances  
 advances: Series  
0_Security(62 [ABT]): 0.157  
1_Security(239 [AIG]): 34.346  
2_Security(337 [AMAT]): 0.9  
3_Security(357 [TWX]): 0.99  
4_Security(368 [AMGN]): 1.04  
5_Security(679 [AXP]): 1.138  
...

declines  
 declines: Series  
0_Security(2602 [EA]): -1.28  
1_Security(3014 [FRX]): -0.255  
2_Security(4313 [KSS]): -0.74  
3_Security(4521 [LOW]): -0.025  
4_Security(4954 [MO]): -0.062  
5_Security(5343 [THC]): -0.64  
...

advances - declines  
 advances - declines: Series  
0_Security(62 [ABT]): nan  
1_Security(239 [AIG]): nan  
2_Security(337 [AMAT]): nan  
3_Security(357 [TWX]): nan  
4_Security(368 [AMGN]): nan  
5_Security(679 [AXP]): nan  

I'd think that would be fixed by both the snippet you posted and the snippet that David originally posted which does a len(advances ) - len(declines). The full code that I was using is actually the following. I was just showing the section that broke it for brevity. Sorry to be confusing!

    prices = history(2, '1d', 'price')  
    change = prices.diff().iloc[-1]  
    advances = change[change > 0]  
    #put a <= here in case the price was the same 2 days in a row  
    declines = change[change <= 0]  
    incCount = float(len(advances))  
    decCount = float(len(declines))  
    context.ADs.append(incCount - decCount)  
    #Ratio Adjusted Net Advances (RANA): (Advances - Declines)/(Advances + Declines)  
    RANA = (incCount - decCount) / (incCount + decCount)  * 1000  

So, here's the question: is there a group of securities, as determined by some filtering of fundamental data, that has nearly leading or at least non-lagging behavior when said securities are applied as an Advance/Decline metric?

The below test begins the search. In this test securities which have a greater than 20 P/E and a market cap greater than 20B are used as the dynamic universe.
But the question is, can there be discovered a different filter criteria set which provides for a better fit to predicting the general market's actual up or down trend?
If such a mechanism can be found to find such a filter criteria, then perhaps other group based indicators could be tuned to fit specific yet dynamic groups of securities.

Such a mechanism would be of considerable value as it would auto-correct (hopefully) as the components come and go.

def before_trading_start(context):  
    fundamental_df = get_fundamentals(  
        query(  
            fundamentals.valuation_ratios.pe_ratio,  
            fundamentals.valuation.market_cap)  
        .filter(fundamentals.valuation_ratios.pe_ratio > 20,  
                fundamentals.valuation.market_cap > 20000000000)  
        .order_by(fundamentals.valuation.market_cap)  
        .limit(50))  
    update_universe(fundamental_df.columns.values)