Something like this may be what you're looking to do?

class Mean_Diff_Return_From_SPY(CustomFactor):

    returns = Returns(window_length=2)  
    returns_spy = returns[symbols('SPY')]

    inputs = [returns, returns_spy]

    # Set window-length to number of days returns to average  
    window_length = 2  


    def compute(self, today, assets, out, returns, returns_spy):  
        out[:] =  (returns - returns_spy).mean(axis=0)   

One issue is that you can't really average arithmetic returns. You should really be using log returns. There may not be a difference if you're just looking for top returns or some like that. I've attached a notebook with both the above custom factor and a similar log return implementation.

Fantastic, so many gems here!

What is the easiest way to include SPY in the mask? Usually the mask would be something like Q500US() that does not include SPY?

To add a specific security to a filter you can use the | function. Note that OR does not work. You will need to use the | operand. The following should work.

my_universe = Q500US() | StaticAssets(symbol('SPY'))  

@ Vladimir
I noticed you had originally wanted to average only return differences which are positive. Is that still the case. The custom factor above averages all returns. If so, I noticed in your example you divided by 3. Would you really want to divide by 2?

I wanted to divide by total count, but it would be useful to have a snippet that would return only the average of differences that are greater than zero.

To only sum the differences greater than 0 you could set any values less than 0 to 0.

class Mean_Positive_Diff_Return_From_SPY_Counting_0(CustomFactor):

    returns = Returns(window_length=2)  
    returns_spy = returns[symbols('SPY')]

    inputs = [returns, returns_spy]

    # Set window-length to number of days returns to average  
    window_length = 2  


    def compute(self, today, assets, out, returns, returns_spy):  
        return_diff = returns - returns_spy

        # Set any negative values to zero. They WILL get counted but not add to the average.  
        return_diff[return_diff < 0] = 0  
        out[:] =  return_diff.mean(axis=0)

To sum only the differences greater than 0 but then NOT include those in the average, you could do something like this. Note the numpy nanmean method excludes NaNs from the mean.

class Mean_Positive_Diff_Return_From_SPY(CustomFactor):

    returns = Returns(window_length=2)  
    returns_spy = returns[symbols('SPY')]

    inputs = [returns, returns_spy]

    # Set window-length to number of days returns to average  
    window_length = 2  


    def compute(self, today, assets, out, returns, returns_spy):  
        return_diff = returns - returns_spy

        # Set any negative values to NaN. They WON'T get counted and not add to the average.  
        return_diff[return_diff < 0] = np.nan  
        out[:] =  np.nanmean(return_diff, (axis=0))

These two factors are at the bottom of the attached notebook.

Good luck!

Still getting this error on line : returns_spy = returns[symbols('SPY')]

Runtime exception: TypeError: zipline.pipeline.term.__getitem__() expected a value of type zipline.assets._assets.Asset for argument 'key', but got list instead.

Another thing, I notice you write very elegant code. What would be the most elegant way to avoid calling pipeline calculations in before_trading_start() every day, but instead only for those days that you actually rebalance (usually monthly or weekly). I assume this would speed up backtests.

When we copy/paste the CustomFactor created by Dan in the IDE, it fails

class Mean_Diff_Return_From_SPY(CustomFactor):

    returns = Returns(window_length=2)  
    returns_spy = returns[symbols('SPY')]

    inputs = [returns, returns_spy]

    # Set window-length to number of days returns to average  
    window_length = 2  


    def compute(self, today, assets, out, returns, returns_spy):  
        out[:] =  (returns - returns_spy).mean(axis=0)  

I do not understand what is going wrong in the IDE...

For this line: returns_spy = returns[symbols('SPY')] error is Runtime exception: TypeError: zipline.pipeline.term.getitem() expected a value of type zipline.assets._assets.Asset for argument 'key', but got list instead.

One difference (ok I'll admit irritating difference) between the IDE (ie algorithm) environment and the research (ie notebook) environment is the symbols method. The following code returns a single asset in the research environment:

asset_object = symbols('SPY')

but will return a list in the IDE. The clue is in the name 'symbols' -plural. One needs to either use the symbol (note the missing 's') or select the first element of the symbols list to get a single asset object in the IDE. So, change the line

    returns_spy = returns[symbols('SPY')]

to either

    returns_spy = returns[symbol('SPY')]  
    returns_spy = returns[symbols('SPY')[0]]

All should work fine. Good luck.

Dan, thank you so much for your reply! I tried to create a basic algorithm using this customfactor but it failed again.
On line

returns_spy = returns[symbols('SPY')[0]]  

I have the following error: "Runtime exception: RuntimeError: zipline api method symbols must be called during a simulation."

Below my full code

import quantopian.algorithm as algo  
from quantopian.pipeline import Pipeline  
from quantopian.pipeline.data.builtin import USEquityPricing  
from quantopian.pipeline.filters import QTradableStocksUS  
from quantopian.pipeline.factors import CustomFactor,Returns  
from quantopian.algorithm import symbols


def initialize(context):

    # Create our dynamic stock selector.  
    algo.attach_pipeline(make_pipeline(), 'pipeline')

class Mean_Diff_Return_From_SPY(CustomFactor):  
    returns = Returns(window_length=2)  
    returns_spy = returns[symbols('SPY')[0]]  
    inputs = [returns,returns_spy]  
    # Set window-length to number of days returns to average  
    window_length = 10  
    def compute(self, today, assets, out, returns, returns_spy):  
        out[:] =  (returns - returns_spy).mean(axis=0)  

def make_pipeline():

    # Base universe set to the QTradableStocksUS  
    base_universe = QTradableStocksUS()  
    # Factor of yesterday's close price.  
    yesterday_close = USEquityPricing.close.latest  
    avg_outperf=Mean_Diff_Return_From_SPY()  
    # filter for stocks to sell  
    sell_signal=avg_outperf>0.05  
    # filter for stocks to buy  
    buy_signal=avg_outperf<-0.05  
    pipe = Pipeline(  
        columns={  
            'close': yesterday_close,  
            'Average_relative_perf':Mean_Diff_Return_From_SPY,  
            'Sell_signal':sell_signal,  
            'Buy_signal':buy_signal  
        },  
        screen=base_universe & (sell_signal|buy_signal)  
    )  
    return pipe


def before_trading_start(context, data):  
    """  
    Called every day before market open.  
    """  
    context.output = algo.pipeline_output('pipeline')

    # These are the securities that we are interested in trading each day.  
    context.security_list = context.output.index

    print context.output.head(5)  

Do I need to import a "symbols" method at start of the code?

The problem is in the pipeline definition:

    pipe = Pipeline(  
        columns={  
            'close': yesterday_close,  
            'Average_relative_perf':Mean_Diff_Return_From_SPY,  
            'Sell_signal':sell_signal,  
            'Buy_signal':buy_signal  
        },  

The factor labeled 'Average_relative_perf' is being assigned to Mean_Diff_Return_From_SPY. That's the name of the class. Need to assign it to an instance of the class. You probably meant avg_outperf something like this:

    pipe = Pipeline(  
        columns={  
            'close': yesterday_close,  
            'Average_relative_perf':avg_outperf,  
            'Sell_signal':sell_signal,  
            'Buy_signal':buy_signal  
        },  

See the attached backtest. It's also helpful when posting here in the forums to attach a backtest rather than copying the code.

See if that works for you. Good luck.

Thanks a lot for your reply and help Dan. My bad, this one was a silly mistake... Reason why I did not post the backtest before is I failed to create the algorithm because of my error. Everything is fine now.
To add value to the community, I created a basic mean-reversal strategy using this CustomFactor. I just start coding in the IDE but the algo seems OK.

Best

Cyril

Dan, is there a way to add excess (relative to SPY) forward returns to a notebook? I would like to backtest a strategy in the research environment and I am trying to compute hedge returns, that is something using the function 'compute_forward_returns'