How can a multi-output custom factor be used for a filter?

I see they are just returning a tuple with all the values per each asset in pipeline's output, but instantiating the custom factor, in your example

custom_factor = MultipleOutputs()

and trying to access the tuple's values with [] operator as in

my_filter = custom_factor[1] < custom_factor[0]

yields

TypeError: 'MultipleOutputs' object does not support indexing

while trying to use the same syntax you showed in the example

my_filter = custom_factor.low < custom_factor.high

results in

BadBinaryOperator: Can't compute RecarrayField > instancemethod  

Hi Andrea,

I think the code you're looking for is custom_factor.lows < custom_factor.highs. Note the s's at the end of attributes: the sub-factors supplied by a multi-output factor have to exactly match the strings provided at class-definition time. This works as expected for me:

from quantopian.pipeline import CustomFactor  
from quantopian.pipeline.data.builtin import USEquityPricing  
​
​class MultipleOutputs(CustomFactor):
    inputs = [USEquityPricing.close]  
    outputs = ('lows', 'highs')  
    window_length = 10  
    def compute(self, today, assets, out, closes):  
        out.lows[:] = 5  
        out.highs[:] = 6  
f = MultipleOutputs()  

In[7]: f  
MultipleOutputs((USEquityPricing.close::float64,), window_length=10)

In [8]: f.lows < f.highs  
Out[8]: NumExprFilter(expr='x_0 < x_1', bindings={'x_0': RecarrayField((MultipleOutputs((USEquityPricing.close::float64,), window_length=10),), window_length=0), 'x_1': RecarrayField((MultipleOutputs((USEquityPricing.close::float64,), window_length=10),), window_length=0)})  

The scary expression in the output of that last cell says that (f.lows < f.highs) is a Filter which, when evaluated, will compute f and then compute the expression x_0 < x_1 with f.lows bound to x_0 and f.highs bound to x_1.

my_filter = custom_factor.low < custom_factor.high  

results in

BadBinaryOperator: Can't compute RecarrayField > instancemethod  

Hrm...I can't reproduce this error. If I execute the code listed above, I instead get this:

In [5]: custom_factor.low < custom_factor.high  
---------------------------------------------------------------------------  
AttributeError                            Traceback (most recent call last)  
<ipython-input-5-12d92f2a97d7> in <module>()  
----> 1 custom_factor.low < custom_factor.high

/home/ssanderson/quantopian/zipline_repo/zipline/pipeline/factors/factor.py in __getattr__(self, attribute_name)
   1211             raise AttributeError(  
   1212                 'Instance of {factor} has no output called {attr!r}.'.format(  
-> 1213                     factor=type(self).__name__, attr=attribute_name,  
   1214                 )  
   1215             )

AttributeError: Instance of MultipleOutputs has no output called 'low'.  

I've opened a PR in Zipline to make that error also include what the options so that it's easier to find issues like this in the future.

That was a typo just in this message, I didn't adapt the actual error I was seeing to Karen's example. I confirm that in my case the attributes accessed are the same of the labels defined in the outputs attribute of the custom factor.

I'm trying to produce a SSCCE to track the error down now, while in the same factor it works but I'm using two difference custom factor' subclass instances and comparing their "fields".

Btw this dotted notation access is a bit odd, you should have gone with a simple key access like f.['low'] and f.['high']. What happens when someone tries to use 'my-beautiful-indicator' as label and the dash won't allow translating it to a proper python identifier?

edit: typos

My issue was due to the fact that one of my custom factors had an output field called 'bottom'. I spent quite a few hours trying to figure what the hell was coercing my value, supposedly a RecarrayField instance, to a method before realizing that there could be an actual attribute masking it, in fact quantopian.pipeline.factors.Factor has a bottom() method.

This falls into the same kind of issues I was referring to earlier, you'll have to state in docs what strings are allowed and whatnot for outputs, and detect non-valid strings at build time.

Andrea,

Apologies for any trouble this issue may have caused. You are correct that the bottom method was overshadowing the output field, which is almost always not the desired behavior. We've released a fix that will catch any naming collisions and do one of the following: (1) if the output name cannot be used, raise a helpful error message, or (2) if the output name conflicts with a method name, return the output instead of the method. For example, "bottom" can still be used as an output name, but accessing the bottom attribute of a multi-output factor will now give you the output. The reasoning behind this is that most of the methods on factors would not make sense when used on a multi-output factor anyway.

Hope this helps,
David

In my algorithm, I have a dispersion matrix (asset x asset) as output, but the custom factor does not like the matrix. Is there a smooth way to handle this issue?

thanks for this feature

can the outputs have different dtype?

The outputs to a factor can be a datetime, integer, or a float or anything which can be safely coerced into one of these. Generally one doesn't need to explicitly cast these but something like the code below demonstrates different types for different outputs.

class TestOutputTypes(CustomFactor):  
    outputs = ['datetime', 'integer', 'float']  
    window_length = 1

    def compute(self, today, assets, out,):  
        output = np.zeros_like(assets)  
        out.datetime[:] = output.astype('datetime64')  
        out.integer[:] = output.astype('int')  
        out.float[:] = output.astype('float')