Hi.

I am still trying to learn the ins and outs of the Pipeline. I have a problem that WI ("when-issued") securities are not correctly filtered, as far as I understand it. The following code is designed specifically to illustrate the point, specifically with start date 08/26/2004 and end date 08/27/2004.

The first part of the code is the standard tradeable_stocks filter from the Quantopian tutorial https://www.quantopian.com/tutorials/pipeline#lesson11

Then at the end I set specific market cap limits, to illustrate my point. When run between 08/26/2004 and 08/27/2004, this code prints out the output from the pipeline, namely the stocks that pass the filter process:

                                 market_cap          morningstar_name  
Equity(7883 [UTX])              4.783750e+10            UTX  
Equity(16134 [UTX_WI])          4.783750e+10            UTX  
Equity(23555 [WYE])             4.721680e+10            WYE

The point is that UTX_WI should be filtered out, but it is not. By playing around with not_wi in the tradeable_stocks filter in the code, one can verify that this filter only sees the names in the 'morningstar_name' column, and so cannot distinguish UTX from UTX_WI!

Can anyone see through this? Or am I misunderstanding UTX_WI? For one thing, in research I cannot retrieve data for UTX_WI either.

I hope this makes sense.

Many thanks,
Ja Lo

from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline import Pipeline  
from quantopian.pipeline.data import morningstar  
from quantopian.pipeline.filters.morningstar import IsPrimaryShare  


def initialize(context):  
    # Create our dynamic stock selector.  
    attach_pipeline(make_pipeline(context), 'my_pipeline')  


def make_pipeline(context):  
    # Filter for primary share equities. IsPrimaryShare is a built-in filter.  
    primary_share = IsPrimaryShare()

    # Equities listed as common stock (as opposed to, say, preferred stock).  
    # 'ST00000001' indicates common stock.  
    common_stock = morningstar.share_class_reference.security_type.latest.eq('ST00000001')

    # Non-depositary receipts. Recall that the ~ operator inverts filters,  
    # turning Trues into Falses and vice versa  
    not_depositary = ~morningstar.share_class_reference.is_depositary_receipt.latest

    # Equities not trading over-the-counter.  
    not_otc = ~morningstar.share_class_reference.exchange_id.latest.startswith('OTC')

    # Not when-issued equities.  
    not_wi = ~morningstar.share_class_reference.symbol.latest.endswith('.WI')

    # Equities without LP in their name, .matches does a match using a regular  
    # expression  
    not_lp_name = ~morningstar.company_reference.standard_name.latest.matches('.* L[. ]?P.?$')

    # Equities with a null value in the limited_partnership Morningstar  
    # fundamental field.  
    not_lp_balance_sheet = morningstar.balance_sheet.limited_partnership.latest.isnull()

    # Equities whose most recent Morningstar market cap is not null have  
    # fundamental data and therefore are not ETFs.  
    have_market_cap = morningstar.valuation.market_cap.latest.notnull()

    # Filter for stocks that pass all of our previous filters.  
    tradeable_stocks = (  
    primary_share  
    & common_stock  
    & not_depositary  
    & not_otc  
    & not_wi  
    & not_lp_name  
    & not_lp_balance_sheet  
    & have_market_cap  
    )

    # Now set specific market cap limits to get the illustrative securities:  
    capFloor = morningstar.valuation.market_cap.latest > 4.7e+10  
    capCeil = morningstar.valuation.market_cap.latest < 4.8e+10

    columns = {'market_cap' : morningstar.valuation.market_cap.latest,  
               'morningstar_name': morningstar.share_class_reference.symbol.latest  
              }  
    return Pipeline(columns=columns, screen=capFloor&capCeil&tradeable_stocks)

def before_trading_start(context, data):  
    context.output = pipeline_output('my_pipeline')  
    print context.output  


def handle_data(context,data):  
    pass