Hi Ross,

You can use the StaticAssets filter to construct a filter that excludes a static list of securities. I've attached an example algorithm that demonstrates how to to do.

Works a charm! Thank you!

I'm curious if I could use StaticAssets to make sure I include my current holdings in my pipeline. My guess is that StaticAssets is evaluated once at the beginning when the pipeline is created and it would not be possible to update the set of assets to include on a daily basis. Am I wrong? Is there a better way to do this?

Hi Rudiger -

I'm confused by your question. Are you wanting to dynamically change the limited set of stocks that one of your pipeline factors uses as inputs? Or are you just wanting to check that your pipeline universe contains a specific set of of stocks at any given point in time? I guess my confusion is that I thought unless restricted in some fashion, pipeline has access to all stocks available to trade point-in-time?

Hi Rudiger,

That's correct, you cannot incorporate your current positions in your pipeline. Since a pipeline is defined in initialize, you won't have access to the current positions each day. Currently, there isn't a good way to include your current positions in your pipeline definition. You will have to get data for these stocks using data.history.

I've passed along your request for the ability to access current positions in pipeline internally.

OK, let me try to clarify.

Let's say I have a strategy where I rank the Q1500US by PE ratio and buy the lowest 10 stocks. Then I will exit a position if it's rank goes above 20 and I find the lowest ranked stock that I don't hold to replace it.

The problem is: what if one of the stocks leaves the Q1500US universe? I would prefer if it stays in my universe so that the pipeline can continue to rank it compared to all the other Q1500US stocks.

How do I force my held positions to remain in my pipeline universe? The StaticAssets factor appears to be the wrong tool because it seems like its constituents are probably only set once.

i.e. I can't go: screen = Q1500US | StaticAssets(context.list_of_currently_held_stocks)

...Because it wouldn't be evaluated each time, only when the pipeline was created.

Hi Rudiger,

If I understand correctly, you'd need to run your PE ratio (or whatever) factor across all stocks, and then use the point-in-time Q1500US list combined with your context.list_of_currently_held_stocks to do the ranking and selection. This way, you wouldn't be forced to drop certain stocks just because they don't persist in the Q1500US universe.

Grant

Yeah, that appears to be the solution for now, although it seems like a waste given that the point of the pipeline is to be able to set screens, compute ratios, rank, etc very efficiently.

I guess in before_trading_start, I could get my pipeline output, see if any holdings are not included in the pipeline output, if a stock is missing I could query for the PE ratio of that stock and compare it to the 20th ranked stock. This would avoid having to rerank.

My hunch is that for a something like PE ratio, it'll be just as efficient to get the PE ratios for all stocks in the database, and eliminate the ones that are not within the point-in-time Q1500US and your current portfolio, do your ranking, etc. Ranking should be a reasonably efficient process. And if you aren't trading every day, you could limit the ranking to weekly/monthly/quarterly.

The current implementation of the pipeline fetches data and calculates all factors, filters, and classifiers asynchronously from the algorithm. This greatly improves performance but has the side-effect of not being able to interact 'real time' with the algorithm. In this case, one cannot modify a filter based upon current algorithm holdings.

However, there may be certain factors which are used to determine closing existing positions which are separate from those used in 'open' decisions. As Grant pointed out, one cannot filter any of the factors used in closing positions with a filter which may change (eg Q1500US). Such a filter potentially would exclude data for existing positions and therefore wouldn't generate a proper close signal. One can however, still filter or mask any factors used in opening positions only.

The Q1500US doesn't change that quickly - it was designed with stability in mind. This may not be a problem you need to engineer for.

Is there a write up somewhere that describes exactly how the Q1500US is determined and how often it is updated?

See https://www.quantopian.com/help#built-in-filters (and https://www.quantopian.com/posts/the-q500us-and-q1500us ).

For the Q1500US, presumably the default update is monthly, as for the Q500US.

Note that you can construct your own universe and control the update period, based on the example given in the help.

Would like to dynamically both include and exclude any particular stocks in pipeline.

cannot incorporate your current positions in your pipeline.
I've passed along your request for the ability to access current positions in pipeline internally.

Any ETA?

from quantopian.pipeline  import Pipeline  
from quantopian.algorithm import attach_pipeline, pipeline_output  
from quantopian.pipeline.filters import Q500US, StaticAssets

def initialize(context):  
    context.includes = [ symbol('GBR') ]    # Random penny stock, definitely not in Q500US  
    context.excludes = [ symbol('XOM'), symbol('JNJ') ]    # In top 10 market cap

    base_Q_US = Q500US()  
    in_filter =  StaticAssets(context.includes)  
    ex_filter = ~StaticAssets(context.excludes)    # ~ means not.  
    pipe      = attach_pipeline(Pipeline(), 'zoo')  
    pipe.set_screen( (base_Q_US & ex_filter) | in_filter )    # & <- and     | <- or

def before_trading_start(context, data):  
    context.z = pipeline_output('zoo')

    log.info('Number of stocks ' + str(len(context.z)))

    for s in context.excludes + context.includes:  
        log.info('{} in pipe: {}'.format(s.symbol, s in context.z.index))

    # Redefine includes  
    context.includes = [ symbol('DAC') ]   # Another random low dollar stock, replacing the first.

    log.info('Static log line _ {} in pipe: {}'.format('GBR', symbol('GBR') in context.z.index))

'''
2017-03-17 05:45 before_trading_start:18 INFO Number of stocks 499  
2017-03-17 05:45 before_trading_start:21 INFO XOM in pipe: False  
2017-03-17 05:45 before_trading_start:21 INFO JNJ in pipe: False  
2017-03-17 05:45 before_trading_start:21 INFO GBR in pipe: True  
2017-03-17 05:45 before_trading_start:26 INFO Static log line _ GBR in pipe: True  
2017-03-20 05:45 before_trading_start:18 INFO Number of stocks 499  
2017-03-20 05:45 before_trading_start:21 INFO XOM in pipe: False  
2017-03-20 05:45 before_trading_start:21 INFO JNJ in pipe: False  
2017-03-20 05:45 before_trading_start:21 INFO DAC in pipe: False                   <== Want to be True  
2017-03-20 05:45 before_trading_start:26 INFO Static log line _ GBR in pipe: True  <== Want False here, replaced  
'''

Something went haywire with the original from 2016, says KeyError: AssetExists()
Here's a replacement. You can toggle the exclusion list on-off.