Why set_universe contains the leveraged EFT's?

Back to Community

posted

It is simple to use set_universe to get all the tradable securities
but when I add set_do_not_order_list(security_lists.leveraged_etf_list)
I am busy error handling.

Can set_universe by default bring the list does not contain these "blacklisted" securities.

import talib  
def initialize(context):  
    # we want to try this on a range of highly liquid stocks  
    set_do_not_order_list(security_lists.leveraged_etf_list)  
    set_universe(universe.DollarVolumeUniverse(98, 99))

def handle_data(context, data):  
    days = 7  
    prices = history(bar_count=days, frequency='1d', field='price').dropna(axis=1)  
    volumes = history(bar_count=days, frequency='1d', field='volume').dropna(axis=1)  
    # Store each stock's OBV value in a dict  
    # obv = {'sid1': val1, 'sid2': val2, ...}  
    obv = {}  
    try:  
        for stock in volumes:  
            obv[stock] = talib.OBV(prices[stock], volumes[stock])[-1]  
        # Rank obv values in ascending order (in a list of tuples)  
        # obv = [(sid1, val1), (sid2, val2), ...]  
    except:  
        log.error("key error")  
    obv = sorted(obv.items(), key=lambda x: x[1])  
    obv_length = len(obv)  
    obv_long = obv[:10]  
    obv_short = obv[obv_length-10:]  
    if context.account.total_positions_value == 0:  
        for stock in obv_long:  
            log.info(stock[0:][-2])  
            #Giving order of 5% of portfolio  
            order_target_percent(stock[0:][-2], 0.05)  
        for stock in obv_short:  
            log.info(stock[0:][-2])  
            order_target_percent(stock[0:][-2], -0.05)  
    record(leverage= context.account.leverage)

8 responses

Christian Kenyeres

You are having errors with particular securities. For example:

2011-09-30handle_data:20ERRORkey error, stock= Equity(26721 [MOS])
2011-10-10handle_data:20ERRORkey error, stock= Equity(5025 [MI])
2012-01-12handle_data:20ERRORkey error, stock= Equity(26390 [BBBB])
2015-04-09handle_data:20ERRORkey error, stock= Equity(205 [AGN])

There is pricing data:

2011-01-04handle_data:9DEBUGAGN price=70.07
2011-01-04handle_data:11DEBUGBBBB price=41.78
2011-01-04handle_data:13DEBUGMI price=6.98
2011-01-04handle_data:15DEBUGMOS price=75.0

If you go farther, you can see both missing pricing and/or volume data.

Take a look at the attached backtest. Look at the logging output.

2011-01-04handle_data:29WARNNo volume for Equity(2 [AA])
2011-01-04handle_data:34WARNNo price for Equity(2 [AA])
2011-01-04handle_data:29WARNNo volume for Equity(62 [ABT])
2011-01-04handle_data:34WARNNo price for Equity(62 [ABT])
2011-01-04handle_data:29WARNNo volume for Equity(114 [ADBE])
2011-01-04handle_data:34WARNNo price for Equity(114 [ADBE])
2011-01-04handle_data:29WARNNo volume for Equity(328 [ALTR])
2011-01-04handle_data:34WARNNo price for Equity(328 [ALTR])

Yagnesh Desai

Hi Christian;

I get this error when running the backtest between 1st Mar 2013 to 31st Aug 2013

TradingControlViolation  
There was a runtime error on line 49.  
Order for 1190 shares of Equity(37133 [TZA]) at 2013-03-01 00:00:00+00:00 violates trading constraint set_do_not_order_list.Learn More

ie my universe is full of leveraged etfs. So now I will need to think of a way around this.
Not being a programmer, this becomes demotivating for me. As I manage to spare about 1hr in a
day on quantopian and waste it in handling these leveraged ETF's in the universe which Quantopian
must have got rid of in first place. . . .

That's My view.
Else there could be function of removing them from universe which avoids these etfs
from algo itself and not throw error when ordering.

Christian Kenyeres

The code provided provides the solution.

You start with a set of securities in your universe (1-n). This code weeds those out:

set_do_not_order_list(security_lists.leveraged_etf_list)

You only have to add it one time for each algo. You can also save time by copying/pasting common code like this into each new algo you design.

Please be aware that each contest has specific criteria. Because of the 2x, 3x leverage, the ETFs are being excluded from one or more contests.

Yagnesh Desai

I think I am not able to communicate (to/fro) properly.
You can find line4 in my shared post.

set_do_not_order_list(security_lists.leveraged_etf_list)

and once I added this line I get this error in the backtest.

Nicola Cannata

The function set_universe gives back a percentile of all the securities ordered by dollar-volume.
And it can include also leveraged ETFs, as the function was designed well before the Quantopian Open contest and its rules.

The guard set_do_not_order_list(security_lists.leveraged_etf_list) "only" prevents you to execute orders on a leveraged Security (all those included in the leveraged_etf_list) and does not remove them from your universe.
So when you try to execute an order on one Security in the list, an exception is raised and your program terminates with an error.

If you wish to trade a universe without (leveraged) ETFs I suggest to use as universe a proxy of the S&P500 as suggested in this post
https://www.quantopian.com/posts/simulating-s-and-p-500-russell-1000-russell-3000-in-research

Yagnesh Desai

Thanks Nicola for pointer
I will use it. . .

Alisa Deychman

Take a look at the example shared here: https://www.quantopian.com/help#ide-trading-guards

To avoid ordering these leveraged ETFs from set_universe you can do something like:

for sec in data:    # loop over the securities in your universe  
   if sec in data:    #check that we have a current price for the security  
        if sec not in security_lists.leveraged_etf_list:     #remove leveraged ETfs  
                 order_target(sec, 100)

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Yatharth Narang

Won't making another array to store all the securities excluding ETFs help ? Maybe something like this

def initialize(context):  
    set_universe(universe.DollarVolumeUniverse(floor_percentile=95.0, ceiling_percentile=100.0))  
    context.reqd_data = []  
def handle_data(context,data):  
    for stock in data:  
        if stock in security_lists.leveraged_etf_list:  
            return  
        else:  
            context.reqd_data.append(stock)             # update the list everyday

You've successfully submitted a support ticket.

Our support team will be in touch soon.