Pylivetrader and Quantopian - Same data filters, different results.

Back to Community

posted Aug 25, 2020

I'm not sure if I am overlooking something or doing something wrong, but, I tried to modify a code on Quantopian to work with Pylivetrader and am getting different results from the Pipeline.

First block is the Quantopian Code

def make_pipeline():

    have_market_cap = morningstar.valuation.market_cap.latest.notnull()  
    avg_volume = SimpleMovingAverage(inputs=[USEquityPricing.volume],window_length=50)  
    filter_volume = avg_volume > 500000  
    last_price = Latest(inputs=[USEquityPricing.close], window_length=1)  
    filter_price = last_price > 1  
    dollar_volume = AverageDollarVolume(window_length=50)  
    filter_dollar_volume = dollar_volume > 2500000  
    sma_150 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=150)  
    filter_sma_150 = USEquityPricing.close.latest > sma_150

    atr_10_percent = atr_10days_percent()  
    filter_atr_10 = atr_10_percent > 4  
    rsi = RSI(inputs=[USEquityPricing.close], window_length=3)  
    filter_overbought = rsi < 30  
    atr_10 = atr_10days()  
    stocks_to_trade = have_market_cap & filter_volume & filter_price & filter_dollar_volume & filter_sma_150 & filter_atr_10 & filter_overbought

Here is the Pylivetrader code

def make_pipeline():  
    have_market_cap = PolygonCompany.marketcap.latest.notnull()  
    avg_volume = SimpleMovingAverage(inputs=[USEquityPricing.volume],window_length=50)  
    filter_volume = avg_volume > 500000  
    last_price = Latest(inputs=[USEquityPricing.close], window_length=1)  
    filter_price = last_price > 1  
    dollar_volume = AverageDollarVolume(window_length=50)  
    filter_dollar_volume = dollar_volume > 2500000  
    sma_150 = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=150)  
    filter_sma_150 = USEquityPricing.close.latest > sma_150

    atr_10_percent = atr_10days_percent()  
    filter_atr_10 = atr_10_percent > 4  
    rsi = RSI(inputs=[USEquityPricing.close], window_length=3)  
    filter_overbought = rsi < 30  
    atr_10 = atr_10days()  
    stocks_to_trade = have_market_cap & filter_volume & filter_price & filter_dollar_volume & filter_sma_150 & filter_atr_10 & filter_overbought

They both output different set of stocks and maybe a couple of stocks will be the same in a week, but 95% are different.

Small fluctuations in historic price data could account for a small difference, but hardly any matching results seems very unlikely.

Does anyone have any insight into this?

2 responses

Dan Whitnable

Aug 26, 2020

I was able to compare similar Quantopian and Pylivetrader pipelines and got about 97% common results between the two when run on 8-25. On 8-26 the number was down to 75%. I did see some discrepancies (which I'll detail later) but not '95% different'. They were pretty much the same.

First, I created a pipeline in each environment. The Quantopian pipeline was the same as above except without the atr factors (I assume those are custom factors?). That pipeline can be found in the attached notebook. Next, I made some big assumptions that the original was using 'pipeline-live' (which is the module Pylivetrader uses as a default) and accessing Alpaca data. I made a Google Colab notebook and the same Pipeline-Live pipeline, again without the atr factors. The colab notebook can be found here. Finally, I exported the Pipeline_Live output and imported it into the attached notebook to compare the two.

I ran the Quantopian pipeline for the current day (ie 8-26-2020). That is very important. Ensure the two pipelines are run on the same day (they will actually use data from yesterday). This may be one source of a difference?

Here's what I found...

Quantopian returned 267 stocks
Pipeline-Live returned 306 stocks
There are 199 stocks common to both
75 percent of Quantopian stocks are common to both

The filter have_market_cap = PolygonCompany.marketcap.latest.notnull() doesn't filter out ETFs. Polygon includes market cap data for ETFs while Morningstar does not. This is perhaps the biggest issue. It accounted for 62 of the 107 stocks in Pipeline-Live not in Quantopian. Omit those and there are only 45 differences.

The average volumes are not consistent between Quantopian and Alpaca (and probably Polygon but I was using Alpaca data). The issue here is Quantopian only uses 'market trades' in the data which excludes the open and close auctions. See this post for more info on that. In most instances, Quantopian volumes will be less than other sources which do include these auctions. This was the next biggest discrepancy. 43 of the 45 stocks in Pipeline-Live but not in Quantopian after the ETFs were removed had average volumes less than 500,000. These were filtered out on Quantopian but left in on Pipeline-Live.

The open and close prices are not consistent between Quantopian and Alpaca. This is the same issue as above. This impacted the RSI calculations and caused the remaining two stocks to be filtered out on Quantopian but left in on Pipeline-Live.

I didn't go the other direction and find out why some were on Quantopian but not on Pipeline-Live. There were 68. I expect it to be related to price or volume again. I did see some instances where splits weren't properly adjusted for on Alpaca which threw off the average price calculation. This seemed to be infrequent but did happen.

Hope that helps. The biggest issue is using Polygon market cap to exclude ETFs which doesn't work. Beyond that, it seems to be price and volume discrepancies. Over the few days I compared, the two the results were 75-97% common between the two.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Kunal Patel

Aug 27, 2020

Hey Dan,

Thanks for your efforts on this and your insight.

I recreated your steps and tired to find the disconnect and it seems it's the ATR (CustomFactor).

This fits with what you mentioned stating that open and close prices are not consistent between Quantopian and Alpaca, and I would assume the highs as well. This is I believe causing a lot of discrepancies in the results.

For instance the 8-26-2020 tests, I recreated your environments and got similar results, but as soon as I added the ATR back into both pipelines, results became extremely skewed. Pipeline live only returned 13 stocks while Quantopian returned 174.

Thanks again for your help.

You've successfully submitted a support ticket.

Our support team will be in touch soon.