SMA calculation mismatching

Back to Community

posted

I'm trying to calculate some simple tech indicator by using Pipeline and by simply calculating mean of last 20 days.
And numbers were mismatching.

from quantopian.pipeline import CustomFactor, Pipeline  
from quantopian.research import run_pipeline  
from quantopian.pipeline.factors import SimpleMovingAverage  
from quantopian.pipeline.data.builtin import USEquityPricing  
import pandas as pd

security = symbols('SPY')


def make_pipeline():  
    sma = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=20)  
    latest_close = USEquityPricing.close.latest  
    return Pipeline(  
        columns = {  
            'latest_close':latest_close,  
            'sma':sma  
        },  
    )


result = run_pipeline(make_pipeline(), '2016-01-01', '2016-06-30')

result = result.reset_index()  
result = result[result['level_1']==security][['level_0','latest_close','sma']]  
result = result.rename(columns={'level_0':'date'}).set_index('date')  
print 'factor',result.tail(1)['sma'].values[0]  
print 'calulation',result.tail(20)['latest_close'].mean()

factor 207.688491281
calulation 208.232578

maybe I doesn't understand some theoretical concept about SMA ???

7 responses

Volodymyr Nepiuk

I probably have a idea - function SimpleMovingAverage, calculates moving averages including weekends.
We don't have trading on weekend so SimpleMovingAverage will fill weekend data with some backfill and calculate mean.

But if you calculate by pandas rolling mean it will skip weekend and take next week data. (as written here http://www.investopedia.com/terms/s/sma.asp)

Am I right ?

Ernesto Perez

Hi Volodymyr,

By default, the pipeline adjusts windows of pricing data for dividend and split events.
In this case you are comparing the SMA of a window of pricing data adjusted for a dividend event that happened on 6/17/2016 (factor's output), and the mean of the last 20 as-traded close prices (not adjusted). This will cause the slight difference you perceive between the factor and calculation values.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Volodymyr Nepiuk

it's confusing

sma = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=20)  
latest_close = USEquityPricing.close.latest

We calculating SMA by using as input USEquityPricing.close, (and this is a Adjusted price)
But we calculating mean by using the USEquityPricing.close.latest (and this is a real close)

So how to understand where is adjusted close where is real ????
Because datasource the same USEquityPricing.close

Ernesto Perez

The price adjustment happens when a factor requests a lookback window of data (window_length > 1). Using the example above, the SimpleMovingAverage factor requests a window of the last 20 closing prices, which have to be adjusted relative to the day in which the SMA is being calculated.

Since USEquityPricing.close.latest only requests one value, it is not necessary to adjust it.

For a different perspective, you can find here a graphical explanation of how price adjustment works in the platform.

Disclaimer

Volodymyr Nepiuk

cool, so do you have some function what return adjusted price ???
Because looks like you can't trust to get_pricing for now.(https://www.quantopian.com/posts/price-mismatch-at-pipeline-and-get-pricing)

Dan Dunn

get_pricing() now uses the same adjusted data source as pipeline and the backtester.

Disclaimer

Nick Blanchet

Sorry to reopen this thread here but was just doing some local work using the open source zipline (awesome stuff by the way, thanks!) and seeing this:

pipe: 3.946, history: 3.942  
pipe: 3.942, history: 3.915  
pipe: 3.915, history: 3.891  
pipe: 3.891, history: 3.893  
pipe: 3.893, history: 3.887  
pipe: 3.887, history: 3.854  
pipe: 3.854, history: 3.811  
pipe: 3.811, history: 3.753  
pipe: 3.753, history: 3.727  
pipe: 3.727, history: 3.696  
pipe: 3.696, history: 3.697  
pipe: 3.697, history: 3.711  
pipe: 3.711, history: 3.728

pipe:  SimpleMovingAverage(window_length=10, inputs=[USEquityPricing.close])  
history:  data.history(context.asset, 'close', bar_count=10, frequency="1d").mean()

Now I understand the 'adjusting' factor described above but this seems to just be offset by one (with 'history' having more recent data). What I am ultimately trying to confirm is which one is right? I'll also note that looking at the most recent price in data.history vs the pipeline also does not match accordingly. Have a feeling I am doing something wrong or not understanding but not sure what else to check!

I'll also note that switching out the SMA from the pipeline to the data.history, while it computes much slower, leads to much more precise timing when just looking at a simple SMA cross strategy. The pipeline does appear to have a day lag. (attempted to upload a screenshot of a graph explaining but that isn't possible)

Thanks

You've successfully submitted a support ticket.

Our support team will be in touch soon.