Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Some help with resampling data

Hi all,

So I'm trying to experiment a bit and I'm having some trouble with the data. I know it's 1minute level... but just for example, ADX going over 20 on a 30 minute chart. I have 1 minute data and can resample, but I get pandas errors when trying to call talib:

period = 14  
bars = 100  
high = data.history(security, 'high', bars, '1m').dropna()  
low = data.history(security, 'low', bars, '1m').dropna()  
price = data.history(security, 'price', bars, '1m').resample('30T').dropna()  
ta_adx = talib.ADX(high, low, price, period)  

I'm basically trying to match up the numbers with what I'm seeing on my TradingView charts for the previous week, but they're wildly off. I think I've got a fundamental misunderstanding somewhere :)

I guess one question is the data.history() with 1 minute data, will only give the highest price in the last 100 minutes? How do I reconcile that with the high/close for the previous day?

Actually looks like a few more posts have come up.

Edit: So I've got prices for 30 minute/1hour working, but I guess my question is how do I feed this to get, for example, RSI or ADX values based on this price interval?

2 responses

It looks like you are doing this in an algo. I'll give an example of how to do this in a notebook because it's easier to output and visualize what's happening.

The first issue is one should grab all the data (high, low, price) at once and then drop nans if any row is a nan. Applying dropna to each individually runs the risk of either misaligning the data, or worse, having different lengths for each set. So, again in a notebook, this would be

# Get minute pricing for our security  
data = get_pricing(security,  
                   start_date=period_start,end_date=period_end,  
                   fields=['high', 'low', 'price'],  
                   frequency='minute')

The second issue is how to make the resample method work as expected. The first is to align and label the data correctly. The second is applying the correct resampling method to each value. Typically the price is just the last price at the end of the bar but Tradestation may use the mean or median of the bar.

# Resample into 30 minute bars  
# Use the max high, min low, and last price for resampling. There could be other choices for price.  
# Use the `agg` method to apply different functions to each column.  
# Typically bars are labeled by the last minute and not the first so choose label='right'  
# To ensure the last data point is in the bar choose closed='right'  
prices_30m = data.resample('5T', label='right', closed='right').agg({'high': 'max', 'low': 'min', 'price': 'last'})

Check out the attached notebook.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks, Dan, for your reply. You're right - sorry, I forgot to mention that's in an algorithm. This makes a lot of sense... I'm missing that it required the last X "periods" worth of data, not the last 14 closes (on a market level), which was throwing me off. I'm still getting some NaN trying to expand it out and using data.history() but I think I can stumble along from here. Thanks!