Hi Josh,

You can use the history() function to compare prices across current and previous bars in minute-mode simulation. There are a few different ways you can implement the logic, if you want to compare the current price to yesterday's close price you can follow this example right out of the help docs:

def initialize(context):  
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):  
    price_history = history(bar_count=2, frequency='1d', field='price')  
    for s in data:  
        prev_bar = price_history[s][-2]  
        curr_bar = price_history[s][-1]  
        if curr_bar > prev_bar:  
            order(s, 20)  

The history() function returns a dataframe of historical prices and dates that you can index into to grab the data you want. So in that example price_history[s][-1] is the most recent price for stock 's' and price_history[s][-2] is the price before that one (i.e. yesterday's close).

If you have a simple algo you are trying to get working with this logic feel free to post it and I'd be happy to try to help you debug it.

Best wishes, Jess

This however only applies on a day-basis i assume? Since the only frequency allowed is 1d (god i hope they give us 1m)

Hello Axel,

Here's some code to accumulate minute bars.

Grant

import pandas as pd

window = 5

def initialize(context):  
    context.stocks = [ sid(19662), # XLY Consumer Discrectionary SPDR Fund  
                       sid(19656), # XLF Financial SPDR Fund  
                       sid(19658), # XLK Technology SPDR Fund  
                       sid(19655), # XLE Energy SPDR Fund  
                       sid(19661), # XLV Health Care SPRD Fund  
                       sid(19657), # XLI Industrial SPDR Fund  
                       sid(19659), # XLP Consumer Staples SPDR Fund  
                       sid(19654), # XLB Materials SPDR Fund  
                       sid(19660)] # XLU Utilities SPRD Fund  
    context.prices = pd.DataFrame()  
def handle_data(context, data):  
    prices = history(1,'1d','price')  
    context.prices = context.prices.append(prices)  
    context.prices = context.prices.tail(window)  
    if len(context.prices) < window:  
        return  
    print context.prices  

i wish we had backtest graphs with minute-level data :(

if you want to grab minute level data, here's basically what i do

def initialize(context):  
 context.prices = {}  
 context.stocks = [sid(123), sid(456)]  
 context.maxLen = 360  
def handle_data(context,data):  
 for sid in data  
  if not context.prices.hasKey(sid):  
   context.prices[sid] = []  
  context.prices[sid].insert(0,data[sid].close_price)  
  del context.prices[context.maxLen:]  
 # do your logic here  

I'm awfully confused by the history method.

I thought that it was presenting the last n number of bars regardless of the '1d' option and that the information would be dependent on whether you are running in minute or daily mode. Such that if I requested the last 10 bars in minute mode but was only 5 minutes into the day that the first 5 slices would be daily data and the last (most recent) slices would be in minute mode (which is very confusing). Do I have this wrong? From Grant's example it would appear that I need to accumulate my own minute data.

Hi Pumplerod,

Yeah, history is a bit convolved. I recommend doing a close read-through of the description on the help page. Perhaps another way of going at this would be if you posted what you are needing to accomplish. If you want minutely bars, then until the history method is upgraded, you'll need to accumulate them in some fashion.

Regarding your specific scenario, keep in mind that history has a warm-up, so as soon as your algo starts, you have access to the full trailing window (daily plus current minutely). So, if you request 10 bars, you'll immediately get 10 bars (9 daily plus one minutely).

Grant

History will return the previous daily bar's data plus the current minute bar data.

For example, if you do history(10, '1d', 'price') this will return the previous 9 days trailing data plus the current day's partial bar. This is useful when you're trying to compare the current data to historical values. If you'd like to only use the 9 day trailing data, you can use history(10, '1d', 'price').iloc[0:-2]

When Grant wrote history(1, "1d", "price") he was getting the current day's partial bar data. I agree that it can get confusing, but take a look at the help docs, hopefully they better explain the concept!

When augmented history function comes out is it possible to keep the same pacing. E.g. if current time is 11 am I call history than the bar is from previous days 11 am to current days 11 am.

When pacing between the bars are not the results get skewed. This would be the utopian way to implement this.

May be a you can specify the date time also to look back. Say if you specify 10 am when it is 11 am then it would 10 am to 10 am bar.

Storing the intraday bars can be end up becoming really memory hungry. This is a version that stores intraday bars, but allows you to specify a bar length. e.g. every 10 min. That way longer windows can be stored without making the algo run brutally slow.

Since you are caching from data do you need to use history(1, '1d', 'price') in this? Trying to understand why you use it.

You are correct about that, I was originally using data but switched to the history function because it's indexed with a datetime and simpler to append to the dataframe. The data argument can be dropped.

Here's the version with no args

class DataCache:  
    '''  
    Stores a trailing time series of intraday bars  
    '''  
    def __init__(self, lookback_days=1, bar_size=10):  
        """  
        :Params  
            lookback_days: int  
                number of business days in trailing window  
                default: 1  
            bar_size: 1 < int/float < 60  
                keeps every i'th bar, where i = bar_size (assumes minute bars)  
                default: 10  
        """  
        self.lookback_days = lookback_days  
        self.bar_size = bar_size  
        self.bar_check = lambda dt: dt.minute % self.bar_size == 0  
        self.df = pd.DataFrame()  
        self.calendar = tradingcalendar  
    def append(self):  
        now = get_datetime()  
        bar = history(1, '1d', 'price')  
        df = self.df.append(bar).dropna()  
        trading_days = self.calendar.trading_days  
        today = self.calendar.canonicalize_datetime(now)  
        idx = trading_days.searchsorted(today) - self.lookback_days  
        shift = (now - today)  
        start_dt = trading_days[idx]  
        self.df = df.truncate(before=start_dt + shift)  
    def process_bar(self):  
        dt = get_datetime()  
        if self.bar_check(dt):  
            self.append()  

@Jessica Stauth,

How would you add prev_bar and curr_bar to dataframe the pipeline produces?

def initialize(context):  
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):  
    price_history = history(bar_count=2, frequency='1d', field='price')  
    for s in data:  
        prev_bar = price_history[s][-2]  
        curr_bar = price_history[s][-1]  
        if curr_bar > prev_bar:  
            order(s, 20)  

@Kelvin Winrow. The code above, along with most of this post, is not really valid anymore. You are on the right track using pipeline which is the current approach.

If I understand the question, you would like your pipeline output, in an algo, to include todays data, yesterdays data, and the day before yesterdays data? By definition, a pipeline returns values of factors 'as of before start of trading on the current simulation day'. So, out of the box, you get yesterdays (actually the previous trading day) values. One can also add 'lagged' factor values to the pipeline definition, for example, to get data data from the day before yesterday or any previous day. Take a look at these posts
https://www.quantopian.com/posts/price-of-security-n-days-ago-from-within-pipeline
https://www.quantopian.com/posts/get-lagged-output-of-pipeline-custom-factor

Pipeline only knows about data before the market open. You can however, get OHLCV pricing data using the data.current or data.history methods and then append that data to the pipeline output dataframe. If that's what you want maybe take a look at this post
https://www.quantopian.com/posts/how-to-add-current-price-to-usequitypricing-dot-close

Hope that helps. If you are looking to do something similar in a notebook, getting the current days info is a bit different.