Hello P.,

In an algo running in 'minutely' mode the 'window_length' in batch_transform denotes days although the results are returned in minutes. See the logs from the attached.

P.

Wow. Okay, that's even more confusing. So a window length of 2 days will return 780 slices in minute mode. While in daily mode it will return just 2 slices. Which, I guess, does make some sense. But it's a pretty important thing to understand before running your algo live. It explains why I'm getting some wildly different results between daily and minutely modes.

Also, if I kick off an algo using batchtransform or history to run live do I have to wait for the window_length of days to pass before the data will pass through or will it use previous data to accumulate in order to run right off the bat?

Hello P.,

Yes, 'batch_transform' in 'minutely' mode did not have a lot of fans. With 'history' the data is immediately available but with 'batch_transform' you have to wait for a full window hence the

    if prices is None:  
        return  

test. Please see attached.

P.

Hello Quantopian,

The above Custom data plot shows a problem I raised a few months ago. Once 'Via_BT' has a displayed value it does not go away when you cursor over just the 'Via_History' plot. Unless you refresh the page, of course.

P.

I'd seen that the batch transform will eventually be deprecated in favor of history, so I have been steering clear of the batch transform. However, to get a window of trailing minutes, you'll need to write your own accumulator (until the full history function is released). --Grant

But it looks like history does not yet support minutes, even though it can only be run in minute mode. Is that correct? This seems like very strange behavior. meaning that while batchtransform may provide every minute of the past n days, history will only return a single value for each day.

Pumplerod,

Correct (sorta), but my understanding is that this was a limited initial release of history (reason?), but that eventually, it'll provide a trailing window of OHLCV minutely data (see https://www.quantopian.com/posts/draft-proposal-of-new-data-history-api).

I said "sorta" above because in some sense, history does return minute-level data per the help page:

Returned Data

The returned data for daily history, for each day, is:

close_price: the close of the last minute bar for that day. For the current day, the most recent close is returned.  
price: same as close_price.  
open_price: the open of the first minute bar of the given day.  
volume: the sum of all minute volumes. For the current day, the sum of volume thus far.  
high: the max of the minute highs for the given day.  
low: the min of the minute bar lows for the given day.

So, history kinda supports minutes, but in a narrow way in that it provide a kind of cumulative daily bar that gets updated minutely until the end of the day. Not so handy, in my opinion, though.

Grant

Is there a test condition for whether or not your running in minute or daily mode? or live for that matter? That way I could used a lookback_window of say 25 for daily mode for testing to make sure everything is working and use a lookback_window of 1 while only selecting the last 25 items from the list during minute mode.

also, regarding...

Yes, 'batch_transform' in 'minutely' mode did not have a lot of fans. With 'history' the data is immediately available but with 'batch_transform' you have to wait for a full window

I can kind of come up with a loose reason for needing to wait to accumulate the data while testing, however once the algo is running live it should really backfill so you don't have to wait for what could be months. But I guess that's part of the reason for the move to history()

I think that everyone here correctly understands the current state of affairs with batch_transform() and history(). Hopefully history() makes this all easier, more powerful, and perform better.

There is no programmatic way to know whether the algo is in daily or minutely mode. In general, the backtester API is designed so that it works no matter what mode you choose. That's why, for instance, the batch_transform is always in days, regardless of what mode it's in. As a hack, you can check the time on the bar. If it's 9:31 in the morning you know you're in minute mode.