Hi Peter,

The current implementation of VWAP uses daily data because pipeline is currently integrated with daily data only. In the future, we would like to be able to incorporate minute-level data in pipeline, but until that happens, we won't be able to create built-in factors that use minute data. The best alternative right now is to make your own VWAP computations outside of pipeline using minute data from data.history.

Sorry for the inconvenience.

Hi Jamie,
Thanks for the quick reply. I understand about the limitations of Pipeline.
What would be wrong with computing VWAP once and then attaching it as an external source along the lines of the morningstar data perhaps like:

from quantopian.pipeline.data import data_from_daily_bars.

Does that sound silly?

Hi Jamie,

Regarding "The current implementation of VWAP uses daily data because pipeline is currently integrated with daily data only" if I understand the discussion around VWAP, the implementation would still be daily, since for each stock a daily VWAP value would be computed, using your minute bar database. So, you would not be incorporating minute-level data into pipeline directly, but rather daily data derived from minute bars (which in effect is what you do to create the daily bar database in the first place). Are you misunderstanding the point here, since you say "we won't be able to create built-in factors that use minute data"?

Computing daily VWAP from minute bars seems tractable, since you'd set up the database from 2002 to present, and then every night/weekend, update it. Of course, as errors in the minute bar data are found (e.g. missing splits), you'd have to make corrections to the database, but the corrections would be on individual stocks, not the entire database. So the mechanics of setting up and maintaining the database don't seem so bad, but maybe I'm missing something?

Intuitively, it does seem like there would be a great advantage in the conceived workflow to use daily prices that are smoothed in some fashion (e.g. VWAP), versus values from individual trades (e.g. the last trade of the day). Although apparently there are aspirations "to incorporate minute-level data in pipeline" there was a discussion when pipeline was launched that computationally, it would be impossible on the present platform (see Scott Sanderson's comments). Do you actually have ideas on how to do it? If so, when?

As I understand it, Quantopian already "computes" all the fields of each daily bar, i.e., OHLCV, by looping over historical 1-minute bars. This is an excellent design choice which results in these OHLCV daily price fields being more realistic relative to actual trading possibilities than traditional daily-only data sources are.

The best way to add robust support for VWAP would be to add it as a native field to each daily bar in the database. Given that Quantopian already constructs their daily bars from intraday data, this would add virtually zero overhead to that calculation process, and would make VWAP easily accessible in a daily pipeline.

Having a native VWAP price field would also give Quantopian another unique advantage vs. competing services. Please consider taking this approach.

VWAP would seem to be a good starting point (and would seem to be trivial if, indeed, the daily OHLCV bars are computed from the minute bars). Any daily statistic computed solely on a stock-by-stock basis would seem to be straightforward. The best approach would be to put users in the drivers seat, with an API that would allow computation of custom daily statistics derived from minute bars (which could include incorporating minute bar data from a trailing window of N days).