Hi Grant,

1. Yes, all of that sounds correct.
   
2. The number of days corresponds to the number of days inside the window (so day 0, -1, and -2). It's gonna be computed over the data frequency of the events, so if it's minute it's gonna be 3 days of minute data.

Hope that helps.

Hey Thomas,

Let me clarify my second question above. Basically, I'm trying to understand what trailing window of price data mavg(days) uses. Again. for data[stock].mavg(3) computed at 10 am on day 0:

1. Does the average include minutely ticks from the first part of day 0 (up to 10 am), all of day -1, all of day -2, and part of day -3 (after 10 am)?
   
2. Or does the average only include minutely ticks from all of days -1, -2, & -3 (excluding any ticks from day 0)?

In other words, is mavg(days) a moving average of ticks, over the trailing N*days ticks, where N is the number of ticks in a day? Or does the average ignore ticks if they are in the current day of the backtest?

Also, for mavg(days), does days need to be an integer? Or can days be fractional (e.g. mavg(1.5))?

Thomas,

I have essentially the same questions as above regarding the batch transform. How is the trailing window defined?

In the full backtest, I'd like to have the trailing window shift every minute, so that I can define a true minutely moving average. Is this feasible?

Thanks,

Grant

Hi Grant,

Sorry for not being clear. All transforms (including batch_transforms) compute their values over a window, so we only have to think about what's in the window.

A window where you set days=3 will always contain values from the most current dt to opening of days-2 (so that at closing of day 0 there are is exactly 3 days worth of data).

This is irrespective of whether the granularity is minutes or days.

Now the problem is with days this works nicely because there are always exactly 3 days worth of data in your window. With minutes that's not true.

There is an alternative internal flag where you can specify exactly which dt you want to use (e.g. to get a window reaching back 5 minutes) but I don't think we expose that yet -- maybe we should?

So to put it another way... there are 390 trading minutes in a day (or there should be, but we'll ignore that possibility for now)

The first minute is normally at 9.31 EST, referring to the previous 60 seconds worth of data. The last minute of a day is at 16.00 EST.

At 10am on day 0, this is minute number 30. calling mavg(3) should include the following minute data:
day 0: the first 30 minutes
day -1: all 390 minutes
day -2: all 390 minutes
day -3: 360 minutes, missing the first 30 minutes of that day.

so the number of minutes included in the calculation should be 390*3 = 1170

Is this correct?

I think having more control over the window would be useful.

Unfortunately no, in your example it will be:

day 0: the first 30 minutes
day -1: all 390 minutes
day -2: all 390 minutes
day -3: none

Because the transform is only concerned with full days.

The more I think about this the clearer it becomes to me that it should work as you suggested though.

Oh dear. So the first call to mavg(3) at 9.31 EST is almost a 2 day moving average.

Also, I'm interested in the edge cases:

(1) When you say mavg(900) but the dates in the backtester do not stretch this far (2) When you say mavg(900) but the security does not stretch that far (e.g. IPO of facebook)

I agree with James (if I'm understanding correctly)...seems like a problem for minutely trading. The trailing window should look back the same number of minutes, regardless of the time of day, right? Otherwise, it appears that the window size will grow over a given trading day, finally reaching a true 3 day window at the end of the day, correct?

Yeah, we'll fix that.

The window gets filled up as events trickle in, so the first day there will be only one and that's what the average will be computed over.

The batch_transform however returns None until the window is full.

Thomas,

So let me see if I can summarize. As it currently works, mavg(3) will compute using the varying window:

day 0: the first 30 minutes
day -1: all 390 minutes
day -2: all 390 minutes
day -3: none

However, if I write my own moving average using the batch transform, I can compute using this fixed window (~1170 minutes regardless of the time of the trade day):

day 0: the first 30 minutes
day -1: all 390 minutes
day -2: all 390 minutes
day -3: (390-30) minutes

Correct?

I was wondering about this myself, after reading the help. These posts are very old. What is the current status?

My fear when reading this is the averaging of apples and pears. Averaging every minute over some window seems a bit esoteric and falsely accurate. Is it right to weight opening and middle of the day and end of day prices equally? What about prices in the overnight session? How does the calculation handle large gaps?

A simpler idea for mavg(n) is: use current price as best estimate of today's close and average it with previous (n-1) days closes.

Hi Dan,

I don't think there is any advantage to using the built-in mavg. Just grab the data you need in a Pandas dataframe using history() and compute whatever kind of average you need (including handling gaps as you see fit).

Grant

Thanks Grant. That's good advice.

I even get a bug when I call data[SPY].mavg(200) on minutely mode on 2002-01-03. I get exception KeyError. Even if I handle this exception and skip to the next day, the mavg continues to fail, even several years on. Weirdly this bug goes away if I start the algo one year later at 2003-01-03.