Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
batch transform w/ set_universe - problem?
Grant Kiehne
posted May 15, 2013

With the attached backtest, I find that the batch transform is returning a trailing window length that starts at 1, and increases by 1 each minute. For the batch transform, I am using:

R_P = 0  # refresh period  
W_L = 1 # window length

So, the window length should be fixed at 390 minute bars, right?

--Grant

1970-01-01initialize:17INFOuniverse is 46 to 48  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT2013-05-10 13:31:00+00:00  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT Securities  
2013-05-10PRINT[10753, 25605, 30380, 40976, 36369, 26542, 38425, 40478, 42018, 16934, 18183, 26157, 3124, 38454, 12553, 35897, 25661, 32840, 42060, 27215, 37970, 14420, 24664, 16476, 25182, 34414, 42100, 1144, 24704, 43653, 39563, 22156, 20624, 32917, 27803, 32419, 26277, 4265, 40106, 41644, 17584, 4792, 7877, 39622, 8394, 9421, 39634, 26324, 24793, 26332, 40673, 20713, 27370, 12011, 27884, 26860, 39673, 35201, 18184, 39177, 42763, 42767, 35088, 24853, 33046, 20763, 20765, 15136, 28450, 7465, 26924, 23857, 35128, 19259, 38721, 23947, 9028, 4421, 27974, 33580, 39479, 36237, 25424, 20818, 27280, 18786, 40763, 21870, 8566, 21879, 6009, 38782, 40319, 42369, 15752, 393, 8586, 7563, 34701, 38296, 38290, 5016, 10054, 41384, 39342, 18863, 7609, 31162, 15306, 27084, 16333, 677, 26593, 5090, 42983, 28144, 41460, 17401]  
2013-05-10PRINT Current prices  
2013-05-10PRINT[ 31.81 5.5 13.56 19.26 18.45 8.65 22.16 28.91 59.97 8.44 9.06 9.47 12.32 81.85 10.1 4.97 25.6 64.2 14.85 6.98 35.14 19.06 25.53 12.78 5.05 27.15 16.5 3.93 15.25 20. 13.76 4.5 12.16 13.61 40.12 4.96 10.73 4.98 79.45 4.2701 3.53 13.25 30.08 14.2 23.33 13.87 5.25 10.43 13.09 22.73 41.24 23.79 3.26 3.7101 3.75 10.81 21.47 25.99 3.3 17.02 27.33 27.62 0.8732 1.06 33.55 8.12 7.01 3.738 17.29 2.36 14.04 29.12 7.9401 22.436 60.64 24.26 12.65 9.45 20.16 42.08 93.16 26.08 14.37 9.25 12.79 21.57 18.61 2.3 23.85 4.4199 24.15 3.01 2.41 29.76 2.77 2.46 43.13 29.97 1.76 16.45 1.51 13.61 19.53 13.5 19.9499 3.65 21.6 1.78 1.14 4.9 4.8 2.42 16.1 14.42 25.3 4...  
2013-05-10PRINT Trailing price window  
2013-05-10PRINT[[ 31.81 5.5 13.56 19.26 18.45 8.65 22.16 28.91 59.97 8.44 9.06 9.47 12.32 81.85 10.1 4.97 25.6 64.2 14.85 6.98 35.14 19.06 25.53 12.78 5.05 27.15 16.5 3.93 15.25 20. 13.76 4.5 12.16 13.61 40.12 4.96 10.73 4.98 79.45 4.2701 3.53 13.25 30.08 14.2 23.33 13.87 5.25 10.43 13.09 22.73 41.24 23.79 3.26 3.7101 3.75 10.81 21.47 25.99 3.3 17.02 27.33 27.62 0.8732 1.06 33.55 8.12 7.01 3.738 17.29 2.36 14.04 29.12 7.9401 22.436 60.64 24.26 12.65 9.45 20.16 42.08 93.16 26.08 14.37 9.25 12.79 21.57 18.61 2.3 23.85 4.4199 24.15 3.01 2.41 29.76 2.77 2.46 43.13 29.97 1.76 16.45 1.51 13.61 19.53 13.5 19.9499 3.65 21.6 1.78 1.14 4.9 4.8 2.42 16.1 14.42...  
2013-05-10PRINT Trailing window length  
2013-05-10PRINT1  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT2013-05-10 13:31:00+00:00  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT Securities  
2013-05-10PRINT[10753, 39938, 25605, 30380, 40976, 36369, 26542, 38425, 40478, 42018, 16934, 18183, 26157, 3124, 38454, 12553, 35897, 25661, 32840, 42060, 27215, 37970, 14420, 24664, 16476, 25182, 34414, 42100, 1144, 24704, 43653, 39563, 22156, 20624, 32917, 27803, 38597, 32419, 26277, 4265, 40106, 41644, 17584, 4792, 7877, 39622, 8394, 9421, 39634, 26324, 24793, 26332, 40673, 20713, 27370, 12011, 27884, 26860, 39673, 35201, 18184, 39177, 42763, 42767, 35088, 24853, 33046, 20763, 20765, 15136, 28450, 7465, 26924, 23857, 35128, 19259, 38721, 23947, 9028, 4421, 27974, 33580, 39479, 36237, 25424, 20818, 27280, 18786, 40763, 21870, 8566, 21879, 6009, 38782, 40319, 42369, 15752, 393, 8586, 7563, 34701, 38296, 38290, 5016, 28058, 10054, 41384, 39342, 18863, 7609, 31162, 15306, 27084, 16333, 677, 26593, 5090, 42983, 28144, 41460, 17401]  
2013-05-10PRINT Current prices  
2013-05-10PRINT[ 31.81 16.03 5.5 13.56 19.26 18.45 8.65 22.07 28.91 60.09 8.44 9.06 9.4611 12.32 81.85 10.1 4.97 25.6 64.2 14.77 6.98 35.14 19.06 25.53 12.78 5.08 27.14 16.5 3.93 15.25 20.0601 13.76 4.5 12.16 13.61 40.12 12.63 4.96 10.73 4.96 79.45 4.2701 3.53 13.22 30.08 14.2432 23.33 13.87 5.25 10.43 13.09 22.75 41.24 23.79 3.38 3.73 3.72 10.81 21.47 25.99 3.3 17. 27.33 27.62 0.86 1.06 33.55 8.12 7.01 3.72 17.29 2.36 14.04 28.84 7.9401 22.436 60.64 24.26 12.65 9.45 20.16 42.08 93.16 26.08 14.37 9.24 12.79 21.57 18.61 2.3 23.85 4.44 24.15 3.01 2.42 29.76 2.77 2.46 43.13 30.16 1.74 16.41 1.51 13.61 25.46 19.53 13.5001 19.9499 3.65 21.6 1.78 1.14 4.9 4.8 2.45 16...  
2013-05-10PRINT Trailing price window  
2013-05-10PRINT[[ 31.81 nan 5.5 13.56 19.26 18.45 8.65 22.16 28.91 59.97 8.44 9.06 9.47 12.32 81.85 10.1 4.97 25.6 64.2 14.85 6.98 35.14 19.06 25.53 12.78 5.05 27.15 16.5 3.93 15.25 20. 13.76 4.5 12.16 13.61 40.12 nan 4.96 10.73 4.98 79.45 4.2701 3.53 13.25 30.08 14.2 23.33 13.87 5.25 10.43 13.09 22.73 41.24 23.79 3.26 3.7101 3.75 10.81 21.47 25.99 3.3 17.02 27.33 27.62 0.8732 1.06 33.55 8.12 7.01 3.738 17.29 2.36 14.04 29.12 7.9401 22.436 60.64 24.26 12.65 9.45 20.16 42.08 93.16 26.08 14.37 9.25 12.79 21.57 18.61 2.3 23.85 4.4199 24.15 3.01 2.41 29.76 2.77 2.46 43.13 29.97 1.76 16.45 1.51 13.61 nan 19.53 13.5 19.9499 3.65 21.6 1.78 1.14 4.9 4....  
2013-05-10PRINT Trailing window length  
2013-05-10PRINT2
26 responses
Thomas Wiecki
May 15, 2013

And this does not occur without a universe?

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Grant Kiehne
May 15, 2013

Hello Thomas,

I see the same behavior with a list of securities:

2013-05-10PRINT ----------------------------  
2013-05-10PRINT2013-05-10 13:31:00+00:00  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT Securities  
2013-05-10PRINT[24, 8229, 8554, 5061, 8459]  
2013-05-10PRINT Current prices  
2013-05-10PRINT[ 457.23 78.47 163. 32.58 59.52]  
2013-05-10PRINT Trailing price window  
2013-05-10PRINT[[ 457.23 78.47 163. 32.58 59.52]]  
2013-05-10PRINT Trailing window length  
2013-05-10PRINT1  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT2013-05-10 13:32:00+00:00  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT Securities  
2013-05-10PRINT[24, 8229, 8554, 5061, 8459]  
2013-05-10PRINT Current prices  
2013-05-10PRINT[ 457.401 78.39 162.89 32.405 59.56 ]  
2013-05-10PRINT Trailing price window  
2013-05-10PRINT[[ 457.23 78.47 163. 32.58 59.52 ] [ 457.401 78.39 162.89 32.405 59.56 ]]  
2013-05-10PRINT Trailing window length  
2013-05-10PRINT2

I've attached the backtest so you can try it yourself. Is my code incorrect, or is there a bug in the batch transform?

Grant

Thomas Wiecki
May 16, 2013

Hi Grant,

OK, I see what the problem is. We are checking if there's a new day event. Thus with a window_length of 1 and daily bars you will update the batch_transform on the first day which is correct. But with minutes you update with the first minute bar of the day which is not correct. Figuring out a fix for that now.

It is a current short-coming of zipline that most unittests test for correct behavior with daily bars although we are working on adding more minute tests to catch the issues in the future. Sorry about that.

Thomas

Thomas Wiecki
May 16, 2013

OK, I have a fix here:
https://github.com/quantopian/zipline/pull/163

Grant Kiehne
May 17, 2013

Thanks Thomas,

I don't have any visibility of when changes get rolled into Quantopian. I just tried the algorithm again, and the problem persists. Should it be gone?

This flags a potential improvement...engineering change control with some level of end-user visibility/notification (and possibly participation). As I mentioned in another post, you probably need a public test instance, as well. Do you have an internal test instance that matches the current release?

Grant

Thomas Wiecki
May 17, 2013

Sorry, the fix just made it into zipline, there are often a couple of days until we update the zipline that's running on Quantopian.

Personally, I really like your idea of a testing server where a set of beta users can try to break things before we ship them to production, especially with live-trading there is an even greater need for stability.

Grant Kiehne
May 18, 2013

Thanks Thomas,

I confirmed the fix (see below and attached).

Grant

2013-05-10PRINT ----------------------------  
2013-05-10PRINT2013-05-10 20:00:00+00:00  
2013-05-10PRINT ----------------------------  
2013-05-10PRINT Securities  
2013-05-10PRINT[24, 8229, 8554, 5061, 8459]  
2013-05-10PRINT Current prices  
2013-05-10PRINT[ 452.88 78.88 163.45 32.6862 61.64 ]  
2013-05-10PRINT Trailing price window  
2013-05-10PRINT[[ 457.23 78.47 163. 32.58 59.52 ] [ 457.401 78.39 162.89 32.405 59.56 ] [ 459.24 78.479 163.11 32.47 60.02 ] ..., [ 452.51 78.82 163.45 32.675 61.61 ] [ 452.65 78.85 163.43 32.67 61.63 ] [ 452.88 78.88 163.45 32.6862 61.64 ]]  
2013-05-10PRINT Trailing window length  
2013-05-10PRINT390  
2013-05-13PRINT ----------------------------  
2013-05-13PRINT2013-05-13 13:31:00+00:00  
2013-05-13PRINT ----------------------------  
2013-05-13PRINT Securities  
2013-05-13PRINT[24, 8229, 8554, 5061, 8459]  
2013-05-13PRINT Current prices  
2013-05-13PRINT[ 453.52 78.71 163.12 32.63 61.32]  
2013-05-13PRINT Trailing price window  
2013-05-13PRINT[[ 457.401 78.39 162.89 32.405 59.56 ] [ 459.24 78.479 163.11 32.47 60.02 ] [ 458.995 78.36 163.12 32.451 60.13 ] ..., [ 452.65 78.85 163.43 32.67 61.63 ] [ 452.88 78.88 163.45 32.6862 61.64 ] [ 453.52 78.71 163.12 32.63 61.32 ]]  
2013-05-13PRINT Trailing window length  
2013-05-13PRINT390
Thomas Wiecki
May 18, 2013

Great, thanks for checking. Any outstanding issues?

Grant Kiehne
May 18, 2013

Thomas,

I don't know of anything else, but I'll be sure to let you know if I find other problems.

Grant

Grant Kiehne
May 18, 2013

Hello Thomas,

One thing (which I can test) is a detail about how the batch transform works. As I understand, it is actually a data accumulator, and as such, builds up a full window of data the then delivers it once it is full (e.g. 390 minute bars). It then iteratively adds the most recent bar, and drops the oldest. In order for the accumulator to work, does the batch transform need to be called every time handle_data is called? Or could the batch transform be part of a conditional statement (e.g. if Wednesday at 10 am, then call the batch transform)? And if it is a conditional call, will it have accumulated the data?

Grant

Thomas Wiecki
May 18, 2013

Hi Grant,

Yes, it's an accumulator. The name is a little unfortunate in retrospect, we might change or alias it.

If you don't pass an event it will not get accumulated, so using it in an if-statement would work.

Thomas

Grant Kiehne
May 18, 2013

Thomas,

I don't understand. You say that if I "don't pass don't pass an event it will not get accumulated, so using it in an if-statement would work." It seems that an if-statement would not work. For example, within handle_data, I would have (with the globals R_P & W_L):

R_P = 0  # refresh period  
W_L = 1 # window length

if run_batch_transform:  
    d = get_data(data, context.stocks)

@batch_transform(refresh_period=R_P, window_length=W_L) # set globals R_P & W_L above  
def get_data(datapanel,sids):  
    return datapanel['price'].as_matrix(sids)

So, say that I am using minute data, and run_batch_transform = True only at 10 am on Weds. What happens? The batch transform is only called once per week, so it will not accumulate 390 minute bars, right?

Grant

Thomas Wiecki
May 18, 2013

Oh, that's how you meant -- sorry. No, it will only accumulate what you pass in. I have plans for making the batch_transform better integrated so that you don't have to call it explicitly each time. Also nice might be an accumulate function that just returns a dataframe and doesn't require a decorator which seems to cause some confusion.

Grant Kiehne
May 18, 2013

O.K. I'll do some testing and report back. If I understand you correctly, a batch transform will only accumulate data if called. So, for example, with minute data, in order to get a window of 390 bars, I need to call the batch transform 390 times, correct?

Also, using my example above, what will happen if I only call the batch transform once per week in minute mode? Will it accumulate until it has 390 minute bars, and then start returning the trailing window? If so, this could lead to some whacky results...after 390 weeks it will start returning.

--Grant

Thomas Wiecki
May 18, 2013

Yes, that's correct. If you wanted, say, to only accumulate the first minute bar of each day you could do a conditional. But otherwise you have to pass it every time.

I suppose accumulate and calling of the batch_transform could be split into separate operations. That way it'd be more explicit.

Thomas

Grant Kiehne
May 21, 2013

Hello Thomas,

I just did a bit more testing of the batch transform on minute bars with set_universe. It is pretty sluggish, taking > 200 ms just to return the data:

1970-01-01initialize:18INFOuniverse is 32 to 34  
2013-05-17handle_data:34INFOget_data 234.745025635 ms  
2013-05-20handle_data:34INFOget_data 237.904071808 ms  
2013-05-20handle_data:34INFOget_data 234.466075897 ms  
2013-05-20handle_data:34INFOget_data 234.211921692 ms  
2013-05-20handle_data:34INFOget_data 231.967926025 ms  
2013-05-20handle_data:34INFOget_data 233.593940735 ms  
2013-05-20handle_data:34INFOget_data 229.863166809 ms  
2013-05-20handle_data:34INFOget_data 229.838848114 ms  
2013-05-20handle_data:34INFOget_data 231.215953827 ms  
2013-05-20handle_data:34INFOget_data 229.033946991 ms  
2013-05-20handle_data:34INFOget_data 231.724977493 ms  
2013-05-20handle_data:34INFOget_data 231.696844101 ms  
2013-05-20handle_data:34INFOget_data 230.749845505 ms

With a smaller universe, I get the same result:

1970-01-01initialize:18INFOuniverse is 92 to 92.2  
2013-05-17handle_data:34INFOget_data 258.349895477 ms  
2013-05-20handle_data:34INFOget_data 315.945863724 ms  
2013-05-20handle_data:34INFOget_data 256.926059723 ms  
2013-05-20handle_data:34INFOget_data 254.40788269 ms  
2013-05-20handle_data:34INFOget_data 313.712835312 ms  
2013-05-20handle_data:34INFOget_data 258.793115616 ms  
2013-05-20handle_data:34INFOget_data 254.894971848 ms  
2013-05-20handle_data:34INFOget_data 314.038038254 ms  
2013-05-20handle_data:34INFOget_data 258.286952972 ms  
2013-05-20handle_data:34INFOget_data 256.145954132 ms  
2013-05-20handle_data:34INFOget_data 318.190097809 ms  
2013-05-20handle_data:34INFOget_data 259.150028229 ms  
2013-05-20handle_data:34INFOget_data 256.146907806 ms  
2013-05-20handle_data:34INFOget_data 313.978910446 ms  
2013-05-20handle_data:34INFOget_data 257.718086243 ms

However, if I run on a list of sids (context.stocks = [sid(24),sid(8229),sid(8554),sid(5061),sid(8459)]), the batch transform delivers the data quickly:

2013-05-17handle_data:34INFOget_data 4.87780570984 ms  
2013-05-20handle_data:34INFOget_data 4.50301170349 ms  
2013-05-20handle_data:34INFOget_data 4.37092781067 ms  
2013-05-20handle_data:34INFOget_data 4.19592857361 ms  
2013-05-20handle_data:34INFOget_data 4.29201126099 ms  
2013-05-20handle_data:34INFOget_data 4.22501564026 ms  
2013-05-20handle_data:34INFOget_data 4.46891784668 ms  
2013-05-20handle_data:34INFOget_data 4.20308113098 ms  
2013-05-20handle_data:34INFOget_data 4.75120544434 ms  
2013-05-20handle_data:34INFOget_data 4.15802001953 ms  
2013-05-20handle_data:34INFOget_data 4.22501564026 ms  
2013-05-20handle_data:34INFOget_data 4.22096252441 ms  
2013-05-20handle_data:34INFOget_data 4.20904159546 ms  
2013-05-20handle_data:34INFOget_data 5.10597229004 ms

So, it seems that the batch transform runs considerably more slowly in combination with set_universe. Is this expected?

Grant

Thomas Wiecki
May 22, 2013

Hi Grant,

That's indeed pretty slow. The reason is probably that with a universe we set up an internal data panel that holds all sids that will be used during the backtest. Those might be quite a lot. Not sure there's an easy fix for that but I'll think some more about it.

Thomas

Grant Kiehne
May 23, 2013

Thanks Thomas,

Please note that in my test results above, even with a relatively small universe of securities, the batch transform execution time is ~ 250-350 ms (see "1970-01-01initialize:18INFOuniverse is 92 to 92.2"). This duration is similar to that of the executing the batch transform on a list of sids, prior to your recent speed-up. With a universe, are you perhaps re-creating an entire data panel every tic, rather than using the new iterative one? Just a guess...

Grant

Thomas Wiecki
May 23, 2013

How many Sids are in this small universe? You'd have to include all sids ever to appear in this universe too, might that make it larger?

I don't think it's that but it does seem curious. I'll have to profile the code.

Grant Kiehne
May 24, 2013

Hello Thomas,

In the attached example, there are 14 SIDS in the small universe:

1970-01-01initialize:18INFOuniverse is 84 to 84.2  
2013-05-21handle_data:34INFOget_data 220.268964767 ms  
2013-05-21PRINT ----------------------------  
2013-05-21PRINT2013-05-21 20:00:00+00:00  
2013-05-21PRINT ----------------------------  
2013-05-21PRINT SIDS & No. of SIDS  
2013-05-21PRINT[8352, 2470, 33127, 4809, 25002, 36907, 624, 4220, 15638, 20759, 40376, 19898, 22844, 1374]  
2013-05-21PRINT14  
2013-05-21PRINT Current prices  
2013-05-21PRINT[ 42.31 53.4 25.35 15.73 47.27 43. 55.63 82.36 112.59 63.61 46.69 96.64 22.47 13.67]  
2013-05-21PRINT Trailing price window  
2013-05-21PRINT[[ 42.2975 53.37 25.274 ..., 97. 22.4 13.7 ] [ 42.26 53.2 25.274 ..., 97. 22.38 13.69 ] [ 42.26 53.2 25.274 ..., 97. 22.41 13.69 ] ..., [ 42.31 53.365 25.34 ..., 96.7105 22.47 13.66 ] [ 42.31 53.395 25.34 ..., 96.7405 22.47 13.66 ] [ 42.31 53.4 25.35 ..., 96.64 22.47 13.67 ]]  
2013-05-21PRINT Trailing window length  
2013-05-21PRINT390

It takes > 200 ms for the batch transform to return price data.

Hope this helps.

Grant

Grant Kiehne
Jun 8, 2013

Hello Thomas,

Thought I'd follow up on this issue. Any insights? Let me know if you need more information.

Thanks,

Grant

Thomas Wiecki
Jun 8, 2013

Hi Grant,

I'm actually working on some batchtransform changes that might help with this (but I can't say for sure that it will). Stay tuned!

Thomas

Thomas Wiecki
Jun 24, 2013

Hi Grant,

Can you try again to see whether the recent fix sped up the batchtransform?

Thanks,
Thomas

Grant Kiehne
Jun 24, 2013

Hello Thomas,

Still pretty slow:

1970-01-01initialize:18INFOuniverse is 51 to 51.2  
2013-06-20handle_data:34INFOget_data 201.071977615 ms  
2013-06-20PRINT ----------------------------  
2013-06-20PRINT2013-06-20 20:00:00+00:00  
2013-06-20PRINT ----------------------------  
2013-06-20PRINT SIDS & No. of SIDS  
2013-06-20PRINT[39714, 28327, 42413, 7597, 22865, 25682, 28149, 42838, 33431, 39000, 39227, 27740, 43583]  
2013-06-20PRINT13  
2013-06-20PRINT Current prices  
2013-06-20PRINT[ 21.26 35.4197 22.23 5.33 13.58 2.98 59.51 26.2999 3.65 24.6345 2.92 22.87 18.23 ]  
2013-06-20PRINT Trailing price window  
2013-06-20PRINT[[ 21.21 36.4299 23.41 ..., 2.96 23.09 nan] [ 21.21 36.4299 22.82 ..., 2.97 23.09 nan] [ 21.21 36.4299 22.82 ..., 2.96 23.09 nan] ..., [ 21.274 35.311 22.35 ..., 2.92 22.85 18.2299] [ 21.274 35.311 22.36 ..., 2.92 22.88 18.2299] [ 21.26 35.4197 22.23 ..., 2.92 22.87 18.23 ]]  
2013-06-20PRINT Trailing window length  
2013-06-20PRINT390

Appears to be running at about 200 ms per call to the batch transform (which grabs a 1-day minute-level trailing window for each sid).

So, there appears to be no improvement over my most recent May 23, 2013 posting above.

Hmm?

Grant

Thomas Wiecki
Jun 25, 2013

Huh, indeed. I tested your earlier version a couple of days ago and it was much faster. But I can't replicate that now. I'll have to investigate...

Dennis C
Jun 25, 2013

@Thomas, are you benchmarking batch transform vs. the other method? I find that server load can affect my runtime by an order of magnitude at heavy load.

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.