Why the gap in cell output data?

posted Jul 21, 2018

Hello Quants! New here :)

Why is there a gap in the data output list?

2018-07-19 14:00:00+00:00 3.225
...
2018-07-19 19:31:00+00:00 2.774

How can I get the full list?
Thank you!
Frank

In [1]: get_pricing('MTSL', start_date="2018-07-19", end_date="2018-07-19", fields='close_price', frequency='minute')

Out[1]: 2018-07-19 13:31:00+00:00 2.959
2018-07-19 13:32:00+00:00 3.125
2018-07-19 13:33:00+00:00 3.353
2018-07-19 13:34:00+00:00 3.491
2018-07-19 13:35:00+00:00 3.370
2018-07-19 13:36:00+00:00 3.310
2018-07-19 13:37:00+00:00 3.430
2018-07-19 13:38:00+00:00 3.390
2018-07-19 13:39:00+00:00 3.350
2018-07-19 13:40:00+00:00 3.580
2018-07-19 13:41:00+00:00 3.669
2018-07-19 13:42:00+00:00 3.470
2018-07-19 13:43:00+00:00 3.600
2018-07-19 13:44:00+00:00 3.520
2018-07-19 13:45:00+00:00 3.520
2018-07-19 13:46:00+00:00 3.280
2018-07-19 13:47:00+00:00 3.200
2018-07-19 13:48:00+00:00 3.249
2018-07-19 13:49:00+00:00 3.289
2018-07-19 13:50:00+00:00 3.191
2018-07-19 13:51:00+00:00 3.110
2018-07-19 13:52:00+00:00 3.130
2018-07-19 13:53:00+00:00 3.040
2018-07-19 13:54:00+00:00 3.080
2018-07-19 13:55:00+00:00 3.070
2018-07-19 13:56:00+00:00 3.170
2018-07-19 13:57:00+00:00 3.180
2018-07-19 13:58:00+00:00 3.176
2018-07-19 13:59:00+00:00 3.150
2018-07-19 14:00:00+00:00 3.225
...
2018-07-19 19:31:00+00:00 2.774
2018-07-19 19:32:00+00:00 2.835
2018-07-19 19:33:00+00:00 2.830
2018-07-19 19:34:00+00:00 2.846
2018-07-19 19:35:00+00:00 2.860
2018-07-19 19:36:00+00:00 2.854
2018-07-19 19:37:00+00:00 2.860
2018-07-19 19:38:00+00:00 2.801
2018-07-19 19:39:00+00:00 2.818
2018-07-19 19:40:00+00:00 2.810
2018-07-19 19:41:00+00:00 2.800
2018-07-19 19:42:00+00:00 2.760
2018-07-19 19:43:00+00:00 2.832
2018-07-19 19:44:00+00:00 2.821
2018-07-19 19:45:00+00:00 2.810
2018-07-19 19:46:00+00:00 2.819
2018-07-19 19:47:00+00:00 NaN
2018-07-19 19:48:00+00:00 2.813
2018-07-19 19:49:00+00:00 2.816
2018-07-19 19:50:00+00:00 2.825
2018-07-19 19:51:00+00:00 2.822
2018-07-19 19:52:00+00:00 2.840
2018-07-19 19:53:00+00:00 2.851
2018-07-19 19:54:00+00:00 2.870
2018-07-19 19:55:00+00:00 2.920
2018-07-19 19:56:00+00:00 2.950
2018-07-19 19:57:00+00:00 2.860
2018-07-19 19:58:00+00:00 2.850
2018-07-19 19:59:00+00:00 2.920
2018-07-19 20:00:00+00:00 2.950
Name: Equity(16933 [MTSL]), dtype: float64

7 responses

Jul 21, 2018

Welcome!

There isn't really a gap in the data. Pandas default display detects if the series or dataframe is quite long and then shows just the top and bottom portion of the data. All the data is there it's just shown that way for readability (what if the series had a million rows).

A convenient way to view all the data in a scrollable format is to use qgrid. See the attached notebook.

Thanks a lot Dan, it works?

Is it possible to copy/paste this qgrid or save it as a text or Excel file?

Regards,
Frank

It's not very easy to cut and paste large amounts of data from a notebook. Also, one can't export to a file. This is intentional, I believe, because of the agreements Quantopian has with their data providers.

Ok I understand, thanks.

Last question: Why is the timestamp of this data delayed by 4 hours?
F

The timestamps by default are in UTC and not US/Eastern which is the US market time. One can convert to US/Eastern time with the 'tz_convert' method.

prices = prices.tz_convert('US/Eastern').tz_localize(None)

The 'tz_localize(None)' method then makes it timezone unaware (and forever in Eastern time). For some reason qgrid doesn't respect the timezone so this trick makes it render properly.

Here is an updated notebook with the timezone converted.