Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Online updates of Kalman filters with inhomogeneous (non-equal-time-step) observations
Simon Thornington
posted Jan 28, 2016
Machine Learning Mean Reversion Research Stat Arb Statistics

Hi folks,

Anyone seen any code for a Kalman filter which admits online updates with (basically) time stamps or time deltas? I'm taking another crack at intraday mean reversion, but I really want to handle overnight gaps better than assuming they are as important as intraday updates. In fact, I wouldn't mind assigning custom weights to each minute based on the volume traded (that would be awesome), based on the idea that we are sampling the underlying process at random/sparse times, and that some minutes' observations are after a greater length of time/action in the volume clock.

I really don't want to have to go back to first principles and see if I can derive the state space update formulae with generalized elapsed time, and frankly, I'm not sure if I am capable of that yet.

Simon.

3 responses
Simon Thornington
Jan 28, 2016

http://www.mathworks.com/matlabcentral/linkexchange/links/3530-kalman-filter-with-non-uniform-timesteps?requestedDomain=www.mathworks.com

Still researching...

Simon Thornington
Jan 28, 2016

Hmm playing with that code in MATLAB, it looks like it is still assigning the same weight to each observation, when really I want further spaced observations to be weighted higher, to reflect the fact that the underlying process has had more time to evolve.

Grant Kiehne
Jan 29, 2016

Perhaps you could fit the price versus time data with a polynomial, and weight the fit by volume. Then use the polynomial to interpolate, creating an equally spaced grid, thereby synthesizing 24/7/365 minutely prices at equal volume. The tricky parts are the length of the trailing window for the fit, and the order of the polynomial. In addition to weighting by volume, you could weight by time, something like exp(-t/tau), where tau is the time constant. Then, you can select a really long trailing window, and fiddle with tau instead. This will weight more recent observations more heavily (e.g. on Monday morning, the prior week will have faded from the analysis).

For completeness, you could include the full OHLCV data in the fit and weights (e.g. points with lower variance could be weighted more heavily).

Once you have your synthetic, equally-spaced price data, then you could try applying your Kalman filter thingy (which may be useless after the fitting...)

Overall, at any point in time, it seems like you need a figure of merit for the goodness-of-fit of your model, to assess the degree to which it can be applied (i.e. how much risk are you taking on by relying on the model).

About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.