Hi Taylor,

The algorithm needs to have a top level handle_data method. So, you could instantiate your Kalman object in the initialize, and then pass updates into it from handle_data. Could you explain the parameters expected by the Kalman class' init and the updateState methods?

You would do something like this:

# assuming the Kalman class above is defined first in the script

def initialize(context):  
    context.stock = sid(26578)  
    # construct the Kalman class  
    context.kalman = Kalman(...)  
def handle_data(context, data):  
    # pass data to Kalman to update state  
    context.kalman.updateState(...)  

It seems like updateState is expecting matrix parameters - you may be able to use a batch transform to build those parameters. The algorithm would look something like this:

# assuming the Kalman class above is defined first in the script

def initialize(context):  
    context.stock = sid(26578)  
    # construct the Kalman class  
    context.kalman = Kalman(...)  
def handle_data(context, data):  
    # pass data to Kalman to update state  
    update_kalman(data, context.kalman)

@batch_transform(refresh_period=1, window_length=20)  
def update_kalman(datapanel, kalman):  
    # do stuff to prices to calculate the parameters for the update  
    kalman.updateState(...)  

thanks,
fawce

:::So, you could instantiate your Kalman object in the initialize, and then pass updates into it from handle_data.

Right. Just worried about defining the class right now.

:::It seems like updateState is expecting matrix parameters

Yeah. And batch transforms return pandas panels, right? So inside update_kalman() I'd have to define some auxiliary pandas stuff as matrices, then make the call to updateState, and then it returns more panels objects? Damn. You guys made the right choice using panels, but state space models are a bit difficult without matrices.

This is your @batch_transform function decorator, no? I may ask you about this later.

def batch_transform(func):  
    """Decorator function to use instead of inheriting from BatchTransform.  
    For an example on how to use this, see the doc string of BatchTransform.  
    """

    def create_window(*args, **kwargs):  
        # passes the user defined function to BatchTransform which it  
        # will call instead of self.get_value()  
        return BatchTransform(*args, func=func, **kwargs)

    return create_window  

::: Could you explain the parameters expected by the Kalman class' init and the updateState methods?>

State space models have a latent variable equation, and an observed variable equation. X is the latent state (think of it as a filtered true price/price vector), and P is the covariance of that filtered price/price vector. Both of these are conditional on all the observations (Y) that have been seen. The notation is a bit deprecated. The Kalman filter algorithm updates these two quantities at every minute. The way it does this is by using the Kalman filter equations. Derivation of these equations requires Baye's rule, and that theorem about the distributions when you condition bits of a joint gaussian vector on itself.

In the latent equation, X is assumed to be Markovian. Phi is its transition matrix. It's assumed to be fixed here. Q is the error term.

R is the error term in the observation equation. A would've been the matrix that transforms the the X into the Y, but I think I forgot to define it.

Hi,

Batch transforms receive a pandas panel, which is a keyed set of dataframes. The dataframe has a method to convert to a numpy matrix (doco here).
Your batch can return anything you wish, so you could return the matrix, or some output from the Kalman class.

thanks,
fawce

maybe something like this? not sure I understand everything completely.