As far as I understood from Q is that the context variables will be saved and re-instated when it recovers after a crash

That would be great if true. I'm hoping a Q employee sees this and can verify. Thanks!

Hey Q people, can you confirm or deny this? I've been searching the documentation trying to find the answer to this question. Are context variables persisted so that during an interruption of service (for whatever reason) they are restored when the service resumes? Or, even better (if possible), can you point us at where in the documentation this is described?

I was thinking about this more last night. The most robust solution is to assume the context is not persisted.

To keep our algo's simple, we can have a separate method that does the initialization steps. We would have a flag in the context that we set to true when we initialize, then just check it at the beginning of the init method and only do it if that flag resolves to false. (If it's not set, python should resolve it as false.) To make sure we're always initialized, we'd call that initialize method at the start of any triggered or scheduled call.

So the initialize method might look like:

def special_preparation_work(context, data):  
    if not 'is_initialized' in context:  
        # blah blah and blah  
        # do all the magic initialization stuff here  
        # blah blah and blah  
        context.is_initialized = True

def before_trading_start(context, data):  
    special_preparation_work(context, data)  
    # continue on with stuff

def handle_data(context,data):  
    special_preparation_work(context, data)  
    # continue on with stuff  

That will ensure the things you rely on are in the context while still keeping things from getting too convoluted.

Update: If you want to do something like this, you have to check for key existence rather than using truthy. Otherwise Python will throw a KeyError. I've updated the code example with that condition.

Thanks for your input Topher, for the record I asked Q support and they definitely said context does not persist. Which is unfortunate.

Hi Rick,

I think there may have a been a miscommunication somewhere in your ticket with Q support. Let me try to clarify what happens with context in live trading:

On the first day of live/paper trading of an algorithm, initialize is run, your context variables are set, and they persist for the remainder of the day. Overnight, the state of context does not persist, however, every subsequent trading day, before market open, your algo undergoes a process that we call 'warmup'. In the warmup process, live/paper algos re-simulate their entire history, from the first day they were run live to today, in order to re-gain the state of the algo (including context). So while the state of context technically doesn't persist each night, the re-simulation process achieves the same goal of making sure your algo can work with the state from the end of the previous day.

In the event that your algo stops due to some sort of technical problem with our live trading infrastructure, we have the ability to 'continue' your algorithm which allows it to continue running on the next trading day. When an algorithm is continued, it will undergo the usual warmup process on the next trading day to re-gain its state. This is not the same as if you manually stop your algorithm and restart it. Currently, there's no way to preserve the state of an algo if you stop and restart it.

Does this help clarify things?

That does clarify things! Sorry, the way the person who answered my ticket put it, made it sound like vars always started fresh. This is much better. So no trading happens overnight? Not even when you attach an IB account that is allowed to trade outside RTH?

That's correct. Quantopian algorithms only run during market hours.

Hi Jamie,

Presumably every time the algo is "warmed up" it runs on the current version of the API (not the one when the algo was launched into live trading), correct?

Also, the "warmup" is done on the current data sets, with any corrections applied after the launch of the algo into live trading? Or do you retain bad data, as it would have been encountered by the algo in live trading?

Just trying to sort out if live trading (with play money) is a cobbling together of backtest (prior days) plus the "live" intraday updates, or something else.

By the way, you might just include such live trading details in the help docs.