I too would like to have this added to the supported modules, but the Quantopian team has delayed in adding it for a while now (~2 years). I inquired about this about a month ago and they informed me that it currently would only be possible to use it in their Research Environment. However, this obviously doesn't help much since using the module within a live trading strategy is the main purpose of needing it.

I have resorted to using scipy.optimize as well and I also need CVXOPT for efficient sparse matrix solvers. The former is noticeably slower in comparison and much nastier in regards to quadratic programming implementations.

Is it a matter of slow backtests, or are you exceeding the ~50 second time-out of handle_data under live trading? --Grant

It is a matter of exceeding the timeout. For example, I want to optimize some function by running multiple scenarios where the number of scenarios should be large, but I am restricted to a certain scenario count due to the restricted time. This is a problem because the less scenarios I run, the less accurate the solution in regards to the target function.

Ah...so can you carry over intermediate results, so that the computation is done over multiple calls to handle_data?

I haven't tried that as of now, but that should work since the scenarios are independent of one another. I guess I would have to save the intermediate results within some structure stored in the context, which is a hack that could be easily avoided with the inclusion of the CVXOPT module. Another good reason to use CVXOPT is because it has a much easier API interface, especially when considering equality/inequality constraints, boundary values, and data structures.

Yeah, I worked my way through an example (https://www.quantopian.com/posts/long-only-minimum-variance-portfolio-using-scipy-dot-optimize-dot-minimize) and I'd agree that scipy.optimize has a learning curve. Aside from a more efficient optimization module, there are a couple other issues:

1. The 50-second time-out seems overly restrictive. It should be possible to have background processes running over multiple calls to handle_data (and even overnight).
   
2. Your problem sounds well-suited to some form of parallel processing. One should be able to launch all trials at once, and then collect the results for global optimization. I wonder how many processor cores and how much memory is available to each algo? Perhaps they could be allocated (up to some limit) when the algo is initialized?

It would be interesting to hear from Quantopian regarding plans for 1 & 2 above.

Grant

Hi Desmond, Grant, and Grant,

Great news, the CVXOPT module was added to the whitelist last night.

We'll have an example algo soon, but wanted to let you know it's there now.

Have fun!

KR

That's great to here Karen! Thanks! It would also be nice if the Quantopian team could tackle points 1 and 2 from (other) Grant's most recent post :)

I think I can speak on behalf of all the other Grants on Quantopian when I say that this is great news.

Great stuff. Thanks Karen !

Thought I'd try the example code give on http://cvxopt.org/examples/tutorial/qp.html. But, when I try to run the code below, I get:

14 Error Runtime exception: ImportError: cannot import name blas

Any ideas how to get it to work?

from cvxopt import matrix, solvers

def initialize(context):  
    context.spy = sid(8554)  
def handle_data(context, data):  
    Q = 2*matrix([ [2, .5], [.5, 1] ])  
    p = matrix([1.0, 1.0])  
    G = matrix([[-1.0,0.0],[0.0,-1.0]])  
    h = matrix([0.0,0.0])  
    A = matrix([1.0, 1.0], (1,2))  
    b = matrix(1.0)  
    sol=solvers.qp(Q, p, G, h, A, b)  
    print sol['x']  

I have also attempted to implement something similar and I receive the following error:

ImportError: cannot import name misc

I was informed last week that it would be looked into, so hopefully the issue will be addressed soon.

We'll take a look to see what's going on!

did somebody get an algo to work with the CVXOPT module

Nope.

Nope, same problem as the Grants.

Did anyone ever get this to work?

Hi Grant K, it works in the research environment but not in live trading yet.

Any update on this? Why is it not working in live trading yet? --Grant

The CVXOPT module isn't currently available, we need to fix some things in the backend. When it's ready, we'll post an update in this thread.

I'm not sure why but it seems like cvxopt is now working on Quantopian (at least for me it did) -- yay! But do let us know if you encounter any problems.

I still get an error for the code below:

14 Error Runtime exception: ImportError: cannot import name misc

from cvxopt import matrix, solvers

def initialize(context):  
    context.spy = sid(8554)  
def handle_data(context, data):  
    Q = 2*matrix([ [2, .5], [.5, 1] ])  
    p = matrix([1.0, 1.0])  
    G = matrix([[-1.0,0.0],[0.0,-1.0]])  
    h = matrix([0.0,0.0])  
    A = matrix([1.0, 1.0], (1,2))  
    b = matrix(1.0)  
    sol=solvers.qp(Q, p, G, h, A, b)  
    print sol['x']  

Pretty odd, does the cloned algorithm from https://www.quantopian.com/posts/the-efficient-frontier-markowitz-portfolio-optimization-in-python-using-cvxopt work for you? It might somehow not call the optimization I suppose.

It runs (see attached). --Grant

I need the cvxopt module. It is officially available now in the backtester?

Attached backtest may demonstrate a workaround for the import error.

At least, Grant's test runs with it and another cvxopt algo I have works in the backtester.

I added a run once latch like is done in zipline to warm up history before calling cvxopt.

How I found this. I noticed I had left in history warm up code that is only needed in zipline in a backtest I posted to this thread:
https://www.quantopian.com/posts/the-efficient-frontier-markowitz-portfolio-optimization-in-python-using-cvxopt When I removed it, I got the import error.

Richard

http://quant-coder.prokopyshen.com/ContactMe

Providing custom quant programming, tutoring, linux cloud support and troubleshooting services.