Hi! I'm trying to train a deep neural network offline before using it in my algorithm. After architecting it I'd feed it the chosen inputs from Quantipian's data, let it run (preferably on a GPU) for several minutes to hours, then import the training parameters into my algorithm for trading and backtesting.
From what I can tell there are a few major roadblocks to this:
There are no deep learning libraries supported by Quantopian. Scikit doesn't allow for flexible deep architectures and doesn't support GPU training, both are needed for interesting work. I can work around this by implementing them by hand, but then I run into the next issue.
Quantopian doesn't allow for GPU training, and doesn't publicize the specs or allowed CPU time. Can I just churn indefinitely on a notebook? That could work, but it would be hundreds of times slower than using a GPU and tens of times slower than using a dedicated CPU.
Assuming I get past the other barriers, I'd need to export up to tens of thousands of parameters from the notebook and into my algorithm. Is there a way to do this?
If I decide to work offline with Zipline, I'd need to find a good data source. This isn't easy, one of the major perks of using Quantopian is their data. Also, it's unlikely that data trained from a different source would respond correctly to Quantopian data. Unless I'm missing something, this is likely a non starter.
Any ideas on the best way to approach this? Anything I'm missing? From what I can tell this use case is neglected right now.
