Jamie, what is Quantopians recommendation for 'sequencing' backtests now? I've only recently joined and am currently implementing my strategies based on futures instruments which will usually have 2-4 parameters for which I'm interested in their best combination, let's say per sector.
Is there any alternative on Quantopian other than run, save the logs and restart the backtest algos manually?? Given the number of possible parameter combinations and different markets I hope the answer isn't 'yes' ;) Thanks, Sven

Hi Sven,

Other than futures pricing data, are you using any other data in your research? If you'd like you can email in to [email protected] to answer privately.

Cheers,
Jamie

Hi Jamie,
thx for quick turnaround.

No other data, my strategies are 100% price data based (which by the way is a challenge currently for futures with low tick sizes such as JY, here the daily data precision on Q is way too low, needed to compile my daily data out of minute data).

Coming to Quantopia I was planning to implement strategy cores in the backtester and do the fine tuning in research by optimizing the parameter levels. Currently I'm investing setting up a zipline environment locally in case that can't be done anymore.

Your advice on what options I have here is very welcome.

Cheers & Regards,
Sven

Hi Jamie -

From an architectural standpoint, this seems like a step backwards. I would think that improving integration of the backtesting engine with the research platform would be the way to go. For example, being able to launch parallel backtests parametrically from the research platform, and then analyze them in batches would seem to be a use case of interest to the community.

I think what Night Hawk is asking is how to launch N backtests to sample a parameter space, create a response surface, and optimize. This use case is not well-supported. One way to do it is to randomly sample the parameter space (with a random number generator in the code), and record the parameters with the record function, whose output is available in the research platform. Each backtest needs to be launched manually, by clicking a button in the browser. And then the backtest IDs need to be collected, and pasted into the research platform.

So, it is do-able, but for some reason, not well-supported, when it would seem to be a common use case for a platform that supports running an unlimited number of backtests in parallel. What are the impediments to supporting the use case? Or perhaps it would not be that useful to the broad community? I understand that running Zipline in a notebook is not the way to go, but not providing an alternative doesn't make sense to me.

Hi Grant, Jamie,

just adding that for my purposes hitting the backtest button manually is not do-able for practical reasons. Why? Typically you are addressing 30-40 different futures in different sectors that you are trading in parallel (A) with some strategies across the board and (B) with specific strategies that may be only suitable for some sectors (such as FX, agrar, metals, ...). You'd do that in order to smooth out drawdowns, increasing your return on account (ROA).

Now, given the number of your strategies (let's assume 3 across all sectors and 2 additional ones per sector) you'd have 40 futures times 5 strategies times # of your parameters/strategy. This is not going to work with manually kicking off backtests and then read out the logs in research with the backtest handles.

For this you'd need to run the inidivual tests batch-wise (through parameterized loops) and collect their test results in arrays or data frames for further analysis.

I'm not a great expert in equity based strategies (other than indices) but I'd be surprised if somebody working in the futures area would be able to test strategies without such feature.

I'd appreciate if such option would be made available here.

Thanks!

Hi Jamie,

If the 'zipline.TradingAlgorithm' is removed, that means one can't do the automated backtesting as described in the following link, right?

https://www.quantopian.com/posts/automated-backtests

Jamie, running automated, parameterized backtests is a very important feature. Did you plan to support automations in backtester?

Hi,

How about the answer to my question? Can we still run the optimation as before?

Up! I am still waiting for the answer from Q.

So nobody found a way to backtest using research? Has anyone done this locally? I cant get mine to work

Cemal,
Quite a bit of setup, yet works well!
See @marketneutral, J.Larkin's Alphatools
https://github.com/marketneutral/alphatools
alan

Hey Alan,

Thank you for the link, ill check it out more thoroughly . I am travelling atm, but I will get back to you!

Kind regards,

Cemal