Hi,
I want parallel execution of a for loop like Matlab. What is the easiest way to make a normal for loop parallel using Qutopian provided libraries?
S
Hi Suminda,
There are no current plans to add this. It's not straight forward because of the python GIL and there is no module that provides this functionality. Maybe in the future we will allow numba that can speed up for loops.
Thomas
Disclaimer
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Hi Thomas,
I just ran one of my trivial algorithms in 10 browser tabs, by sequentially clicking the "Run Full Backtest" button in each tab. So, it appears that Quantopian can support some form of batch processing. How feasible would it be make it more user-friendly? For example, say I'd like to run an algorithm for a range of parameters x_n = 1,2,3,4,5. Presently, for each x_n, I'd need to edit the code, and then manually run it. It would be preferred to be able to submit a table of values of x_n (and in general, y_n, z_n, etc.) and then execute the batch of backtests (either in parallel or serially). The results would be output in a user-defined table.
Best regards,
Grant
Hi Grant,
Yes, that's certainly one way to do it and you are right that we already solved many of the parallelization problems required for this. I'm currently evaluating hyperopt (jaberg.github.com/hyperopt/) which allows for very free specification of a search space and more clever algorithms to optimize it (including simple grid-search). Certainly we want that but it doesn't make a lot of sense to work on a hackish version that works in one particular case but isn't very general. Instead, I prefer to do it right the first time.
Thomas
Disclaimer
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Hello Thomas,
Being able to run an efficient optimization routine would be great. I was thinking more along the lines of making it convenient to explore the characteristics of an algorithm versus adjustable parameters (including security choice). As a specific example, for parameters x_n & y_m and n and m both ranging from 0-9, I could generate a rough heatmap of the backtest return (or other performance/risk metric) by running 100 backtests. I just need an input table for x_n & y_n and an output table for the n x m = 100 return values. Then I could download (or copy and paste) the table for further analysis and visualization.
What's the thinking at Quantopian regarding facilitating this batch mode of backtesting? It's already possible manually, just very awkward...I'm not sure Firefox will support 100 tabs all streaming data from Quantopian update plots and performance results! : )
Cheers,
Grant
Hi Grant,
I see. Well what I'm proposing would give you that as well (it's then a matter of plotting). Do note that there are many design decisions and implementational costs associated with even doing the apparently simple thing you are proposing. In any case though, I agree that it's a nice feature and we definitely want it :).
Thomas
Disclaimer
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Thanks Thomas,
In my mind, Quantopian should strive to provide tools and access to data that are comparable to that of institutional algorithmic traders (more of a desktop computing environment). By relaxing the paradigm of getting instant "point solution" feedback (via the browser gui), I think that you could do a lot more. For example, I can imagine submitting a batch job and then getting an e-mail notification minutes/hours/days later that my results are available for download and analysis. Perhaps the costs could be kept in check by being able to take advantage of periods of low load on your vendor's server.
Grant
Please sign in or
join Quantopian to
post a reply.
Need help?
Contact support.
Company
Learn & Support
Notices
Follow Us
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.