Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
5 Minutes Execution Time Limit

Hello:
I would like to find out if the multiprocessing library is available for us because I could not import it at this point. My trading algorithm needs to analyze the trading characteristics of each underlying asset for the past 24 ~ 72 hours each day after market is closed. Obviously, 5 minutes allowed execution time limit is not enough to perform such computational task for the hundreds of selected assets. Nonetheless, I noticed that I am able to limit analyzation process for each given asset within 30 seconds. The multiprocess.process (or .pool) could execute the analytical process ( 30s or less in execution) in parallel, as such it could handle the entire task under 5 minutes. I understood that allowing multiprocessing for everyone on Quantopian platform might hammer the system resources and most likely increase the cost. What if the number of subprocess is restricted, say around 30 subprocess for each per user job. It might be able to mitigated impact to the system resources. It would help to overcome this 5 minutes computational bottleneckā€¦
Thanks,

Frank