Has anyone done any measurements of the maximum elapsed time per Pipeline-day beyond which you start to risk backtests timing out?
I am working on a strategy which had a Pipeline calculation that took about 18 seconds in Research for a single day (297 seconds for 6 months), yet timed out in backtests fairly quickly (after a couple of years), presumably because it's pre-calculating 6 months worth of data at a time.
I optimized it a bit, so now it only takes 6 seconds in Research for a single day (46 seconds for 6 months) and so far so good.
Everyone's ratio of one day vs six month time cost will be different, but I thought I would throw this out there. For what it's worth, the major optimization was replicating the market cap universe code inside my linear regression factor so that I only ran regressions on those stocks which were ultimately going to be screened in. I'll share the code tomorrow or next week.
