It seems like there might be some improvements to be had in software, without adding the cost of more memory.

Alternatively, if the entire honkin' backtest dataset is not actually needed by Pyfolio, then perhaps a subset of it could be loaded, or pre-processed prior to the being loaded (e.g. get_backtest(usage = 'pyfolio', frequency = 'daily'). For example, in most cases, I'd think that daily returns would be sufficient, but as I understand the entire dump of minutely data is provided to the research platform.

One way potentially to manage things would be to release all of the memory consumed by the backtest, and then use it to pre-process the backtest data into various more compact forms, as a standard (or user-specified) backtest step.

Also, the backtester should probably issue some sort of warning if the result is incompatible with the research platform due to memory constraints, or at least report the overall size of the backtest dataset, upon completion. As I recall, too, the research platform just chugs along loading the backtest, until it runs out of memory. It could check the size first, and if too big, then just report an error without attempting to load the data.

However, adding RAM on a per user basis month-by-month basis for a fee would seem straightforward, given that Quantopian is all set up to offer premium services (e.g. data). Maybe there just isn't that much demand for it?

Another potential software assist would be to look at the necessity of using double precision for backtest data. This could be another switch:

get_backtest(usage = 'pyfolio', frequency = 'daily', precision = 'single')

But in the end, there's a trade, since engineering resources are expensive, and there is an opportunity cost, too. So, at some point, it might make sense just to boost up the RAM.