checkm lineage_wf memory usage about checkm

[donovan-h-parks]

Hello,
This is a bit of a concern. It appears to be an issue with how pplacer handles threads. On our system, I can see that CheckM is using 13.9G with the -r options, but that right when pplacer is finishing it appears to spawn a number of threads which all indicate they are using 13.9G. I suspect they aren't actually using this memory, but perhaps it is being detected as such by PBS.

Can you up the requested memory threshold for PBS? It would be helpful to know if this is at least a temporary solution.

Cheers,
Donovan

[xvazquezc]

Hi Donovan,
I increased the memory up to 216GB using 2 nodes in the cluster and it completed without problems. The log indicates it used about 135GB of memory.
Cheers,
Xabier

[donovan-h-parks]

Hello Xavier,

Glad to hear it worked. The peak memory usage does appear to be correct and is due to pplacer. I have released a new version of CheckM (v1.0.5) that allows the number of threads used by pplacer to be set. This can be used to substantially reduce the peak memory requirement (back to 14G if pplacer is only given 1 thread), at the cost of some extra computation time.

Cheers,
Donovan