polybeast is slower than monobeast about torchbeast

[heiner]

Hey Da-Capo, thanks for your report.

I think to start with a batch size of 4 isn't very large. The reason your CPUs are less busy for Polybeast is that the actor forward passes ("inference") happen on the GPU in that case. Options include:

• Increase batch size
• Use different GPUs for inference and learning
• Potentially increase the number of parallel inference and learner threads
• You can also play around with the unroll length

[MachengShen]

I'm having similar issue on a ubuntu machine with 32cpu cores, and 4 V100 gpus. with monobeast, it only uses 1 gpu, and full cpu power, the frame rate is ~5000SPS; while with polybeast, I set batch_size=16, num_inference/learner_threads=8, but the frame rate is only ~300SPS, and only 2 gpus are running. Were you able to speed up polybeast? Can you share some insight with me? Thanks!