My benchmark results about rust-blog

[pretzelhammer]

I've updated the article to use a DigitalOcean linux VPS as my benchmark test environment since that's much more realistic environment than a macbook pro. The benchmarks should be more accurate to real-world use-cases, and switching to a linux environment did significantly improve the performance of the DR server. Thanks for your help guys! I'm gonna close this issue.

[pretzelhammer]

Wow, those throughputs are amazing! Your test machines seem to have specs similar to mine, I'm also confused why the numbers you are getting are so much better. Did you make any code changes to the SA or DR code in any way?

1 - Why is it completely different from what we have here in the blog post?

The only noticeable difference between your setup and mine seems to be that you are running the benchmarks on a Linux machine (with PostgresQL in a docker container running Linux) and I'm running them on macOS (with PostgresQL in docker container running Linux). Docker has to do more work to simulate a Linux environment on a macOS machine than a Linux machine, of course... so it's possible I'm losing out on some performance that's getting lost in the virtualization layer... but even then, the difference in performance is so massive it's hard to believe...

That's my best guess, but even I'm not even happy with that guess. Hopefully someone else can chime in? It would be interesting to see more people run the benchmarks on their machines and share what results they get, maybe we can find some pattern in all this chaos.

2 - Why DR is so much different on these two machines?! (I ran them several times, it's almost the same)

No clue.

[omid]

This difference is the root of my confusion too! I did some changes and saw the difference and thought it's because of my changes, but nope! It's exactly what you have in the main branch.

The only noticeable difference between your setup and mine seems to be that you are running the benchmarks on a Linux machine...

Yep, but it should be the same, for all envs. So the same performance boost :/

[weiznich]

I believe I can explain at least some of that differences.

1- Why is it completely different from what we have here in the blog post?

This is likely caused by macOS having being slow at using many threads compared to linux. See here for details.

2 - Why DR is so much different on these two machines?! (I ran them several times, it's almost the same)

Did you use the default configuration provided by rocket, or the configuration provided by @pretzelhammer example? The likely cause here is that rocket is quite sensitive to the number of workers. Increasing this number should increase performance (at least till some point). Now it seems that your second test machine has more logical cores available + the optimal number of worker threads is depended on the core count. This suggests that you need to increase that number of worker threads even further to get comparable performance. I would try something like doubling the number to 512.

[omid]

Thanks @weiznich the first answer looks logical 🙏🏼

About the second one, I tried with 512, saw a worse result, like 180K.
Set it to 32 (logical CPU * 2, based on their documentations) saw a similar result as 218K.