nanobenchmark: RaiseThreadPriority seems to have adverse effects? about highwayhash

[lorenzhs]

Well after playing around with it a bit more I now have a binary for the original case that is much slower, compiled with the same flags and all (but not identical). I really don't understand what's happening any more.

Running on CPU #1, APIC ID  2
TimerResolution32 175
NumReplicas 3166
 3: median= 15.8 cycles; median abs. deviation= 0.0 cycles
 4: median= 12.0 cycles; median abs. deviation= 0.1 cycles
 7: median= 11.9 cycles; median abs. deviation= 0.1 cycles
 8: median= 15.2 cycles; median abs. deviation= 0.0 cycles

...I'm thoroughly confused.

[jan-wassenberg]

Thanks for getting in touch! This is really strange. I see the same on our machines. With sudo+RaiseThreadPriority:
Resolution<32> 112 NumReplicas 747 3: median= 60.3 cycles; median abs. deviation= 0.5 cycles 4: median= 23.3 cycles; median abs. deviation= 0.8 cycles 7: median= 23.5 cycles; median abs. deviation= 0.4 cycles 8: median= 79.0 cycles; median abs. deviation= 0.5 cycles
and sudo without RaiseThreadPriority:
Resolution<32> 112 NumReplicas 1652 3: median= 20.9 cycles; median abs. deviation= 0.1 cycles 4: median= 8.9 cycles; median abs. deviation= 0.1 cycles 7: median= 9.0 cycles; median abs. deviation= 0.1 cycles 8: median= 27.6 cycles; median abs. deviation= 0.1 cycles

The NumReplicas also indicates the initial measurement took twice as long in the first case.
With priority max-1, we see a bit less slowdown and lower variability:
Resolution<32> 112 NumReplicas 1606 3: median= 31.0 cycles; median abs. deviation= 0.5 cycles 4: median= 12.7 cycles; median abs. deviation= 0.3 cycles 7: median= 12.8 cycles; median abs. deviation= 0.3 cycles 8: median= 41.4 cycles; median abs. deviation= 0.1 cycles

With priority max/2, it's worse again:
NumReplicas 776 3: median= 59.2 cycles; median abs. deviation= 1.2 cycles 4: median= 23.7 cycles; median abs. deviation= 0.5 cycles 7: median= 23.6 cycles; median abs. deviation= 1.1 cycles 8: median= 78.4 cycles; median abs. deviation= 0.7 cycles

SCHED_FIFO instead of RR and max-1:
NumReplicas 782 3: median= 59.5 cycles; median abs. deviation= 0.6 cycles 4: median= 23.5 cycles; median abs. deviation= 0.7 cycles 7: median= 23.2 cycles; median abs. deviation= 0.6 cycles 8: median= 78.1 cycles; median abs. deviation= 0.6 cycles

and finally FIFO, max/2:
NumReplicas 963
3: median= 59.5 cycles; median abs. deviation= 0.8 cycles 4: median= 24.1 cycles; median abs. deviation= 0.6 cycles 7: median= 24.3 cycles; median abs. deviation= 0.7 cycles 8: median= 78.3 cycles; median abs. deviation= 0.5 cycles

and without RaiseThreadPriority we're back to 20/9/9/27 again. Very mysterious - if anyone has a possible explanation, I'd love to hear it. In the meantime, we'll be skipping RaiseThreadPriority on Linux.

[lorenzhs]

Thanks for confirming that it's not just my machine acting up :) Very mysterious indeed, please let me know if you ever find out what's causing it.

[jan-wassenberg]

Will do :) Let's leave this issue open until we have some new insights.

[jan-wassenberg]

Update: all our performance tests since then have not changed thread priority. I still have no explanation why it increases variability but am OK with omitting it on Linux.