SCTP: >10x BPF program runtime compared to TCP about loxilb

[nik-netlox]

Hi @luisgerhorst, This may be due to checksum calculation. TCP checksum calculation are offloaded but there is no support to offload SCTP checksum so loxilb calculates the SCTP checksum in its eBPF core engine. If you disable checksum calculations at your client and server side and run loxilb with --disable-csum option then you will see better performance numbers.

If you happen to see low throughput for multi-threaded iperf3 SCTP traffic then this may be due to RSS distribution in Linux. For SCTP, the traffic is not well distributed and goes to only couple of cores which leads to bottleneck. Though, we have a workaround where loxilb takes care of the RSS distribution. Please run loxilb with --rss-enable option then you will see better performance in that case too.

[luisgerhorst]

If you disable checksum calculations at your client and server side and run loxilb with --disable-csum option then you will see better performance numbers.

Thanks! As I am trying to have a realistic real world test, I assume it is better to keep it enabled. Or would you disagree? (i.e., do you think people do this frequently for SCTP?)

multi-threaded iperf3 SCTP traffic

At least on Debian 10, multi-threaded iperf3 does not seem to exist. Or were you referring to running multiple instances of the iperf3 against a single server?

man netperf:

-P, --parallel n
number of parallel client streams to run. Note that iperf3 is single threaded, so if you are CPU bound, this will not yield higher throughput.

This also explains why I could not reproduce the TCP iperf2 numbers with iperf3. Thus I decided to stick to single-threaded benchmarks as they should be sufficient to measure the overhead of my kernel BPF runtime patches.

Please run loxilb with --rss-enable option then you will see better performance in that case too.

As this does not appear to have any downsides I will do that, thanks!

[luisgerhorst]

Closing because the original issue has been resolved.