Change default for MaxBandwidth to 1000 about vnstat

[vergoh]

That's indeed a valid question. MaxBandwidth at 1000 by default shouldn't be a problem if the kernel is using 64 bit counters for the interface. However, detecting the overflow of 32 bit counters may become harder if unless UpdateInterval is decreased. I'll have to at least check if a 32 bit kernel always results in 32 bit counters and if this change could result in any unwanted behaviour that I can't now quickly find of.

Obviously, the best approach would be to detect the correct value automatically, or at least what the NIC has negotiated, and then use that by default (if available). For example ethtool doesn't appear to require root permissions for the query.

[vergoh]

At least a 32 bit CentOS 6.5 with 2.6.32 kernel appears to have also 32 bit counters for interfaces.

[Lekensteyn]

An interface might not be configured when vnstat is started, so maybe it is more wise to use the maximum supported speed?

[vergoh]

I did some digging and it looks like getting the current interface speed value in both Linux and BSD is possible without too much effort. It's also possible to check if the interface link is up or not. However, not all interface types are supported but unsupported interfaces can also be detected. If the interface isn't configured at all then it isn't visible for vnStat either and that's already currently handled.

In order to make this feature configurable enough, there probably should be an option like AutomaticBandwidthDetection and when enabled, the current MaxBandwidth would act as a fallback value (and I could bump the default to 1000 too). Otherwise MaxBandwidth would be used directly if set to something else than 0. Setting it to 0 would disable the feature like it already does.

If the detection is enabled then during startup the daemon would try to read the current value if the link is up. If that fails then the fallback value is used and a retry scheduled for a later time. If a certain number of retries fail while the link is up then it would be assumed that the interface doesn't support giving the speed information. If the information is available then it most likely wouldn't hurt to check the value from time to time; BandwidthDetectionInterval in minutes, 0 disables and a default value of something between 5 to 30 would most likely be suitable.

[Lekensteyn]

Adding another option seems like to complicate it further, but it is indeed needed if you need to fallback to a value when auto-detection fails / is not available.

Does this auto-detection feature use the maximum supported bw from the NIC or the current negiotiated speed? In the latter case, it might trigger false positives when the speed is changed on Linux with ethtool and a high throughput is observed.

[vergoh]

The currently negotiated speed is the easiest to get. Ethtool appears to be able to also get those values that have been enabled for autonegotiation but that list turns empty if some specific value is forced. I'm not even sure if the autonegotiation list is available on the BSD side.

I was yesterday thinking that checking the current speed with some interval would solve the "high throughput peak after speed change" situation but now that I think of it, a better solution would be to recheck the speed also when a peak value that's higher than the current maximum is detected.

[Lekensteyn]

Or just drop the periodic check, and only check when the calculated throughput is higher than the expected maximum. This has a potential race where the link speed is temporarily increased and then decreased again though.

I think i prefer the static 1 Gb bandwidth without smart attempts to detect deviations. The only case where the stats are wrong is when an interface is removed and re-added (e.g. when the driver gets reloaded). In that case, the counter will much larger than 1e9 / 8 bytes (and still larger for 10 Gbit links).

So do you really think that adding this auto-detection is worth the additional complexity?

[vergoh]

Dropping the periodic check would result in the value being mostly only able to increase itself since there's no "check the link speed since the user doesn't transfer much data currently for some reason" kind of scenario. Do you have some specific use case in mind where the link speed would constantly fluctuate?

With a default MaxBandwidth value of 1000 you'll still also have to consider those users with a 32 bit kernel. For them, such default value would have the same effect as simply disabling the feature altogether and that's a bigger problem at least from my point of view. It's worth noting that vnStat doesn't have a way of knowing if the interface counter value getting processed is 32 bit or 64 bit when the value is below the 32 bit limit.

The latest version (1.12) shows extra instructions when the daemon is started in order to give the user a hint about the existence of a configuration file and the MaxBandwidth setting. However, with distributions starting to adopt systemd, those instructions are no longer directly visible when the daemon is started for the first time because systemd redirects all output to its (binary) log file. Another issue are also those distribution provided binary packages that modify the default settings. For example, the EPEL repository rpm appears to have MaxBandwidth set to 0 which ends up causing issues even for those who use 64 bit kernels. I'm not sure what the reason for the change has been but I'd guess it's something like "since we don't know the speed, it's better to disable the feature".

In the end, it appears to be better if the default values of options are such that they fit most users without changes since most users don't seem to change the settings anyway and would rather have a minimal amount of things to configure when installing something. Those users who want to be more in control or know the facts better would still be able to tune the settings. That's why I think this autodetection feature would be worth implementing if it ends up decreasing the number of users having a misconfigured install. For example, introducing the daemon in version 1.7 has already proven to have been worth the effort since it removed the hassle of configuring cron entries and additional support scripts (that many forgot).

Regarding complexity, I'm planning on doing a database format rewrite next once I'm happy enough with the current state of the software. That's likely going to end up much more complex than this but it's something that has to be eventually done in order to provide a base for more advanced features and especially the thing many have requested; longer term statistics at user selectable precision.

[Lekensteyn]

I just use vnstat for monitoring a desktop, laptop and Raspberry Pi. Nothing fancy. In the desktop, I sometimes have to rmmod/modprobe due to NIC driver changes. That resets the speed. The RPi never changes its speed (theoretical 100 Mbit).

Do you know a use case where the speed changes often enough to be worth to add periodic checks? Also consider the battery life of mobile devices please :-)

[bpvarsity]

I think making the default 1000 is fine but its not necessary to check for port changes. Also has anyone tested this on a 10Gbit connection?

[vergoh]

@Lekensteyn
That reported speed is indeed much dependent on the implementation of the NIC driver and I haven't so far found any wlan driver that would provide the speed. However, I did notice that when binding several NICs together, the resulting bond interface will report the sum of the interfaces used to create it.

Using a separate NIC for device testing/debugging with a direct ethernet cable connection is at least one case where the speed will vary depending to what sort of device the interface is attached to.

@bpvarsity
If you have a 10Gbit connection and set MaxBandwidth to 10000 then you'll be fine as long as the kernel isn't 32 bit. vnStat itself doesn't have issues as long as it doesn't have to guess at that rate when a 32 bit integer overflows.

Example by ping flooding localhost in a VM and "vnstat -l -i lo":

                           rx         |       tx
--------------------------------------+------------------
  bytes                    88.79 GiB  |       88.79 GiB
--------------------------------------+------------------
          max            8.73 Gbit/s  |     8.73 Gbit/s
      average            8.66 Gbit/s  |     8.66 Gbit/s
          min            8.56 Gbit/s  |     8.56 Gbit/s
--------------------------------------+------------------
  packets                    5814112  |         5814112
--------------------------------------+------------------
          max              68120 p/s  |       68120 p/s
      average              67605 p/s  |       67605 p/s
          min              66836 p/s  |       66836 p/s
--------------------------------------+------------------
  time                  1.43 minutes