vnStat is sometimes adding 4GB of data to statistic after interfaces restarts. about vnstat

[0ki]

I think that happened when computer was going into suspend. As I rarely use the mode, I think that matches the number of extra 4GBs I have on the interface.

[0ki]

I kept having 4GB/4GB added (symmetrically tx/rx) for my wwan0 interface without any relevant information in the log. So I did install "vnStat 2.2 beta1 by Teemu Toivola" and now it is correctly displaying if wwan0 is "disabled". Let's hope this doesn't repeat. Is there anything I can do to debug it further if needed?

[vergoh]

Your MaxBandwidth setting in /etc/vnstat.conf is most likely too high or disabled. Another alternative would be to stop the vnStat daemon before wlan0 restart and then restart the daemon using the --sync parameter after wlan0 is up again.

[Misiek304]

How should I set my MaxBandwidth? I have set it to 300 for wlan0 and 1000 for the rest, because I have a gigabit switch.

Also now I see that this 4GB problem is not limited to wlan0. I just had some connections issues and vnstat added 4GB to my eth0.1 and eth0.2 interfaces.

[vergoh]

What is the UpdateInterval value in OpenWRT in the provided /etc/vnstat.conf?

If you see extra 4GB on a 32-bit system then the problem is usually caused by either MaxBandwidth being disabled or high or UpdateInterval being too high. Those two settings essentially work together when vnStat is trying to figure out if a traffic counter rollover should be considered as real traffic or as a interface reset.

[Misiek304]

It's set to 60. I just lowered it to 30.
Is 1000 value for MaxBandwidth correct for 1Gb interface? I think that MaxBandwidth set to 300 might have fixed the issue for wlan0 but 1000 for ethernet adds 4GB sometimes.

[vergoh]

The default value for UpdateInterval is actually 30. If you had it at 60 then the default has most likely been modified in OpenWRT for some reason. 60 is anyway too high when a 1 Gbit interface with 32-bit kernel is used as the interface can transfer 4 GB in less than that time. If you want to be on the safe side, you should lower the value to say for example 15.

The value of 1000 is correct for a 1 Gbit interface. If you want a different MaxBandwidth setting for wlan0 then you can use the MaxBWwlan0 keyword. Remember to restart the daemon after configuration file changes.

[Misiek304]

I can't set UpdateInterval to 15. I'm getting an error that it's to low and it's resetting to default 30.

[vergoh]

I'm guessing the PollInterval isn't then the default 5 in OpenWRT either. The lowest value UpdateInterval can have is the same value as PollInterval so also that will need to be adjusted if a lower UpdateInterval is needed.

[Misiek304]

Can you maybe send me a default config file so I could ask OpenWRT developers to adjust the values?

[vergoh]

The default config file is provided in the cfg directory of the source package: https://github.com/vergoh/vnstat/blob/master/cfg/vnstat.conf

Those settings are explained in more details in the vnstat.conf man page ( http://humdi.net/vnstat/man/vnstat.conf.html ) but also the daemon man page ( http://humdi.net/vnstat/man/vnstatd.html ) RESTRICTIONS section should be noted as it documents the limitations of 32-bit kernels.

[Misiek304]

Saving UpdateInterval to 15 and PollInterval to 5 fixed the issue for me.
Thank you for your help. I will also contact OpenWRT developers and link them to this issue so they could adjust the default vnStat config files to implement the fix.

[0ki]

Why won't vnstat ship with UpdateInterval 15 by default?
I had this problem since I installed vnstat and went to investigate. Now trying out this fix to see if it works.

[vergoh]

I don't think there are any config values which would be perfect for all usecases. That's why those values are in the config file instead of being hardcoded. Even if I'd release vnStat with some specific tuned values then the binary package creators of various distributions still have the possibility of changing those values, like I recall is the case with OpenWRT. I'm also not sure if this issue report still applies to, for example, version 2.0.

[0ki]

1Gbit is a typical max speed for an interface on a home user machine.

Is there any drawback to having UpdateInterval set to 15 instead of 30?

Btw, I'm still having this 4GB problem even though my machine is 64-bit and vnstat is also a 64-bit executable.

[vergoh]

I'd appreciate if you'd create a new issue for the scenario you are having as it's starting to sound that it doesn't fully match the situation originally described in this issue and some other solution would be a better fix for the problem than modifying the UpdateInterval as a workaround.

Regarding the typical interface speeds, it's true that 1 Gbit NICs are far more common today than several years ago. However, triggering the original reason why a UpdateInterval of 60 seconds isn't enough (as described on the man page) and 30 is the minimum still requires that the home user machine has something attached to it that can handle the full 1 Gbit rate and that there's at least 4 GB data to transfer in one go... and that this home user machine is still 32-bit.

[0ki]

I'd love to help out by creating a new issue, but in truth the UpdateInterval fixed it for me and I'm not really looking forward in screwing with stats on my production machine.

Thanks for the great tool! :)

[0ki]

I was still having intermittent troubles. Finally checking the logfile (doh!) I discovered that what actually needed to be set is BandwidthDetection 0 otherwise it would override any decent bandwidth settings I had. (This is still on a 64bit machine).

[vergoh]

Could you give the latest 2.2 beta a try?

I've fixed the scenario that most likely causes the issue you are having with commit a76c906. The problem was essentially such that vnStat didn't do any real detection of the size of the interface counters and only assumed that any value greater than 4GB are for 64-bit and less then 32-bit. That then results in the interface counter looking like a 32-bit value even on 64-bit machines until that 4GB limit is exceeded which in turn could, at least in theory, cause problems before the limit is reached.

Note that if you upgrade to that version then it could help to adjust back the changed settings closer to default so that it would be more obvious if that commit fixes the issue.

With the beta version, you should see either "32-bit" or "64-bit" string visible in the daemon startup prints on the same line the version is shown. That's there to help verifying if the detection succeeded.

[0ki]

I do appreciate your work. However what I said still applies: «I'm not really looking forward in screwing with stats on my production machine.»

FYI the bug I have/had is for a 3G (HSDPA) interface with max hardware speed of around 20Mbps.

Jan 31 19:01:02 yyy vnstatd[1208]: Info: Monitoring: wwan0 (1000 Mbit) xxx (1000 Mbit) xxx (1000 Mbit)
Jan 31 19:01:02 yyy vnstatd[1208]: Error: Unable to get interface "wwan0" statistics.
Jan 31 19:01:02 yyy vnstatd[1208]: Info: Interface "wwan0" not available, disabling.
Jan 31 19:01:02 yyy vnstatd[1208]: Info: Interface "wwan0" enabled.
Jan 31 19:01:02 yyy vnstatd[1208]: Info: Traffic rate for "wwan0" higher than set maximum 1000 Mbit (15->2063, r4088 t4095), syncing.

[vergoh]

You haven't mentioned which version you are currently using. If it's a 1.x version then you'd even have the option of running the 2.2 beta version in parallel as long as the binaries are installed in a different location. The current beta doesn't either have any other relevant changes compared to version 2.1 then this specific fix.

BandwidthDetection works mainly only for wired ethernet + some wifi interfaces. Many other don't provide the necessary details for vnStat and the MaxBandwidth value ends up being use a fallback. If you know the real interface speed then the correct way to configure it, in your case, would be to add MaxBWwwan0 20 to the configuration file. That causes the setting to affect only that one interface.

If those log messages are from the same second then what can happen (and explain the extra traffic) is that the wwan0 interface goes down and back up faster that vnStat can see it happen. PollInterval defines the check interval. If vnStat doesn't see the down-up cycle happening then it doesn't know that it should reset the interface counters but instead assumes the 32-bit counter (due to a76c906 fix missing) has overflown and adds 4GB traffic if such number would be realistic based on the used bandwidth limit within the UpdateInterval period. With a76c906 this would get detected as 64-bit all the time and the possible calculated traffic value would always be so high that vnStat would discard it as false reading instead of adding that information to the database (what's being done on that last line of the log you've copypasted).

[0ki]

Somehow BandwidthDetection works [terribly] on my "0bdb:1926 Ericsson Business Mobile Networks BV"...
Does MaxBWwwan0 20 set BandwidthDetection to 0 on that specific interface?
My version is "vnStat 1.14 by Teemu Toivola". I'll think about running it in parallel.

[vergoh]

What's the detected value for that interface in your case? vnstat --iflist should show it and it will not care about the possibly configured values in the configuration file.

MaxBWwwan0 20 doesn't set BandwidthDetection to zero for the specified interface. If the interface provides some other value by itself then that value will get used, otherwise the user provided value gets used. That's possibly something I'll have to consider changing to act more as an override.

The support for running 1.x and 2.x versions in parallel in the same database directory was added in version 1.12 so version 1.14 is recent enough to ignore the possible extra database file in the directory.

[0ki]

vnstat --iflist
Available interfaces: xxx wwan0 xxx xxx

[vergoh]

No value after "wwan0" indicates that the interface driver doesn't provide that information and as a result, the MaxBWwwan0 value would get used. If it's missing then the value of MaxBandwidth gets used as a fallback.