When including most _OUT fields in V9 flow generation mode the bytes/packet values are incorrect about nprobe

[lucaderi]

I have tried to reproduce this issue using a pcap file.
IPV4_SRC_ADDR|IPV4_DST_ADDR|INPUT_SNMP|OUTPUT_SNMP|IN_PKTS|IN_BYTES|OUT_PKTS|OUT_BYTES|FIRST_SWITCHED|LAST_SWITCHED|L4_SRC_PORT|L4_DST_PORT|TCP_FLAGS|PROTOCOL|SRC_TOS|IPV6_SRC_ADDR|IPV6_DST_ADDR|IN_SRC_MAC|OUT_DST_MAC|FRAGMENTS|APPL_LATENCY_MS|RETRANSMITTED_IN_PKTS|RETRANSMITTED_OUT_PKTS|OOORDER_IN_PKTS|OOORDER_OUT_PKTS
192.168.1.233|192.168.99.1|0|0|336|13881|525|758335|1455142159|1455142159|2645|143|31|6|0|::|::|00:90:F5:09:B0:27|00:04:75:B5:B4:97|0|0.000|0|0|0|0

and

IPV4_SRC_ADDR|IPV4_DST_ADDR|INPUT_SNMP|OUTPUT_SNMP|IN_PKTS|IN_BYTES|FIRST_SWITCHED|LAST_SWITCHED|L4_SRC_PORT|L4_DST_PORT|TCP_FLAGS|PROTOCOL|SRC_TOS|IPV6_SRC_ADDR|IPV6_DST_ADDR|IN_SRC_MAC|OUT_DST_MAC|FRAGMENTS|APPL_LATENCY_MS|RETRANSMITTED_IN_PKTS|RETRANSMITTED_OUT_PKTS|OOORDER_IN_PKTS|OOORDER_OUT_PKTS
192.168.1.233|192.168.99.1|0|0|336|13881|1455142202|1455142202|2645|143|31|6|0|::|::|00:90:F5:09:B0:27|00:04:75:B5:B4:97|0|0.000|0|0|0|0

so it looks to me the value for IN_PKTS/IN_BYTES do not change. How can I reproduce this bug (full command line)?

[DanEllis197]

Commands below, and a graph adding the total traffic for each flows. In
previous runs the "not good" was sending flows with a small amount of
bytes/packets in the flows. Now I'm not seeing any "valid" flows with the
template having data in in_bytes or out_bytes. I didn't pcap the flow data
and examine it, can if you need.

Good:
./nprobes -n 208.76.14.242:20013 --lifetime-timeout 15 --idle-timeout 15
--queue-timeout 15 --sample-rate 1:16 --flow-version 9 --no-promisc -Q 11
-u 10 -T "%IPV4_SRC_ADDR %IPV4_DST_ADDR %INPUT_SNMP %OUTPUT_SNMP %IN_PKTS
%IN_BYTES %FIRST_SWITCHED %LAST_SWITCHED %L4_SRC_PORT %L4_DST_PORT
%TCP_FLAGS %PROTOCOL %SRC_TOS %IPV6_SRC_ADDR %IPV6_DST_ADDR %IN_SRC_MAC
%OUT_DST_MAC %FRAGMENTS %CLIENT_NW_LATENCY_MS %SERVER_NW_LATENCY_MS
%APPL_LATENCY_MS %RETRANSMITTED_IN_PKTS %OOORDER_IN_PKTS"

Not good:
./nprobes -n 208.76.14.242:20013 --lifetime-timeout 15 --idle-timeout 15
--queue-timeout 15 --sample-rate 1:16 --flow-version 9 --no-promisc -Q 11
-u 10 -T "%IPV4_SRC_ADDR %IPV4_DST_ADDR %INPUT_SNMP %OUTPUT_SNMP %IN_PKTS
%IN_BYTES %OUT_PKTS %OUT_BYTES %FIRST_SWITCHED %LAST_SWITCHED %L4_SRC_PORT
%L4_DST_PORT %TCP_FLAGS %PROTOCOL %SRC_TOS %IPV6_SRC_ADDR %IPV6_DST_ADDR
%IN_SRC_MAC %OUT_DST_MAC %FRAGMENTS %CLIENT_NW_LATENCY_MS
%SERVER_NW_LATENCY_MS %APPL_LATENCY_MS
%RETRANSMITTED_IN_PKTS %RETRANSMITTED_OUT_PKTS %OOORDER_IN_PKTS
%OOORDER_OUT_PKTS"

Output from starting good:
root@novia2:~# ./nprobes -n 208.76.14.242:20013 --lifetime-timeout 15
--idle-timeout 15 --queue-timeout 15 --sample-rate 1:16 --flow-version 9
--no-promisc -Q 11 -u 10 -T "%IPV4_SRC_ADDR %IPV4_DST_ADDR %INPUT_SNMP
%OUTPUT_SNMP %IN_PKTS %IN_BYTES %FIRST_SWITCHED %LAST_SWITCHED %L4_SRC_PORT
%L4_DST_PORT %TCP_FLAGS %PROTOCOL %SRC_TOS %IPV6_SRC_ADDR %IPV6_DST_ADDR
%IN_SRC_MAC %OUT_DST_MAC %FRAGMENTS %CLIENT_NW_LATENCY_MS
%SERVER_NW_LATENCY_MS %APPL_LATENCY_MS %RETRANSMITTED_IN_PKTS
%OOORDER_IN_PKTS" -i eth2
11/Feb/2016 01:35:02 [nprobe.c:3182] Valid nProbe Pro license found
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin SIP Plugin:
missing license [/etc/nprobe.license.voippro]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin RTP Plugin:
missing license [/etc/nprobe.license.voippro]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin HTTP Protocol:
missing license [/etc/nprobe.license.http]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin SMTP Protocol:
missing license [/etc/nprobe.license.email]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin Netflow-Lite
Plugin: missing license [/etc/nprobe.license.nflite]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin DNS/LLMNR
Protocol: missing license [/etc/nprobe.license.dns]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin Oracle
Protocol: missing license [/etc/nprobe.license.oracle]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin GTPv0 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv0]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin GTPv1 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv1]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin GTPv2 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv2]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin Radius
Protocol: missing license [/etc/nprobe.license.radius]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin DHCP Protocol:
missing license [/etc/nprobe.license.dhcp]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin IMAP Protocol:
missing license [/etc/nprobe.license.email]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin POP3 Protocol:
missing license [/etc/nprobe.license.email]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin System process
information: missing license [/etc/nprobe.license.process]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin Diameter
Protocol: missing license [/etc/nprobe.license.diameter]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin S1AP Protocol:
missing license [/etc/nprobe.license.S1AP]
11/Feb/2016 01:35:02 [plugin.c:744] Unable to enable plugin Export Plugin:
missing license [/etc/nprobe.license.export]
11/Feb/2016 01:35:02 [nprobe.c:4679] Welcome to nProbe Pro v.7.3.151219
($Revision: 4748 $) for x86_64-unknown-linux-gnu with native PF_RING
acceleration
11/Feb/2016 01:35:02 [nprobe.c:4689] Running on Debian GNU/Linux 8.2
(jessie)
11/Feb/2016 01:35:02 [nprobe.c:4700] [LICENSE] nProbe SystemId:
7EF340067104A1D2
11/Feb/2016 01:35:02 [nprobe.c:6707] Welcome to nProbe v.7.3.151219 for
x86_64-unknown-linux-gnu
11/Feb/2016 01:35:02 [nprobe.c:5965] Using NetFlow Packet Payload Len: 1472
11/Feb/2016 01:35:02 [plugin.c:1007] 0 plugin(s) enabled
11/Feb/2016 01:35:02 [nprobe.c:6362] Each flow is 93 bytes long
11/Feb/2016 01:35:02 [nprobe.c:6363] The # packets per flow has been set to
14
11/Feb/2016 01:35:02 [nprobe.c:5251] Using packet capture length 128
11/Feb/2016 01:35:02 [nprobe.c:6926] Flows ASs will not be computed
(missing GeoIP support)
11/Feb/2016 01:35:02 [nprobe.c:7011] Capturing packets from interface eth2
[snaplen: 128 bytes]
11/Feb/2016 01:35:02 [util.c:3091] nProbe changed user to 'nobody'
11/Feb/2016 01:35:02 [nprobe.c:7221] nProbe started successfully
11/Feb/2016 01:36:31 [nprobe.c:2512] Flow export stats: [102114217
bytes/38808 pkts][131 flows/12 pkts sent]
11/Feb/2016 01:36:31 [nprobe.c:2522] Flow drop stats: [159674 bytes/995
pkts][0 flows]
11/Feb/2016 01:36:31 [nprobe.c:2527] Total flow stats: [102273891
bytes/39803 pkts][131 flows/12 pkts sent]

Output from starting not good:
root@novia2:~# ./nprobes -n 208.76.14.242:20013 --lifetime-timeout 15
--idle-timeout 15 --queue-timeout 15 --sample-rate 1:16 --flow-version 9
--no-promisc -Q 11 -u 10 -T "%IPV4_SRC_ADDR %IPV4_DST_ADDR %INPUT_SNMP
%OUTPUT_SNMP %IN_PKTS %IN_BYTES %OUT_PKTS %OUT_BYTES %FIRST_SWITCHED
%LAST_SWITCHED %L4_SRC_PORT %L4_DST_PORT %TCP_FLAGS %PROTOCOL %SRC_TOS
%IPV6_SRC_ADDR %IPV6_DST_ADDR %IN_SRC_MAC %OUT_DST_MAC %FRAGMENTS
%CLIENT_NW_LATENCY_MS %SERVER_NW_LATENCY_MS %APPL_LATENCY_MS
%RETRANSMITTED_IN_PKTS %RETRANSMITTED_OUT_PKTS %OOORDER_IN_PKTS
%OOORDER_OUT_PKTS"
11/Feb/2016 01:41:15 [nprobe.c:3182] Valid nProbe Pro license found
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin SIP Plugin:
missing license [/etc/nprobe.license.voippro]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin RTP Plugin:
missing license [/etc/nprobe.license.voippro]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin HTTP Protocol:
missing license [/etc/nprobe.license.http]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin SMTP Protocol:
missing license [/etc/nprobe.license.email]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin Netflow-Lite
Plugin: missing license [/etc/nprobe.license.nflite]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin DNS/LLMNR
Protocol: missing license [/etc/nprobe.license.dns]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin Oracle
Protocol: missing license [/etc/nprobe.license.oracle]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin GTPv0 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv0]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin GTPv1 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv1]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin GTPv2 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv2]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin Radius
Protocol: missing license [/etc/nprobe.license.radius]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin DHCP Protocol:
missing license [/etc/nprobe.license.dhcp]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin IMAP Protocol:
missing license [/etc/nprobe.license.email]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin POP3 Protocol:
missing license [/etc/nprobe.license.email]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin System process
information: missing license [/etc/nprobe.license.process]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin Diameter
Protocol: missing license [/etc/nprobe.license.diameter]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin S1AP Protocol:
missing license [/etc/nprobe.license.S1AP]
11/Feb/2016 01:41:15 [plugin.c:744] Unable to enable plugin Export Plugin:
missing license [/etc/nprobe.license.export]
11/Feb/2016 01:41:15 [nprobe.c:4679] Welcome to nProbe Pro v.7.3.151219
($Revision: 4748 $) for x86_64-unknown-linux-gnu with native PF_RING
acceleration
11/Feb/2016 01:41:15 [nprobe.c:4689] Running on Debian GNU/Linux 8.2
(jessie)
11/Feb/2016 01:41:15 [nprobe.c:4700] [LICENSE] nProbe SystemId:
7EF340067104A1D2
11/Feb/2016 01:41:15 [nprobe.c:6707] Welcome to nProbe v.7.3.151219 for
x86_64-unknown-linux-gnu
11/Feb/2016 01:41:15 [nprobe.c:5965] Using NetFlow Packet Payload Len: 1472
11/Feb/2016 01:41:15 [plugin.c:1007] 0 plugin(s) enabled
11/Feb/2016 01:41:15 [nprobe.c:6362] Each flow is 109 bytes long
11/Feb/2016 01:41:15 [nprobe.c:6363] The # packets per flow has been set to
12
11/Feb/2016 01:41:15 [nprobe.c:5251] Using packet capture length 128
11/Feb/2016 01:41:16 [nprobe.c:6926] Flows ASs will not be computed
(missing GeoIP support)
11/Feb/2016 01:41:16 [nprobe.c:7011] Capturing packets from interface eth2
[snaplen: 128 bytes]
11/Feb/2016 01:41:16 [util.c:3091] nProbe changed user to 'nobody'
11/Feb/2016 01:41:16 [nprobe.c:7221] nProbe started successfully
11/Feb/2016 01:42:32 [nprobe.c:2486] Fragment queue length: 0
11/Feb/2016 01:42:32 [nprobe.c:2512] Flow export stats: [4450677
bytes/51925 pkts][128 flows/15 pkts sent]
11/Feb/2016 01:42:32 [nprobe.c:2522] Flow drop stats: [292617 bytes/1923
pkts][0 flows]
11/Feb/2016 01:42:32 [nprobe.c:2527] Total flow stats: [4743294
bytes/53848 pkts][128 flows/15 pkts sent]

Dan Ellis, (m) 610-349-9017

Kentik.com http://kentik.com

On Wed, Feb 10, 2016 at 2:12 PM, Luca Deri [email protected] wrote:

I have tried to reproduce this issue using a pcap file.

IPV4_SRC_ADDR|IPV4_DST_ADDR|INPUT_SNMP|OUTPUT_SNMP|IN_PKTS|IN_BYTES|OUT_PKTS|OUT_BYTES|FIRST_SWITCHED|LAST_SWITCHED|L4_SRC_PORT|L4_DST_PORT|TCP_FLAGS|PROTOCOL|SRC_TOS|IPV6_SRC_ADDR|IPV6_DST_ADDR|IN_SRC_MAC|OUT_DST_MAC|FRAGMENTS|APPL_LATENCY_MS|RETRANSMITTED_IN_PKTS|RETRANSMITTED_OUT_PKTS|OOORDER_IN_PKTS|OOORDER_OUT_PKTS

192.168.1.233|192.168.99.1|0|0|336|13881|525|758335|1455142159|1455142159|2645|143|31|6|0|::|::|00:90:F5:09:B0:27|00:04:75:B5:B4:97|0|0.000|0|0|0|0

and

IPV4_SRC_ADDR|IPV4_DST_ADDR|INPUT_SNMP|OUTPUT_SNMP|IN_PKTS|IN_BYTES|FIRST_SWITCHED|LAST_SWITCHED|L4_SRC_PORT|L4_DST_PORT|TCP_FLAGS|PROTOCOL|SRC_TOS|IPV6_SRC_ADDR|IPV6_DST_ADDR|IN_SRC_MAC|OUT_DST_MAC|FRAGMENTS|APPL_LATENCY_MS|RETRANSMITTED_IN_PKTS|RETRANSMITTED_OUT_PKTS|OOORDER_IN_PKTS|OOORDER_OUT_PKTS

192.168.1.233|192.168.99.1|0|0|336|13881|1455142202|1455142202|2645|143|31|6|0|::|::|00:90:F5:09:B0:27|00:04:75:B5:B4:97|0|0.000|0|0|0|0

so it looks to me the value for IN_PKTS/IN_BYTES do not change. How can I
reproduce this bug (full command line)?

—
Reply to this email directly or view it on GitHub
#29 (comment).

[ValentinaViscarelli]

I see you use a nprobe version of December.
I already fixed a similar bug in January.
Please update to new version and try again.

[DanEllis197]

Closing this as it's likely we should be using direction and in_bytes/packets vs in_bytes/packets and out_bytes/packets.