nProbe not capturing flows in proxy mode about nprobe

[lmangani]

I believe the issue is with the use of -n none. From the command description:

If you specify none as value, no flow will be exported; in this case the -P parameter is mandatory.

Adding a dummy -n collector seems to confirm this is the case:

07/Dec/2015 21:38:15 [nprobe.c:2503] Processed packets: 3356 (max bucket search: 0)
07/Dec/2015 21:38:15 [nprobe.c:2486] Fragment queue length: 0
07/Dec/2015 21:38:15 [nprobe.c:2512] Flow export stats: [1407752 bytes/3356 pkts][2 flows/1 pkts sent]
07/Dec/2015 21:38:15 [nprobe.c:2519] Flow collection: [collected pkts: 0][processed flows: 0]
07/Dec/2015 21:38:15 [nprobe.c:2522] Flow drop stats:   [0 bytes/0 pkts][0 flows]
07/Dec/2015 21:38:15 [nprobe.c:2527] Total flow stats:  [1407752 bytes/3356 pkts][2 flows/1 pkts sent]

[smerkal]

Flows should still be emitted via the zmq endpoint, just not exported to an external collector. The use of the '-n none' parameter keeps a waterfall condition from occurring due to the default behavior of exporting on port 2055 if -n is not specified, which is the same port we are collecting on. This exact scenario worked in the past.

Trying to process the attached pcap file using the '-i ' option just results in collecting and emitting/exporting flow data from ABOUT the packets between the router and the nProbe server, not proxying the flow data contained WITHIN the packets received from the router.

Adding the -i option to the command line in my original post also results in nProbe collecting and emitting flow data via zmq from the sessions between the router and the server as well, no -P option is specified. Flows are emitted via zmq, but not exported as seen below:

nprobe -3 2055 -zmq=tcp://*:5556 -n none -b 2 -i eth4

07/Dec/2015 18:36:07 [engine.c:3210] Starting bucket dequeue thread
07/Dec/2015 18:36:08 [engine.c:2361] New Flow: [udp] 172.31.0.0:50101 -> 172.31.200.50:2055 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 0][tos 0][ifIdx: 65535 -> 65535][subflowId: 0/0x0000][idx=76663]
07/Dec/2015 18:36:08 [pro/pf_ring.c:94] PF_RING stats (Average): 4/0 [0.0 %] pkts rcvd/dropped
07/Dec/2015 18:36:08 [engine.c:2361] New Flow: [udp] 172.31.0.8:50101 -> 172.31.200.50:2056 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 0][tos 0][ifIdx: 65535 -> 65535][subflowId: 0/0x0000][idx=76674]
07/Dec/2015 18:36:08 [engine.c:2361] New Flow: [udp] 172.31.0.8:50103 -> 172.31.200.50:2056 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 0][tos 0][ifIdx: 65535 -> 65535][subflowId: 0/0x0000][idx=76680]
07/Dec/2015 18:36:08 [engine.c:2361] New Flow: [udp] 172.31.0.0:50103 -> 172.31.200.50:2055 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 0][tos 0][ifIdx: 65535 -> 65535][subflowId: 0/0x0000][idx=76669]
^C07/Dec/2015 18:36:09 [pro/pf_ring.c:94] PF_RING stats (Average): 12/0 [0.0 %] pkts rcvd/dropped
07/Dec/2015 18:36:09 [pro/pf_ring.c:105] PF_RING stats (Current): 8/0 [0.0 %] pkts rcvd/dropped
07/Dec/2015 18:36:09 [cache.c:1200] Redis Cache [0 total/0.0 get/sec][0 total/0.0 set/sec]
07/Dec/2015 18:36:09 [nprobe.c:386] Received shutdown request... [signal: 2]
07/Dec/2015 18:36:09 [pro/pf_ring.c:300] Terminated PF_RING packet processing
07/Dec/2015 18:36:10 [nprobe.c:4716] nProbe is shutting down...
07/Dec/2015 18:36:10 [nprobe.c:4722] Waiting for PF_RING termination
07/Dec/2015 18:36:10 [nprobe.c:4731] PF_RING terminated
07/Dec/2015 18:36:10 [nprobe.c:4752] Exporting pending buckets...
07/Dec/2015 18:36:10 [engine.c:2673] About to flush hash (threadId 0)
07/Dec/2015 18:36:10 [engine.c:2675] Completed hash walk (thread 0)
07/Dec/2015 18:36:10 [nprobe.c:4758] Waiting to export queued buckets... [queue len=4]
07/Dec/2015 18:36:10 [engine.c:2541] Emitting Flow: [->][udp] 172.31.0.0:50101 -> 172.31.200.50:2055 [2 pkt/304 bytes][ifIdx 65535->65535][0.0 sec][init Unknown][AS: 0 -> 0]
07/Dec/2015 18:36:10 [engine.c:2541] Emitting Flow: [->][udp] 172.31.0.8:50101 -> 172.31.200.50:2056 [1 pkt/152 bytes][ifIdx 65535->65535][0.0 sec][init Unknown][AS: 0 -> 0]
07/Dec/2015 18:36:10 [engine.c:2541] Emitting Flow: [->][udp] 172.31.0.8:50103 -> 172.31.200.50:2056 [2 pkt/164 bytes][ifIdx 65535->65535][0.0 sec][init Unknown][AS: 0 -> 0]
07/Dec/2015 18:36:10 [engine.c:2541] Emitting Flow: [->][udp] 172.31.0.0:50103 -> 172.31.200.50:2055 [4 pkt/328 bytes][ifIdx 65535->65535][0.0 sec][init Unknown][AS: 0 -> 0]
07/Dec/2015 18:36:11 [nprobe.c:4773] Pending buckets have been exported...
07/Dec/2015 18:36:13 [engine.c:3293] Export thread terminated [exportQueue=0]
07/Dec/2015 18:36:13 [nprobe.c:4839] Flushing queued flows...
07/Dec/2015 18:36:13 [nprobe.c:4842] Freeing memory...
07/Dec/2015 18:36:13 [plugin.c:277] Terminating plugins.
07/Dec/2015 18:36:13 [cache.c:1200] Redis Cache [0 total/0.0 get/sec][0 total/0.0 set/sec]
07/Dec/2015 18:36:13 [nprobe.c:4934] Still allocated 0 hash buckets
07/Dec/2015 18:36:13 [nprobe.c:2457] Processed packets: 12 (max bucket search: 0)
07/Dec/2015 18:36:13 [nprobe.c:2440] Fragment queue length: 0
07/Dec/2015 18:36:13 [nprobe.c:2466] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
07/Dec/2015 18:36:13 [nprobe.c:2473] Flow collection: [collected pkts: 0][processed flows: 0]
07/Dec/2015 18:36:13 [nprobe.c:2476] Flow drop stats: [0 bytes/0 pkts][0 flows]
07/Dec/2015 18:36:13 [nprobe.c:2481] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
07/Dec/2015 18:36:13 [nprobe.c:4947] Cleaning globals
07/Dec/2015 18:36:13 [nprobe.c:4967] nProbe terminated.

The desire is to emit the flow data contained WITHIN these frames, not the flow data ABOUT these frames. Using -n instead of --zmq does not work either. The net observed behavior is that nProbe is not seeing the flow data contained in the frames from the router.

[lucaderi]

The apps work exactly as you described. Please do

1. start nprobe
   nprobe -i none -n none -3 2055 --zmq tcp://127.0.0.1:1234 <<== note the --zmq (double dash)
2. start ntopng
   ntopng -i tcp://127.0.0.1:1234
3. send nProbe some flows
   Example: nprobe -i ~/pcap/http/http.pcap

You will see them appear in ntopng.

[smerkal]

That's just it, I don't. Nor do I see them if I export to an external collector. I know it should work and it did before.

I have tried it exactly as you describe above using the .pcap files I have attached, as well as using live flow data being sent from the router on port 2055.

For the latter, I see the flows from the router hitting the server by the thousand using tcpdump. Decoding them with Wireshark shows they are valid IPFIX flows on the correct port. But they never get picked up by nProbe that is listening on that port.

This all worked when we first tested it a couple of months ago, but after a fresh install in preparation for production use, it no longer does. I am willing to give you access to the server if it helps, we are at a loss as to what changed.

[lucaderi]

Please contact me via email next week and we'll see what we can do

Thanks

[smerkal]

OK. Thanks Luca. In the mean time I'll keep poking at it.

Erik

On Tue, Dec 8, 2015 at 8:36 AM, Luca Deri [email protected] wrote:

Please contact me via email next week and we'll see what we can do

Thanks

—
Reply to this email directly or view it on GitHub
#10 (comment).

[smerkal]

Luca,

If you are still willing/able, we would appreciate any assistance you can
provide. Let me know what you would need from us.

Erik

On Tue, Dec 8, 2015 at 8:36 AM, Luca Deri [email protected] wrote:

Please contact me via email next week and we'll see what we can do

Thanks

—
Reply to this email directly or view it on GitHub
#10 (comment).

[lucaderi]

@ValentinaViscarelli Hi Valentina can help please?

[ValentinaViscarelli]

Smerkal,
please do this:

1. check you don't have nprobe and ntopng instances run.
2. Open a shell and run this command:
   nprobe -i none -n none -3 2055 --zmq tcp://127.0.0.1:1234 -b2
3. Open another shell and run this command:
   ntopng -i tcp://127.0.0.1:1234
4. Open another shell and run this command (use pcap that you attached):
   nprobe -i cflow.pcap -b2
5. wait a couple of minutes
6. send me the output of three commands.

Thanks

[smerkal]

Valentina,

This all works as expected. However, what I see in ntopng is flow data
ABOUT the packets in the capture (two hosts exchanging flow data) not the
flow data sent from the router that is contained WITHIN these packets
(hundreds of hosts communicating with the internet). It's acting the same
as if I were running -i while this .pcap file was being
captured.

I am trying to get nprobe to receive the flow data that the router is
sending on port 2055 and proxy it to ntopng or other collectors, but all I
can get it to do is send flow data about the headers of the frames it is
receiving from the router, not the data about the routers transient traffic
that is contained within the frame.

I am not sure how else to describe it that makes more sense. I just know
that it used to work and now does not.

Erik

root@uncsnbox:~# nprobe -i none -n none -32055 --zmq tcp://127.0.0.1:1234
-b2
15/Dec/2015 09:02:04 [nprobe.c:3130] Valid nProbe Pro license found
15/Dec/2015 09:02:04 [plugin.c:166] No plugins found in ./plugins
15/Dec/2015 09:02:04 [plugin.c:174] Loading 22 plugins [.so] from
/usr/local/lib/nprobe/plugins
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin DHCP Protocol:
missing license [/etc/nprobe.license.dhcp]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin Diameter
Protocol: missing license [/etc/nprobe.license.diameter]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin DNS Protocol:
missing license [/etc/nprobe.license.dns]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin Export Plugin:
missing license [/etc/nprobe.license.export]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin FTP Protocol:
missing license [/etc/nprobe.license.ftp]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin GTPv0 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv0]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin GTPv1 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv1]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin GTPv2 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv2]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin HTTP Protocol:
missing license [/etc/nprobe.license.http]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin IMAP Protocol:
missing license [/etc/nprobe.license.email]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin Netflow-Lite
Plugin: missing license [/etc/nprobe.license.nflite]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin Oracle
Protocol: missing license [/etc/nprobe.license.oracle]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin POP3 Protocol:
missing license [/etc/nprobe.license.email]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin System process
information: missing license [/etc/nprobe.license.process]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin Radius
Protocol: missing license [/etc/nprobe.license.radius]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin RTP Plugin:
missing license [/etc/nprobe.license.voippro]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin S1AP Protocol:
missing license [/etc/nprobe.license.S1AP]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin SIP Plugin:
missing license [/etc/nprobe.license.voippro]
15/Dec/2015 09:02:04 [plugin.c:742] Unable to enable plugin SMTP Protocol:
missing license [/etc/nprobe.license.email]
15/Dec/2015 09:02:04 [nprobe.c:4488] WARNING: The output interfaceId is set
to 0: did you forget to use -Q perhaps ?
15/Dec/2015 09:02:04 [nprobe.c:4491] WARNING: The input interfaceId is set
to 0: did you forget to use -u perhaps ?
15/Dec/2015 09:02:04 [nprobe.c:4552] Welcome to nProbe Pro v.7.2.151211
($Revision: 4471 $) for x86_64-unknown-linux-gnu with native PF_RING
acceleration
15/Dec/2015 09:02:04 [nprobe.c:4562] Running on Ubuntu 14.04.2 LTS
15/Dec/2015 09:02:04 [nprobe.c:4573] [LICENSE] nProbe SystemId:
FA623D157104A1D2
15/Dec/2015 09:02:04 [nprobe.c:4620] Tracing enabled
15/Dec/2015 09:02:04 [bgpPlugin.c:375] BGP plugin is disabled (--bgp-port
has not been specified)
15/Dec/2015 09:02:04 [dbPlugin.c:49] Initializing DB plugin
15/Dec/2015 09:02:04 [mysqlPlugin.c:111] Initialized MySQL plugin
15/Dec/2015 09:02:04 [plugin.c:248] 3 plugin(s) loaded [3 delete][2 packet].
15/Dec/2015 09:02:04 [nprobe.c:6526] Welcome to nprobe v.7.2.151211 for
x86_64-unknown-linux-gnu
15/Dec/2015 09:02:04 [nprobe.c:5752] Compiling flow templates...
15/Dec/2015 09:02:04 [plugin.c:851] Scanning plugin BGP Update Listener
[bgp]
15/Dec/2015 09:02:04 [plugin.c:851] Scanning plugin MySQL DB [db]
15/Dec/2015 09:02:04 [plugin.c:851] Scanning plugin MySQL Plugin [mysql]
15/Dec/2015 09:02:04 [plugin.c:1000] 0 plugin(s) enabled
15/Dec/2015 09:02:04 [nprobe.c:6203] Non IPv4/v6 traffic is discarded
according to the template
15/Dec/2015 09:02:04 [util.c:287] GeoIP: loaded AS config file
/usr/share/ntopng/httpdocs/geoip/GeoIPASNum.dat
15/Dec/2015 09:02:04 [util.c:296] GeoIP: loaded AS IPv6 config file
/usr/share/ntopng/httpdocs/geoip/GeoIPASNumv6.dat
15/Dec/2015 09:02:04 [nprobe.c:5121] Using packet capture length 128
15/Dec/2015 09:02:04 [nprobe.c:6698] IPv6 traffic will NOT be
exported/accounted by this probe
15/Dec/2015 09:02:04 [nprobe.c:6699] due to configuration options (e.g. use
NetFlow v9)
15/Dec/2015 09:02:04 [nprobe.c:6702] The flows hash has 131072 buckets
15/Dec/2015 09:02:04 [nprobe.c:6704] Flows older than 120 seconds will be
exported
15/Dec/2015 09:02:04 [nprobe.c:6707] Flows inactive for at least 30 seconds
will be exported
15/Dec/2015 09:02:04 [nprobe.c:6710] Expired flows will not be queued for
more than 30 seconds
15/Dec/2015 09:02:04 [nprobe.c:6717] Exported flows with engineType 0 and
engineId 108
15/Dec/2015 09:02:04 [nprobe.c:6739] TCP TOS will be ignored and set to 0.
15/Dec/2015 09:02:04 [nprobe.c:6757] After 1 flow packets are sent, we'll
delay at least 1 ms
15/Dec/2015 09:02:04 [nprobe.c:6777] Flows will be emitted in NetFlow 5
format
15/Dec/2015 09:02:04 [nprobe.c:6807] Flow input interface index is set to 0
15/Dec/2015 09:02:04 [nprobe.c:6813] Flow output interface index is set to 0
15/Dec/2015 09:02:04 [nprobe.c:6827] Not capturing packet from interface
(collector mode)
15/Dec/2015 09:02:04 [util.c:3840] Succesfully created ZMQ endpoint tcp://
127.0.0.1:1234
15/Dec/2015 09:02:04 [plugin.c:813] Disabling plugin BGP Update Listener
(no template is using it)
15/Dec/2015 09:02:04 [plugin.c:813] Disabling plugin MySQL DB (no template
is using it)
15/Dec/2015 09:02:04 [plugin.c:813] Disabling plugin MySQL Plugin (no
template is using it)
15/Dec/2015 09:02:04 [collect.c:86] Created UDP sockets
15/Dec/2015 09:02:04 [collect.c:90] Created a SCTP socket (102)
15/Dec/2015 09:02:04 [collect.c:145] Flow collector listening on port 2055
(IPv4/v6)
15/Dec/2015 09:02:04 [nprobe.c:6947] Starting 1 packet fetch thread(s)
15/Dec/2015 09:02:04 [nprobe.c:7035] nProbe started successfully
15/Dec/2015 09:02:04 [engine.c:3210] Starting bucket dequeue thread
15/Dec/2015 09:02:22 [engine.c:2361] New Flow: [udp] 172.31.0.0:50101 ->
172.31.200.50:2055 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 65535][tos
0][ifIdx: 0 -> 0][subflowId: 0/0x0000][idx=11126]
15/Dec/2015 09:02:22 [engine.c:2361] New Flow: [udp] 172.31.0.0:50103 ->
172.31.200.50:2055 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 65535][tos
0][ifIdx: 0 -> 0][subflowId: 0/0x0000][idx=11132]
15/Dec/2015 09:02:51 [util.c:3865] [ZMQ]
{"8":"172.31.0.0","12":"172.31.200.50","15":"0.0.0.0","10":0,"14":0,"2":3257,"1":1399640,"22":1450191741,"21":1450191741,"7":50101,"11":2055,"6":0,"4":17,"5":0,"16":0,"17":0,"9":0,"13":0,"42":1}
15/Dec/2015 09:02:51 [engine.c:2541] Emitting Flow: [->][udp]
172.31.0.0:50101 -> 172.31.200.50:2055 [3257 pkt/1399640 bytes][ifIdx
0->0][0.0 sec][init Unknown][AS: 0 -> 0]
15/Dec/2015 09:02:51 [util.c:3865] [ZMQ]
{"8":"172.31.0.0","12":"172.31.200.50","15":"0.0.0.0","10":0,"14":0,"2":99,"1":8112,"22":1450191741,"21":1450191741,"7":50103,"11":2055,"6":0,"4":17,"5":0,"16":0,"17":0,"9":0,"13":0,"42":2}
15/Dec/2015 09:02:51 [engine.c:2541] Emitting Flow: [->][udp]
172.31.0.0:50103 -> 172.31.200.50:2055 [99 pkt/8112 bytes][ifIdx 0->0][0.0
sec][init Unknown][AS: 0 -> 0]

root@uncsnbox:~# ntopng -i tcp://127.0.0.1:1234
15/Dec/2015 09:02:16 [Ntop.cpp:933] Setting local networks to 127.0.0.0/8
15/Dec/2015 09:02:16 [Redis.cpp:106] Successfully connected to redis
127.0.0.1:6379@0
15/Dec/2015 09:02:16 [NtopPro.cpp:119] [LICENSE] Reading license from
/etc/ntopng.license
15/Dec/2015 09:02:16 [Ntop.cpp:1152] Registered interface tcp://
127.0.0.1:1234 [id: 0]
15/Dec/2015 09:02:16 [Ntop.cpp:1165] Registered interface view tcp://
127.0.0.1:1234 [id: 0]
15/Dec/2015 09:02:16 [Utils.cpp:304] User changed to nobody
15/Dec/2015 09:02:16 [main.cpp:240] PID stored in file /var/tmp/ntopng.pid
15/Dec/2015 09:02:16 [HTTPserver.cpp:465] Please read
https://github.com/ntop/ntopng/blob/dev/doc/README.SSL if you want to
enable SSL.
15/Dec/2015 09:02:16 [HTTPserver.cpp:482] -->3000<--
15/Dec/2015 09:02:16 [HTTPserver.cpp:510] Web server dirs
[/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts]
15/Dec/2015 09:02:16 [HTTPserver.cpp:513] HTTP server listening on port 3000
15/Dec/2015 09:02:16 [main.cpp:290] Working directory: /var/tmp/ntopng
15/Dec/2015 09:02:16 [main.cpp:292] Scripts/HTML pages directory:
/usr/share/ntopng
15/Dec/2015 09:02:16 [Ntop.cpp:260] Welcome to ntopng x86_64 v.2.2.151211 -
(C) 1998-15 ntop.org
15/Dec/2015 09:02:16 [Ntop.cpp:265] Built on Ubuntu 14.04.2 LTS
15/Dec/2015 09:02:16 [PeriodicActivities.cpp:53] Started periodic
activities loop...
15/Dec/2015 09:02:16 [RuntimePrefs.cpp:32] Dumping alerts into syslog
15/Dec/2015 09:02:16 [NtopPro.cpp:233] [LICENSE] ntopng systemId:
FA623D157104A1D2
15/Dec/2015 09:02:16 [NtopPro.cpp:244] [LICENSE] ntopng license:
2163EA9A6D3FEBD13E0940ACB875D3DC1480454122251EC47F
15/Dec/2015 09:02:16 [NtopPro.cpp:265] [LICENSE] Maintenance is available
until Tue Nov 29 15:15:22 2016 [350 days left]
15/Dec/2015 09:02:16 [NetworkInterface.cpp:1426] Started packet polling on
interface tcp://127.0.0.1:1234 [id: 0]...
15/Dec/2015 09:02:17 [CollectorInterface.cpp:94] Collecting flows on tcp://
127.0.0.1:1234

root@uncsnbox:~# nprobe -i cflow.pcap -b2
15/Dec/2015 09:02:21 [nprobe.c:3130] Valid nProbe Pro license found
15/Dec/2015 09:02:21 [plugin.c:166] No plugins found in ./plugins
15/Dec/2015 09:02:21 [plugin.c:174] Loading 22 plugins [.so] from
/usr/local/lib/nprobe/plugins
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin DHCP Protocol:
missing license [/etc/nprobe.license.dhcp]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin Diameter
Protocol: missing license [/etc/nprobe.license.diameter]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin DNS Protocol:
missing license [/etc/nprobe.license.dns]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin Export Plugin:
missing license [/etc/nprobe.license.export]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin FTP Protocol:
missing license [/etc/nprobe.license.ftp]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin GTPv0 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv0]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin GTPv1 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv1]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin GTPv2 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv2]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin HTTP Protocol:
missing license [/etc/nprobe.license.http]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin IMAP Protocol:
missing license [/etc/nprobe.license.email]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin Netflow-Lite
Plugin: missing license [/etc/nprobe.license.nflite]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin Oracle
Protocol: missing license [/etc/nprobe.license.oracle]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin POP3 Protocol:
missing license [/etc/nprobe.license.email]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin System process
information: missing license [/etc/nprobe.license.process]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin Radius
Protocol: missing license [/etc/nprobe.license.radius]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin RTP Plugin:
missing license [/etc/nprobe.license.voippro]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin S1AP Protocol:
missing license [/etc/nprobe.license.S1AP]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin SIP Plugin:
missing license [/etc/nprobe.license.voippro]
15/Dec/2015 09:02:21 [plugin.c:742] Unable to enable plugin SMTP Protocol:
missing license [/etc/nprobe.license.email]
15/Dec/2015 09:02:21 [nprobe.c:4488] WARNING: The output interfaceId is set
to 0: did you forget to use -Q perhaps ?
15/Dec/2015 09:02:21 [nprobe.c:4491] WARNING: The input interfaceId is set
to 0: did you forget to use -u perhaps ?
15/Dec/2015 09:02:21 [nprobe.c:4552] Welcome to nProbe Pro v.7.2.151211
($Revision: 4471 $) for x86_64-unknown-linux-gnu with native PF_RING
acceleration
15/Dec/2015 09:02:21 [nprobe.c:4562] Running on Ubuntu 14.04.2 LTS
15/Dec/2015 09:02:21 [nprobe.c:4573] [LICENSE] nProbe SystemId:
FA623D157104A1D2
15/Dec/2015 09:02:21 [nprobe.c:4620] Tracing enabled
15/Dec/2015 09:02:21 [nprobe.c:4658] WARNING: -n parameter is missing.
127.0.0.1:2055 will be used.
15/Dec/2015 09:02:21 [nprobe.c:2948] Exporting flows towards 127.0.0.1:2055
using UDP
15/Dec/2015 09:02:21 [bgpPlugin.c:375] BGP plugin is disabled (--bgp-port
has not been specified)
15/Dec/2015 09:02:21 [dbPlugin.c:49] Initializing DB plugin
15/Dec/2015 09:02:21 [mysqlPlugin.c:111] Initialized MySQL plugin
15/Dec/2015 09:02:21 [plugin.c:248] 3 plugin(s) loaded [3 delete][2 packet].
15/Dec/2015 09:02:21 [nprobe.c:6526] Welcome to nprobe v.7.2.151211 for
x86_64-unknown-linux-gnu
15/Dec/2015 09:02:21 [nprobe.c:5752] Compiling flow templates...
15/Dec/2015 09:02:21 [plugin.c:851] Scanning plugin BGP Update Listener
[bgp]
15/Dec/2015 09:02:21 [plugin.c:851] Scanning plugin MySQL DB [db]
15/Dec/2015 09:02:21 [plugin.c:851] Scanning plugin MySQL Plugin [mysql]
15/Dec/2015 09:02:21 [plugin.c:1000] 0 plugin(s) enabled
15/Dec/2015 09:02:21 [nprobe.c:6203] Non IPv4/v6 traffic is discarded
according to the template
15/Dec/2015 09:02:21 [util.c:287] GeoIP: loaded AS config file
/usr/share/ntopng/httpdocs/geoip/GeoIPASNum.dat
15/Dec/2015 09:02:21 [util.c:296] GeoIP: loaded AS IPv6 config file
/usr/share/ntopng/httpdocs/geoip/GeoIPASNumv6.dat
15/Dec/2015 09:02:21 [nprobe.c:5121] Using packet capture length 128
15/Dec/2015 09:02:21 [nprobe.c:6698] IPv6 traffic will NOT be
exported/accounted by this probe
15/Dec/2015 09:02:21 [nprobe.c:6699] due to configuration options (e.g. use
NetFlow v9)
15/Dec/2015 09:02:21 [nprobe.c:6702] The flows hash has 131072 buckets
15/Dec/2015 09:02:21 [nprobe.c:6704] Flows older than 120 seconds will be
exported
15/Dec/2015 09:02:21 [nprobe.c:6707] Flows inactive for at least 30 seconds
will be exported
15/Dec/2015 09:02:21 [nprobe.c:6710] Expired flows will not be queued for
more than 30 seconds
15/Dec/2015 09:02:21 [nprobe.c:6717] Exported flows with engineType 0 and
engineId 125
15/Dec/2015 09:02:21 [nprobe.c:6739] TCP TOS will be ignored and set to 0.
15/Dec/2015 09:02:21 [nprobe.c:6757] After 1 flow packets are sent, we'll
delay at least 1 ms
15/Dec/2015 09:02:21 [nprobe.c:6777] Flows will be emitted in NetFlow 5
format
15/Dec/2015 09:02:21 [nprobe.c:6807] Flow input interface index is set to 0
15/Dec/2015 09:02:21 [nprobe.c:6813] Flow output interface index is set to 0
15/Dec/2015 09:02:21 [plugin.c:813] Disabling plugin BGP Update Listener
(no template is using it)
15/Dec/2015 09:02:21 [plugin.c:813] Disabling plugin MySQL DB (no template
is using it)
15/Dec/2015 09:02:21 [plugin.c:813] Disabling plugin MySQL Plugin (no
template is using it)
15/Dec/2015 09:02:21 [nprobe.c:6947] Starting 1 packet fetch thread(s)
15/Dec/2015 09:02:21 [nprobe.c:5496] Fetch packets thread started [thread 0]
15/Dec/2015 09:02:21 [engine.c:3210] Starting bucket dequeue thread
15/Dec/2015 09:02:21 [engine.c:2361] New Flow: [udp] 172.31.0.0:50101 ->
172.31.200.50:2055 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 0][tos
0][ifIdx: 65535 -> 65535][subflowId: 0/0x0000][idx=76663]
15/Dec/2015 09:02:21 [engine.c:2361] New Flow: [udp] 172.31.0.0:50103 ->
172.31.200.50:2055 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 0][tos
0][ifIdx: 65535 -> 65535][subflowId: 0/0x0000][idx=76669]
15/Dec/2015 09:02:21 [nprobe.c:5592] fetchPcapPackets(): no more packets to
read (capture file over?)
15/Dec/2015 09:02:21 [nprobe.c:5636] fetchPcapPackets(threadId=0) terminated
15/Dec/2015 09:02:21 [nprobe.c:7035] nProbe started successfully
15/Dec/2015 09:02:21 [nprobe.c:7044] No more packets to read. Sleeping...
15/Dec/2015 09:02:21 [nprobe.c:4716] nProbe is shutting down...
15/Dec/2015 09:02:21 [nprobe.c:4752] Exporting pending buckets...
15/Dec/2015 09:02:21 [engine.c:2673] About to flush hash (threadId 0)
15/Dec/2015 09:02:21 [engine.c:2675] Completed hash walk (thread 0)
15/Dec/2015 09:02:21 [nprobe.c:4758] Waiting to export queued buckets...
[queue len=2]
15/Dec/2015 09:02:21 [engine.c:2541] Emitting Flow: [->][udp]
172.31.0.0:50101 -> 172.31.200.50:2055 [3257 pkt/1399640 bytes][ifIdx
0->0][0.0 sec][init Unknown][AS: 0 -> 0]
15/Dec/2015 09:02:21 [engine.c:2541] Emitting Flow: [->][udp]
172.31.0.0:50103 -> 172.31.200.50:2055 [99 pkt/8112 bytes][ifIdx 0->0][0.0
sec][init Unknown][AS: 0 -> 0]
15/Dec/2015 09:02:22 [export.c:1266] Sending 2 flows (NetFlow v5 format)
15/Dec/2015 09:02:22 [nprobe.c:4773] Pending buckets have been exported...
15/Dec/2015 09:02:24 [engine.c:3293] Export thread terminated
[exportQueue=0]
15/Dec/2015 09:02:24 [nprobe.c:4839] Flushing queued flows...
15/Dec/2015 09:02:24 [nprobe.c:4842] Freeing memory...
15/Dec/2015 09:02:24 [plugin.c:277] Terminating plugins.
15/Dec/2015 09:02:24 [cache.c:1200] Redis Cache [0 total/0.0 get/sec][0
total/0.0 set/sec]
15/Dec/2015 09:02:24 [nprobe.c:4934] Still allocated 0 hash buckets
15/Dec/2015 09:02:24 [nprobe.c:2457] Processed packets: 3356 (max bucket
search: 0)
15/Dec/2015 09:02:24 [nprobe.c:2440] Fragment queue length: 0
15/Dec/2015 09:02:24 [nprobe.c:2466] Flow export stats: [1407752 bytes/3356
pkts][2 flows/1 pkts sent]
15/Dec/2015 09:02:24 [nprobe.c:2476] Flow drop stats: [0 bytes/0 pkts][0
flows]
15/Dec/2015 09:02:24 [nprobe.c:2481] Total flow stats: [1407752 bytes/3356
pkts][2 flows/1 pkts sent]
15/Dec/2015 09:02:24 [nprobe.c:4947] Cleaning globals
15/Dec/2015 09:02:24 [nprobe.c:4967] nProbe terminated.

On Tue, Dec 15, 2015 at 1:51 AM, ValentinaViscarelli <
[email protected]> wrote:

Smerkal,
please do this:

1. check you don't have nprobe and ntopng instances run.
2. Open a shell and run this command:
   nprobe -i none -n none -3 2055 --zmq tcp://127.0.0.1:1234 -b2
3. Open another shell and run this command:
   ntopng -i tcp://127.0.0.1:1234
4. Open another shell and run this command (use pcap that you attached):
   nprobe -i cflow.pcap -b2
5. wait a couple of minutes
6. send me the output of three commands.

Thanks

—
Reply to this email directly or view it on GitHub
#10 (comment).

[ValentinaViscarelli]

Smerkal,

this was just a try. I wanted to see if ntopng received something.
it's normal you see only "two hosts exchanging flow data". I try to explain....
The nprobe instance at point 4 simulate your router; so if you use in input a pcap file with a IPFIX traffic it's wrong. Your router receive in input normal traffic and export IPFIX traffic; so you have to use in input a pcap with normal traffic. Try this and if it works, repeat the procedure without point 4 but with your router that exports IPFIX flows.
If it doesn't work, please send me commands outputs and after, if possible, we can think a remote connection.

Thanks

[smerkal]

nProbe works just fine if using -i or feeding it a .pcap file of
normal transient traffic captured from the router. It just does not work
when acting as a proxy for the flow data received from the router. It acts
like it doesn't see the data coming in, even though I can verify that the
data is being received. I have tried sending it IPFIX from Juniper and
Netflow v5 from Cisco at point 4 with the same results.

Erik

root@uncsnbox:/home/nbox# nprobe -i none -n none -3 2055 --zmq tcp://
127.0.0.1:1234 -b 2
16/Dec/2015 13:35:17 [nprobe.c:3130] Valid nProbe Pro license found
16/Dec/2015 13:35:17 [plugin.c:166] No plugins found in ./plugins
16/Dec/2015 13:35:17 [plugin.c:174] Loading 22 plugins [.so] from
/usr/local/lib/nprobe/plugins
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin DHCP Protocol:
missing license [/etc/nprobe.license.dhcp]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin Diameter
Protocol: missing license [/etc/nprobe.license.diameter]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin DNS Protocol:
missing license [/etc/nprobe.license.dns]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin Export Plugin:
missing license [/etc/nprobe.license.export]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin FTP Protocol:
missing license [/etc/nprobe.license.ftp]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin GTPv0 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv0]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin GTPv1 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv1]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin GTPv2 Signaling
Protocol: missing license [/etc/nprobe.license.gtpv2]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin HTTP Protocol:
missing license [/etc/nprobe.license.http]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin IMAP Protocol:
missing license [/etc/nprobe.license.email]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin Netflow-Lite
Plugin: missing license [/etc/nprobe.license.nflite]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin Oracle
Protocol: missing license [/etc/nprobe.license.oracle]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin POP3 Protocol:
missing license [/etc/nprobe.license.email]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin System process
information: missing license [/etc/nprobe.license.process]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin Radius
Protocol: missing license [/etc/nprobe.license.radius]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin RTP Plugin:
missing license [/etc/nprobe.license.voippro]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin S1AP Protocol:
missing license [/etc/nprobe.license.S1AP]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin SIP Plugin:
missing license [/etc/nprobe.license.voippro]
16/Dec/2015 13:35:17 [plugin.c:742] Unable to enable plugin SMTP Protocol:
missing license [/etc/nprobe.license.email]
16/Dec/2015 13:35:17 [nprobe.c:4488] WARNING: The output interfaceId is set
to 0: did you forget to use -Q perhaps ?
16/Dec/2015 13:35:17 [nprobe.c:4491] WARNING: The input interfaceId is set
to 0: did you forget to use -u perhaps ?
16/Dec/2015 13:35:17 [nprobe.c:4552] Welcome to nProbe Pro v.7.2.151211
($Revision: 4471 $) for x86_64-unknown-linux-gnu with native PF_RING
acceleration
16/Dec/2015 13:35:17 [nprobe.c:4562] Running on Ubuntu 14.04.2 LTS
16/Dec/2015 13:35:17 [nprobe.c:4573] [LICENSE] nProbe SystemId:
FA623D157104A1D2
16/Dec/2015 13:35:17 [nprobe.c:4620] Tracing enabled
16/Dec/2015 13:35:17 [bgpPlugin.c:375] BGP plugin is disabled (--bgp-port
has not been specified)
16/Dec/2015 13:35:17 [dbPlugin.c:49] Initializing DB plugin
16/Dec/2015 13:35:17 [mysqlPlugin.c:111] Initialized MySQL plugin
16/Dec/2015 13:35:17 [plugin.c:248] 3 plugin(s) loaded [3 delete][2 packet].
16/Dec/2015 13:35:17 [nprobe.c:6526] Welcome to nprobe v.7.2.151211 for
x86_64-unknown-linux-gnu
16/Dec/2015 13:35:17 [nprobe.c:5752] Compiling flow templates...
16/Dec/2015 13:35:17 [plugin.c:851] Scanning plugin BGP Update Listener
[bgp]
16/Dec/2015 13:35:17 [plugin.c:851] Scanning plugin MySQL DB [db]
16/Dec/2015 13:35:17 [plugin.c:851] Scanning plugin MySQL Plugin [mysql]
16/Dec/2015 13:35:17 [plugin.c:1000] 0 plugin(s) enabled
16/Dec/2015 13:35:17 [nprobe.c:6203] Non IPv4/v6 traffic is discarded
according to the template
16/Dec/2015 13:35:17 [util.c:287] GeoIP: loaded AS config file
/usr/share/ntopng/httpdocs/geoip/GeoIPASNum.dat
16/Dec/2015 13:35:17 [util.c:296] GeoIP: loaded AS IPv6 config file
/usr/share/ntopng/httpdocs/geoip/GeoIPASNumv6.dat
16/Dec/2015 13:35:17 [nprobe.c:5121] Using packet capture length 128
16/Dec/2015 13:35:17 [nprobe.c:6698] IPv6 traffic will NOT be
exported/accounted by this probe
16/Dec/2015 13:35:17 [nprobe.c:6699] due to configuration options (e.g. use
NetFlow v9)
16/Dec/2015 13:35:17 [nprobe.c:6702] The flows hash has 131072 buckets
16/Dec/2015 13:35:17 [nprobe.c:6704] Flows older than 120 seconds will be
exported
16/Dec/2015 13:35:17 [nprobe.c:6707] Flows inactive for at least 30 seconds
will be exported
16/Dec/2015 13:35:17 [nprobe.c:6710] Expired flows will not be queued for
more than 30 seconds
16/Dec/2015 13:35:17 [nprobe.c:6717] Exported flows with engineType 0 and
engineId 245
16/Dec/2015 13:35:17 [nprobe.c:6739] TCP TOS will be ignored and set to 0.
16/Dec/2015 13:35:17 [nprobe.c:6757] After 1 flow packets are sent, we'll
delay at least 1 ms
16/Dec/2015 13:35:17 [nprobe.c:6777] Flows will be emitted in NetFlow 5
format
16/Dec/2015 13:35:17 [nprobe.c:6807] Flow input interface index is set to 0
16/Dec/2015 13:35:17 [nprobe.c:6813] Flow output interface index is set to 0
16/Dec/2015 13:35:17 [nprobe.c:6827] Not capturing packet from interface
(collector mode)
16/Dec/2015 13:35:17 [util.c:3840] Succesfully created ZMQ endpoint tcp://
127.0.0.1:1234
16/Dec/2015 13:35:17 [plugin.c:813] Disabling plugin BGP Update Listener
(no template is using it)
16/Dec/2015 13:35:17 [plugin.c:813] Disabling plugin MySQL DB (no template
is using it)
16/Dec/2015 13:35:17 [plugin.c:813] Disabling plugin MySQL Plugin (no
template is using it)
16/Dec/2015 13:35:17 [collect.c:86] Created UDP sockets
16/Dec/2015 13:35:17 [collect.c:90] Created a SCTP socket (102)
16/Dec/2015 13:35:17 [collect.c:145] Flow collector listening on port 2055
(IPv4/v6)
16/Dec/2015 13:35:17 [nprobe.c:6947] Starting 1 packet fetch thread(s)
16/Dec/2015 13:35:17 [engine.c:3210] Starting bucket dequeue thread
16/Dec/2015 13:35:17 [nprobe.c:7035] nProbe started successfully
^C16/Dec/2015 13:37:16 [cache.c:1200] Redis Cache [0 total/0.0 get/sec][0
total/0.0 set/sec]
16/Dec/2015 13:37:16 [nprobe.c:386] Received shutdown request... [signal: 2]
16/Dec/2015 13:37:16 [nprobe.c:4716] nProbe is shutting down...
16/Dec/2015 13:37:16 [nprobe.c:4752] Exporting pending buckets...
16/Dec/2015 13:37:16 [nprobe.c:4773] Pending buckets have been exported...
16/Dec/2015 13:37:18 [engine.c:3293] Export thread terminated
[exportQueue=0]
16/Dec/2015 13:37:18 [nprobe.c:4839] Flushing queued flows...
16/Dec/2015 13:37:18 [nprobe.c:4842] Freeing memory...
16/Dec/2015 13:37:18 [plugin.c:277] Terminating plugins.
16/Dec/2015 13:37:18 [cache.c:1200] Redis Cache [0 total/0.0 get/sec][0
total/0.0 set/sec]
16/Dec/2015 13:37:18 [nprobe.c:4934] Still allocated 0 hash buckets
16/Dec/2015 13:37:18 [nprobe.c:2457] Processed packets: 0 (max bucket
search: 0)
16/Dec/2015 13:37:18 [nprobe.c:2440] Fragment queue length: 0
16/Dec/2015 13:37:18 [nprobe.c:2466] Flow export stats: [0 bytes/0 pkts][0
flows/0 pkts sent]
16/Dec/2015 13:37:18 [nprobe.c:2473] Flow collection: [collected pkts:
0][processed flows: 0]
16/Dec/2015 13:37:18 [nprobe.c:2476] Flow drop stats: [0 bytes/0 pkts][0
flows]
16/Dec/2015 13:37:18 [nprobe.c:2481] Total flow stats: [0 bytes/0 pkts][0
flows/0 pkts sent]
16/Dec/2015 13:37:18 [nprobe.c:4947] Cleaning globals
16/Dec/2015 13:37:18 [nprobe.c:4967] nProbe terminated.

root@uncsnbox:/home/nbox# ntopng -i tcp://127.0.0.1:1234
16/Dec/2015 13:35:27 [Ntop.cpp:933] Setting local networks to 127.0.0.0/8
16/Dec/2015 13:35:27 [Redis.cpp:106] Successfully connected to redis
127.0.0.1:6379@0
16/Dec/2015 13:35:27 [NtopPro.cpp:119] [LICENSE] Reading license from
/etc/ntopng.license
16/Dec/2015 13:35:27 [Ntop.cpp:1152] Registered interface tcp://
127.0.0.1:1234 [id: 0]
16/Dec/2015 13:35:27 [Ntop.cpp:1165] Registered interface view tcp://
127.0.0.1:1234 [id: 0]
16/Dec/2015 13:35:27 [Utils.cpp:304] User changed to nobody
16/Dec/2015 13:35:27 [main.cpp:240] PID stored in file /var/tmp/ntopng.pid
16/Dec/2015 13:35:27 [HTTPserver.cpp:465] Please read
https://github.com/ntop/ntopng/blob/dev/doc/README.SSL if you want to
enable SSL.
16/Dec/2015 13:35:27 [HTTPserver.cpp:482] -->3000<--
16/Dec/2015 13:35:27 [HTTPserver.cpp:510] Web server dirs
[/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts]
16/Dec/2015 13:35:27 [HTTPserver.cpp:513] HTTP server listening on port 3000
16/Dec/2015 13:35:27 [main.cpp:290] Working directory: /var/tmp/ntopng
16/Dec/2015 13:35:27 [main.cpp:292] Scripts/HTML pages directory:
/usr/share/ntopng
16/Dec/2015 13:35:27 [Ntop.cpp:260] Welcome to ntopng x86_64 v.2.2.151211 -
(C) 1998-15 ntop.org
16/Dec/2015 13:35:27 [Ntop.cpp:265] Built on Ubuntu 14.04.2 LTS
16/Dec/2015 13:35:27 [PeriodicActivities.cpp:53] Started periodic
activities loop...
16/Dec/2015 13:35:27 [RuntimePrefs.cpp:32] Dumping alerts into syslog
16/Dec/2015 13:35:27 [NtopPro.cpp:233] [LICENSE] ntopng systemId:
FA623D157104A1D2
16/Dec/2015 13:35:27 [NtopPro.cpp:244] [LICENSE] ntopng license:
2163EA9A6D3FEBD13E0940ACB875D3DC1480454122251EC47F
16/Dec/2015 13:35:27 [NtopPro.cpp:265] [LICENSE] Maintenance is available
until Tue Nov 29 15:15:22 2016 [349 days left]
16/Dec/2015 13:35:27 [NetworkInterface.cpp:1426] Started packet polling on
interface tcp://127.0.0.1:1234 [id: 0]...
16/Dec/2015 13:35:28 [CollectorInterface.cpp:94] Collecting flows on tcp://
127.0.0.1:1234
^C16/Dec/2015 13:37:10 [main.cpp:37] Shutting down...
16/Dec/2015 13:37:12 [ProtoStats.cpp:35] [IPv4] 0 B/0.00 Packets
16/Dec/2015 13:37:12 [ProtoStats.cpp:35] [IPv6] 0 B/0.00 Packets
16/Dec/2015 13:37:12 [ProtoStats.cpp:35] [ARP] 0 B/0.00 Packets
16/Dec/2015 13:37:12 [ProtoStats.cpp:35] [MPLS] 0 B/0.00 Packets
16/Dec/2015 13:37:12 [ProtoStats.cpp:35] [Other] 0 B/0.00 Packets
16/Dec/2015 13:37:13 [Ntop.cpp:1191] Interface tcp://127.0.0.1:1234
[running: 0]
16/Dec/2015 13:37:13 [main.cpp:48] Deleted PID /var/tmp/ntopng.pid [rc: 0]
16/Dec/2015 13:37:13 [HTTPserver.cpp:525] HTTP server terminated
16/Dec/2015 13:37:13 [AddressResolution.cpp:54] Address resolution stats [0
resolved][0 failures]

root@uncsnbox:/home/nbox# tcpdump -n -l -i p1p1 port 2055
13:36:52.134060 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.135056 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.136078 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.137066 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.138039 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.139069 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.140067 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.141067 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.142079 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.143057 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.144053 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.145054 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.146059 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.147057 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.148059 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.149054 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.150058 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.151056 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.152057 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.153056 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.154054 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.155056 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.156057 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.157057 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.158057 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.159057 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.160056 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.161056 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420
13:36:52.162056 IP 172.31.0.0.50101 > 172.31.200.50.2055: UDP, length 420

On Wed, Dec 16, 2015 at 3:32 AM, ValentinaViscarelli <
[email protected]> wrote:

Smerkal,

this was just a try. I wanted to see if ntopng received something.
it's normal you see only "two hosts exchanging flow data". I try to
explain....
The nprobe instance at point 4 simulate your router; so if you use in
input a pcap file with a IPFIX traffic it's wrong. Your router receive in
input normal traffic and export IPFIX traffic; so you have to use in input
a pcap with normal traffic. Try this and if it works, repeat the procedure
without point 4 but with your router that exports IPFIX flows.
If it doesn't work, please send me commands outputs and after, if
possible, we can think a remote connection.

Thanks

—
Reply to this email directly or view it on GitHub
#10 (comment).

[ValentinaViscarelli]

Erik,
I think you have a problem with firewall.... I believe that firewall allows traffic on port 2055 from localhost and denies traffic from remote.
You see the traffic with tcpdump because it acts on level 2 but then the traffic is blocked by "iptables".
Please try these two scenarios with netcat:

Scenario 1:

1. run this command:
   netcat -ul 2055 | hexdump -C
2. run nprobe:
   nprobe -i file.pcap

Scenario 2:

1. run this command:
   netcat -ul 2055 | hexdump -C
2. send flow data with router

If you have problem with firewall in "Scenario 2" you should receive no bytes on netcat.

[smerkal]

I see nothing in scenario 2, UFW (iptables) is disabled.

On Wed, Dec 16, 2015 at 4:32 PM, ValentinaViscarelli <
[email protected]> wrote:

Erik,
I think you have a problem with firewall.... I believe that firewall
allows traffic on port 2055 from localhost and denies traffic from remote.
You see the traffic with tcpdump because it acts on level 2 but then the
traffic is blocked by "iptables".
Please try these two scenarios with netcat:

Scenario 1:

1. run this command:
   netcat -ul 2055 | hexdump -C
2. run nprobe:
   nprobe -i file.pcap

Scenario 2:

1. run this command:
   netcat -ul 2055 | hexdump -C
2. send flow data with router

If you have problem with firewall in "Scenario 2" you should receive no
bytes on netcat.

—
Reply to this email directly or view it on GitHub
#10 (comment).

[smerkal]

Resolved. Apparently Ubuntu is doing RPF checks. Flow data was coming in on
one interface but the route back to the source of the flow data was out
another interface (default gateway) and traffic was being rejected. Adding
a route to the flow source through the interface that it is being received
on (or disabling rp_filter) resolved the issue.

Thank you for the assistance and I apologize for wasting your time.

On Wed, Dec 16, 2015 at 5:25 PM, Erik Schmersal [email protected] wrote:

I see nothing in scenario 2, UFW (iptables) is disabled.

On Wed, Dec 16, 2015 at 4:32 PM, ValentinaViscarelli <
[email protected]> wrote:

Erik,
I think you have a problem with firewall.... I believe that firewall
allows traffic on port 2055 from localhost and denies traffic from remote.
You see the traffic with tcpdump because it acts on level 2 but then the
traffic is blocked by "iptables".
Please try these two scenarios with netcat:

Scenario 1:

1. run this command:
   netcat -ul 2055 | hexdump -C
2. run nprobe:
   nprobe -i file.pcap

Scenario 2:

1. run this command:
   netcat -ul 2055 | hexdump -C
2. send flow data with router

If you have problem with firewall in "Scenario 2" you should receive no
bytes on netcat.

—
Reply to this email directly or view it on GitHub
#10 (comment).

[ValentinaViscarelli]

Hi Erik,
no problem. Don't hesitate to contact us if you have other problems.

Cheers,
Valentina