[Still NOT CLOSED] Why the JSON output isn't created? about ndpi

[lucaderi]

@frforesta configure checks if the json-c library is installed. If so, JSON is generated. I think this is your problem

[unusualfor]

@lucaderi already checked, of course. The environment is perfectly build: the old dpi works well and writes the json, the new one does not. But I really appreciate your help, thanks.

[unusualfor]

@lucaderi You have closed the topic, but the issue remains :/ and this is not due to json-c or any other JSON library, cause they are working with the old program in the same environment!
I'm telling you for future users, not for my personal's sake :)

[lucaderi]

@frforesta Is the code on git able to reproduce the issue?

[lucaderi]

@frforesta It works for me
[email protected] 244> ./ndpiReader -i ~/pcap/http/http.pcap -j capture.json -v 2

nDPI Memory statistics:
nDPI Memory (once): 94.54 KB
Flow Memory (per flow): 1.92 KB
Actual Memory: 1.61 MB
Peak Memory: 1.61 MB
[email protected] 245> cat capture.json
{ "traffic.statistics": { "ethernet.bytes": 136886, "discarded.bytes": 0, "ip.packets": 191, "total.packets": 191, "ip.bytes": 132302, "avg.pkt.size": 692, "unique.flows": 6, "tcp.pkts": 191, "udp.pkts": 0, "vlan.pkts": 0, "mpls.pkts": 0, "pppoe.pkts": 0, "fragmented.pkts": 0, "max.pkt.size": 1472, "pkt.len_min64": 89, "pkt.len_64_128": 0, "pkt.len_128_256": 3, "pkt.len_256_1024": 19, "pkt.len_1024_1500": 80, "pkt.len_grt1500": 0, "guessed.flow.protos": 0 }, "detected.protos": [ { "name": "HTTP", "breed": "Acceptable", "packets": 191, "bytes": 132302, "flows": 6 } ], "known.flows": [ { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50140, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 6, "bytes": 914, "host.server.name": "www.macintouch.com" }, { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50141, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 6, "bytes": 941, "host.server.name": "www.macintouch.com" }, { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50142, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 6, "bytes": 938, "host.server.name": "www.macintouch.com" }, { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50143, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 6, "bytes": 941, "host.server.name": "www.macintouch.com" }, { "protocol": "TCP", "host_a.name": "207.210.67.146", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50144, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 6, "bytes": 2104, "host.server.name": "api.leoslyrics.com" }, { "protocol": "TCP", "host_a.name": "207.44.206.43", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50145, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 6, "bytes": 1062, "host.server.name": "elyrics.net" }, { "protocol": "TCP", "host_a.name": "207.210.67.146", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50144, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 19, "bytes": 10510, "host.server.name": "api.leoslyrics.com" }, { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50140, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 68, "bytes": 53426, "host.server.name": "www.macintouch.com" }, { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50142, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 21, "bytes": 12763, "host.server.name": "www.macintouch.com" }, { "protocol": "TCP", "host_a.name": "207.44.206.43", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50145, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 10, "bytes": 1326, "host.server.name": "elyrics.net" }, { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50141, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 35, "bytes": 25995, "host.server.name": "www.macintouch.com" }, { "protocol": "TCP", "host_a.name": "64.243.24.160", "host_a.port": 80, "host_b.name": "192.168.0.200", "host_n.port": 50143, "detected.protocol": 7, "detected.protocol.name": "HTTP", "packets": 38, "bytes": 28282, "host.server.name": "www.macintouch.com" } ] }

[unusualfor]

@lucaderi I've used the git code and tried on my PC (Ubuntu 14.04.3), on two different VMs built from different images, always Ubuntu, and a Fedora one. In all of them I've done all the steps required in order to make it work, but it still doesn't work properly. It starts, it goes on, but it doesn't create the json file. In all of these machines are been installed the json libraries and all things are checked (we have been using the dpi for at least one year, so we were keen on all the little issues concerned with the dpi).
After the required step are made (also the one which the dpi process has to be closed with a SIGINT), we don't have the capture.json file.

francesco@X550LD:/nDPI/example$ ls
Makefile Makefile.am Makefile.in ndpiReader ndpiReader.c ndpiReader.o protos.txt Win32
francesco@X550LD:/nDPI/example$ sudo ./ndpiReader -i lo -v 2 -j capture.json

nDPI Memory statistics:
nDPI Memory (once): 94.54 KB
Flow Memory (per flow): 1.92 KB
Actual Memory: 1.64 MB
Peak Memory: 1.64 MB
francesco@X550LD:/nDPI/example$ ls
Makefile Makefile.am Makefile.in ndpiReader ndpiReader.c ndpiReader.o protos.txt Win32
francesco@X550LD:/nDPI/example$ cat capture.json
cat: capture.json: No such file or directory

On the same environment on every VM or host in general, the old version prints the json well.

francesco@X550LD:/nDPI_Tool/nDPI/example$ ls
dpi.cap Makefile Makefile.am Makefile.in ndpiReader ndpiReader.c ndpiReader.o protos.txt Win32
francesco@X550LD:/nDPI_Tool/nDPI/example$ sudo ./ndpiReader -i lo -v 2 -j capture.json

francesco@X550LD:~/nDPI_Tool/nDPI/example$ cat capture.json | python -m json.tool
{
"detected.protos": [
{
"bytes": 1536,
"flows": 12,
"name": "Unknown",
"packets": 24
},
{
"bytes": 588,
"flows": 1,
"name": "ICMP",
"packets": 6
}
],
"known.flows": [
{
"bytes": 98,
"detected.protocol": 81,
"detected.protocol.name": "ICMP",
"host_a.name": "127.0.0.1",
"host_a.port": 0,
"host_b.name": "127.0.0.1",
"host_n.port": 0,
"packets": 1,
"protocol": "ICMP"
},
{
"bytes": 588,
"detected.protocol": 81,
"detected.protocol.name": "ICMP",
"host_a.name": "127.0.0.1",
"host_a.port": 0,
"host_b.name": "127.0.0.1",
"host_n.port": 0,
"packets": 6,
"protocol": "ICMP"
}
],
"traffic.statistics": {
"avg.pkt.size": 70,
"discarded.bytes": 0,
"ethernet.bytes": 2844,
"fragmented.pkts": 0,
"guessed.flow.protos": 0,
"ip.bytes": 2124,
"ip.packets": 30,
"max.pkt.size": 64,
"mpls.pkts": 0,
"pkt.len_1024_1500": 0,
"pkt.len_128_256": 0,
"pkt.len_256_1024": 0,
"pkt.len_64_128": 6,
"pkt.len_grt1500": 0,
"pkt.len_min64": 24,
"pppoe.pkts": 0,
"tcp.pkts": 24,
"total.packets": 30,
"udp.pkts": 0,
"unique.flows": 13,
"vlan.pkts": 0
},
"unknown.flows": [
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43960,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43962,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43964,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43966,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43968,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43970,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43959,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43961,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43963,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43965,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43967,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
},
{
"bytes": 128,
"detected.protocol": 0,
"detected.protocol.name": "Unknown",
"host_a.name": "127.0.0.1",
"host_a.port": 43969,
"host_b.name": "127.0.0.1",
"host_n.port": 6633,
"packets": 2,
"protocol": "TCP"
}
]
}

[lucaderi]

@frforesta See below

./ndpiReader -i lo -v 2 -j capture.json

^C
nDPI Memory statistics:
nDPI Memory (once): 94.54 KB
Flow Memory (per flow): 1.92 KB
Actual Memory: 1.60 MB
Peak Memory: 1.60 MB

ls -l capture.json

-rw-r--r-- 1 root root 1001 Jun 24 13:46 capture.json
root@i7:/home/deri/nDPI/example# cat capture.json
{ "traffic.statistics": { "ethernet.bytes": 1264, "discarded.bytes": 0, "ip.packets": 12, "total.packets": 12, "ip.bytes": 976, "avg.pkt.size": 81, "unique.flows": 1, "tcp.pkts": 12, "udp.pkts": 0, "vlan.pkts": 0, "mpls.pkts": 0, "pppoe.pkts": 0, "fragmented.pkts": 0, "max.pkt.size": 73, "pkt.len_min64": 8, "pkt.len_64_128": 4, "pkt.len_128_256": 0, "pkt.len_256_1024": 0, "pkt.len_1024_1500": 0, "pkt.len_grt1500": 0, "guessed.flow.protos": 1 }, "detected.protos": [ { "name": "Redis", "breed": "Acceptable", "packets": 12, "bytes": 976, "flows": 1 } ], "known.flows": [ { "protocol": "TCP", "host_a.name": "127.0.0.1", "host_a.port": 46377, "host_b.name": "127.0.0.1", "host_n.port": 6379, "detected.protocol": 182, "detected.protocol.name": "Redis", "packets": 11, "bytes": 910 }, { "protocol": "TCP", "host_a.name": "127.0.0.1", "host_a.port": 46377, "host_b.name": "127.0.0.1", "host_n.port": 6379, "detected.protocol": 182, "detected.protocol.name": "Redis", "packets": 12, "bytes": 976 } ] }

[unusualfor]

@lucaderi it's exactly what I've done, as you may have seen above. Of course, I've run it as superuser, otherwise it won't use the pcap library.

I've understood that it worked for you, I've only tried to submit a bug that probably affects this program, but it doesn't matter. Since everything I've done is correct, I'll be using the old version, that, I repeat, in the same environment works well (so I thought it was a program problem). The report was indeed created just to notify you this as it could happen again with other users.

Bye and thanks for your time.

[lucaderi]

@frforesta Can you please run gdb and see what happens?