Matching not correct. about ndpi-netfilter

[fcastello]

I got it working on ubuntu 14.04 and 12.04. It work's. compiles and let me use iptables targets but. I don't think it's doing what it is supposed to do.

[betolj]

I did some tests yesterday and it has worked for me.
For example:

iptables -I INPUT -m ndpi --http -j DROP
lynx www.google.com
2 613 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol HTTP

iptables -I INPUT -m ndpi --ssh -j DROP
ssh x.y.z.w -p 2222
19 26728 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol SSH

iptables -I OUTPUT -m ndpi --rdp -j DROP
rdesktop x.y.z.w
7 602 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol RDP

Unfortunately, I noticed that the choice of the proper chain depends on the flow through which data was received.

And, unless you recompile the kernel with the patch, the module nf_conntrack_netlink can not be loaded into memory (but you will have problems with conntrack tool).
rmmod nf_conntrack_netlink
modprobe xt_ndpi

[betolj]

root@humberto-XPS-8300:# cat /etc/debian_version
jessie/sid
root@humberto-XPS-8300:# uname -a
Linux humberto-XPS-8300.ms 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

root@humberto-XPS-8300:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.1 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.1 LTS"

[betolj]

For ssh you must have at INPUT (or http).
iptables -I INPUT -m ndpi --ssh -j DROP

The signature is in the response.

For routed traffic you probably will not have any problem because the rule may cover both directions, but be careful to define the flow in the rule.

[fcastello]

I have been testing it and you are correct.
It is not matching http services like google facebook whatsapp etc.
I am trying to find out why.
It is not working with any of the services defined in nDPI/src/lib/ndpi_content_match.c.inc

[betolj]

Yes... and this is an old problem!

Apparently, it is more safe and guaranteed to work with protocols where there was a specific module build (ftp_data, http, rdp and others).

Most filters in nDPI/src/lib/ndpi_content_match.c.inc or do not work well or take unwanted action. When I filter for twitter, several sites that reference it did not open. This is certainly a problem.

[betolj]

It works well for protocols in /usr/src/nDPI/src/lib/protocols/
This is why works with the twitter

[fcastello]

I have no idea. I will get some pcaps and see why is matchin twitter and not the others.
The good thing is that I don't want to block twitter. I just want to apply some traffic control policy to it. Which won't be as bad as filterning anything with the word twitter

[fcastello]

In ntop everything works like a charm, I guess they do analisis of the entire network flow matching the criteria.

[betolj]

It's different.

I think that the filters in ndpi_content_match.c.inc are internal for nDPI and netfilter can not use them because it depends on which modules are compiled and integrated.

The url filters is also indicated in Squid or other proxy.
You can use zph to set a specific DSCP if you are looking for QoS.

[syadnom]

I'm building against 8323 and it's not matching anything... This is the most recent nDPI I can get the build module to load.

[betolj]

Hi, Unfortunately this module is incompatible with nf_conntrack_netlink.

And, unless you recompile the kernel with the patch, the module
nf_conntrack_netlink can not be loaded into memory (but you will have
problems with conntrack tool).

rmmod nf_conntrack_netlink
modprobe xt_ndpi

2014-11-13 15:22 GMT-03:00 syadnom [email protected]:

I'm building against 8323 and it's not matching anything... This is the
most recent nDPI I can get the build module to load.

—
Reply to this email directly or view it on GitHub
#1 (comment).

[betolj]

I will make several fixes as soon and protocol id bugfix too.
Wait a little bit.

[alrferreira]

Hello,
I'm testing and really many filters do not work. I would like to contribute you with signatures, but first wanted to know what the situation is.

[mcitew]

I tested this too. I found it's working for some few protocols such as SSL, http, ftp.

But one of the most important protocol, youtube ( specifically youtube over SSL ) is not working at all.

I just add an iptables rule,

iptables -I FORWARD -m ndpi --youtube 

Matches nothing. But when I run ndpiReader -i interface, it picks up youtube traffic

[Mile-Lile]

same for me. Did you founded solution?

[betolj]

Hi,

I'm working in a patch for the latest nDPI version and i will include support
for youtube too.
And i'm currently testing youtube detection - it seems to be working.

I'll probably update the repository tomorrow.

2015-06-11 4:49 GMT-04:00 Mile-Lile [email protected]:

same for me. Did you founded solution?

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[alrferreira]

Working to me when I enable first a SSL Rule. But Skype don't.

André Luiz R. Ferreira - NETDEEP

-----Mensagem Original-----
De: "Humberto Jucá" [email protected]
Enviada em: ‎11/‎06/‎2015 22:45
Para: "betolj/ndpi-netfilter" [email protected]
Cc: "alrferreira" [email protected]
Assunto: Re: [ndpi-netfilter] Matching not correct. (#1)

Hi,

I'm working in a patch for the latest nDPI version and i will include support
for youtube too.
And i'm currently testing youtube detection - it seems to be working.

I'll probably update the repository tomorrow.

2015-06-11 4:49 GMT-04:00 Mile-Lile [email protected]:

same for me. Did you founded solution?

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

—
Reply to this email directly or view it on GitHub.
*** Secured by Netdeep ClearMail ***

[Mile-Lile]

@betolj
Thx. appreciate it!

[betolj]

The "ndpi-netfilter" projects with logical structure based on "ewildgoose" model don't work well nowadays.

1. Only one exclusive "conntrack notify" call: For this reason, it cann't be used in conjunction with nfnetlink. But, when you remove the nfnetlink kernel module, the conntrack application wont works anymore.
2. The web host detection depends on http or ssl protocols enabled: Youtube or Facebook protocols are not external modules (like /usr/src/nDPI/src/lib/protocols/*).

I finished xt_ndpi fixes today:

• Remove "conntrack notifier" from source code and include a new ndpi flow garbage collector for 3.x kernel series. Now, no longer need remove the nfnetlink kernel module to install xt_ndpi.
• Enable http and ssl protocol for youtube, facebook and others.

My tests, showed satisfactory results.
https://github.com/betolj/ndpi-netfilter

[Mile-Lile]

Thx again!

[hemanth4ap]

how to block particular http/https urls by using ndpi