Coder Social home page Coder Social logo

ndpi-netfilter's Introduction

This package is a GPL implementation of an iptables and netfilter module for nDPI integration into the Linux kernel.

PayPal — The safer, easier way to pay online.

The prerequisites are:

  • Tested on Ubuntu 14.04.1 LTS (Kernel 3.13.0-37-generic)
  • Following packages to compile kernel-modules: linux-headers iptables-dev >= version 1.4.21-1ubuntu1 nDPI source package

Compiled kernel features

You do not need to do the below steps for Ubuntu 14.04.1 LTS

In order to use nDPI as a kernel module notice that:

  • You should ENABLE Netfilter conntrack events (and also enable Advanced netfilter features to see it).

In kernel 2.6.34 or greater its defined as:

Connection tracking events Symbol: NF_CONNTRACK_EVENTS Location: -> Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Core Netfilter Configuration -> Netfilter connection tracking support

In kernel 2.6.34 or greater its defined as:

Connection tracking netlink interface Symbol: NF_CT_NETLINK Location: -> Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Core Netfilter Configuration -> Netfilter connection tracking support

Once you have downloaded/installed each package and checked for the above kernel features you can read the INSTALL file.

ndpi-netfilter's People

Contributors

betolj avatar ccdkp avatar maartenw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ndpi-netfilter's Issues

Unable to compile

Hi,
I've downloaded the latest source, but I'm unable to complete the installation. The process is stopping following this command:

NDPI_PATH=/root/ndpi-master/ndpi-netfilter-master/nDPI make

The error it's giving is:

cp ndpi_cpy/lib/third_party/include/.h ndpi_cpy/lib -R;
cp ndpi_cpy/lib/third_party/include/
.h ndpi_cpy/include -R;
sed -i "s/^\s*void ndpi_free_flow///void ndpi_free_flow/" ndpi_cpy/include/ndpi_api.h;
make -C /lib/modules/4.9.0.minkernel-mid/build M=$PWD;
make[2]: Entering directory '/root/ndpi-master/ndpi-netfilter-master/src'
make[2]: *** /lib/modules/4.9.0.minkernel-mid/build: No such file or directory. Stop.
make[2]: Leaving directory '/root/ndpi-master/ndpi-netfilter-master/src'
Makefile:155: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/root/ndpi-master/ndpi-netfilter-master/src'
Makefile:5: recipe for target 'all' failed
make: *** [all] Error 2

Can you advise how to overcome this issue?

Thanks
Anubis

Server freezing

Hi Guys,
got 2 servers in the role of an firewall (mainly FORWARD chain filtering)

server 1:
debian jessie 8.6 - kernel 3.16.0-4-amd64

server 2:
debian stretch 9.0 - kernel 4.8.0-1-amd64

The server 1 is an production server, server 2 is for testing purposes.

The thing is, both the servers insta freez (sometimes within 5min, sometimes it lasts some hours -> applying NDPI rules on 20:00 server dead on 04:00) in random intervals (mainly the production one, where the traffic is 200Mbit/s 5min average to 500Mbit/s 5min average). Through the production server goes the traffic of around 500 - 700 client PC's.

1Gbit uplink

The server 1 has much more complicated iptables rules ( ca 1170 in the FORWARD, PREROUTING chain)..., but the server 2 has only few, so the debug is here possible (37 rules in the FORWARD chain, no PREROUTING, some INPUT chain rules just for security reasons).

The test server is much more stable, because the minimal traffic, but freezes are also happening.

If i apply the dpi_check rules:
iptables -t mangle -A PREROUTING -m ndpi --dpi_check
iptables -t mangle -A POSTROUTING -m ndpi --dpi_check

the servers are insta dead.... nothing in the console, the server is unresponsive, nothing on the screen, nothing in the logs. The only possible thing is to reboot the servers.

Any idea how to debug this behavior? Got the NDPI kernel module build from this GIT.

ii iptables 1.6.0+snapshot20161117-5
ii iptables-dev 1.6.0+snapshot20161117-4
ii iptables-persistent 1.0.4+nmu1
ii xtables-addons-common 2.11-3
ii xtables-addons-dkms 2.11-3

the rules are:

-A FORWARD -m ndpi --applejuice --directconnect --gnutella --edonkey --bittorrent --soulseek -m comment --comment "ndpi checker - dropper" -j DROP

The loopback is enabled:
-A INPUT -i lo -j ACCEPT

Add custom protocol

Hi,
How can i define a custom protocol and drop that protocol with netfilter? I have a set of URLs and IPs and i need to define those URLs and IPs as a one protocol and drop them all using single netfilter rule. How can i do this?

Error On Make

It was my bad. Worked like a charm with the archived version.

Add telegram protocol

On latest nDpi SVN Code a new protocol was added: Telegram

To compile with -master series of this project, we need to add this line to Makefile under src.

${NDPI_PRO}/telegram.o

Now it compiles Ok and when I run insmod it works (Ubuntu 14.10)

skb->nfct is NULL every time inside ndpi_mt

Hi,
I am using ndpi-netfilter for openwrt based wireless ACCESS Point .
I am trying to capture the traffic using PREROUTING chain for that purpose i am using following iprule.

iptables -t raw -A PREROUTING -m ndpi --youtube -j ACCEPT

And after this when i am connecting my wireless client to access point and trying to capture the youtube or some other traffic , inside ndpi_mt, nf_ft_get() returns NULL . When i trying to check why it is NULL i found that skb->nfct pointer which is coming inside ndpi_mt is NULL.

Can any onme please help me to resolve this issue.????/

all content-type-matches seem's not to work

Hi,

i am using the latest version from Jun 18. Facebook, Google, Youtube work's all fine,
but all of the streaming protocols doesn't.

For example i tried to download an .mp4 File and dropped via iptables the quicktime-Filter,
which includes the video/mp4 content-type. But the iptables rule does not match.
If i tcpdump the traffic, and analyse it with ndpiReader (compile from the includes nDPI.tar.gz), the traffic is recognised as QUICKTIME.

Kernel Version is V3.14.61

hope for you help
Michael

main.c bug

I have added a new NDPI_SERVICE in 'ndpi_protocol_ids.h' (like wikipedia and HTTP based protocol). This service is the last implemented (#define NDPI_LAST_IMPLEMENTED_PROTOCOL 190).

If you build nDPI for iptables with your tool, this service is never matching because http protocol is not added.

In the current loop, the last protocol/service is never executed for added or not http support protocol

To correct this, apply this path in 'main.c' file (L787) :

        for (i = 0; i < NDPI_LAST_IMPLEMENTED_PROTOCOL +1; i++){
                atomic_set (&protocols_cnt[i], 0);

                // Set HTTP based protocols
                if ((i > 118 && i < 127) || (i > 139 && i < 146) || (i > 175 && i < 182 ) || i == 70 || i == 133) nfndpi_protocols_http[i]=1;
                else nfndpi_protocols_http[i]=0;
        }

ssh protocol redirection

Hi,

In my s-m ndpi -sshystem (debian router) I want to do ssh protocole redirection.
I successfully installed ndpi-netfilter and added following rules for ssh redirection

iptables -t mangle -A PREROUTING -m ndpi --dpi_check
iptables -t mangle -A POSTROUTING -m ndpi --dpi_check
iptables -t nat -A PREROUTING -p tcp -m ndpi -ssh -j REDIRECT --to-ports 9051

with this rules I only get first package redirected to 9051, then redirection stops.

It is working ok with
iptables -t nat -A PREROUTING -p tcp --dport 22 -j REDIRECT --to-ports 9051

but I don't want use ssh port, I want detect and redirect sssh protocole.
Please help to implement this,
it is possible to detect and redirect encrypted protocoles, like ssh, https ?
what am I doing wrong in my iptables configs?

/proc/net/nf_conntrack protocol information

Hi @betolj

Firstly, thank you for maintaining ndpi-netfilter.
I wanted to know if it was possible to write a similar modification to nf_conntrack as seen here for layer7:
nf_conntrack Patch

The idea is that the information at /proc/net/nf_conntrack then contains information about the protocol a connection is using (if it is using one ndpi has identified).
Is there any place we can pull this data from easily and insert it here?

IPv6 support

Is there support for IPv6? (Or will there be?)

While IPv4 works fine, trying to use IPv6 throws this error:

ip6tables v1.4.21: Couldn't load match 'ndpi':No such file or directory

add proto

From libprotoident Are there any ways to add I need to do?

wechat udp
`/*
*

  • Copyright (c) 2011-2016 The University of Waikato, Hamilton, New Zealand.
  • All rights reserved.
  • This file is part of libprotoident.
  • This code has been developed by the University of Waikato WAND
  • research group. For further information please see http://www.wand.net.nz/
  • libprotoident is free software; you can redistribute it and/or modify
  • it under the terms of the GNU Lesser General Public License as published by
  • the Free Software Foundation; either version 3 of the License, or
  • (at your option) any later version.
  • libprotoident is distributed in the hope that it will be useful,
  • but WITHOUT ANY WARRANTY; without even the implied warranty of
  • MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  • GNU Lesser General Public License for more details.
  • You should have received a copy of the GNU Lesser General Public License
  • along with this program. If not, see http://www.gnu.org/licenses/.

*/

#include <string.h>

#include "libprotoident.h"
#include "proto_manager.h"
#include "proto_common.h"

/* Thanks to http://www.cse.cuhk.edu.hk/~pclee/www/pubs/iwqos15chatdissect.pdf

  • for helping confirm this rule */

static inline bool match_wechat_uplink_hb(uint32_t payload, uint32_t len) {

    /* Byte 3 appears to be a length indicator */
    if (MATCH(payload, 0xd1, 0x0a, 0x2e, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x2d, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x2c, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x1e, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x1d, 0x0a))
            return true;

    return false;

}

static inline bool match_wechat_downlink_hb(uint32_t payload, uint32_t len) {

    /* Byte 3 appears to be a length indicator */
    if (MATCHSTR(payload, "\xd1\x0a\x2b\x0a"))
            return true;
    if (MATCHSTR(payload, "\xd1\x0a\x2a\x0a"))
            return true;
    if (MATCHSTR(payload, "\xd1\x0a\x2d\x0a"))
            return true;
    if (MATCHSTR(payload, "\xd1\x0a\x29\x0a"))
            return true;

    return false;

}

static inline bool match_wechat_voip_a175(uint32_t payload, uint32_t len) {

    if (MATCH(payload, 0xa1, 0x08, ANY, ANY) && len == 75)
            return true;

    return false;

}

static inline bool match_wechat_voip_a192(uint32_t payload, uint32_t len) {

    if (MATCH(payload, 0xa1, 0x08, ANY, ANY) && len == 92)
            return true;

    return false;

}

static inline bool match_wechat_voip_a396(uint32_t payload, uint32_t len) {
if (len == 0)
return true;
if (len == 96 && MATCH(payload, 0xa3, ANY, ANY, ANY))
return true;
return false;
}

static inline bool match_wechat_voip_d6200(uint32_t payload, uint32_t len) {
if (len == 200 && MATCH(payload, 0xd6, ANY, ANY, ANY))
return true;
return false;
}

static inline bool match_wechat_voip_d591(uint32_t payload, uint32_t len) {

    if (len < 89 || len > 91)
            return false;

if (MATCH(payload, 0xd5, ANY, ANY, ANY))
	return true;
return false;

}

static inline bool match_wechat_voip_d5104(uint32_t payload, uint32_t len) {

    if (len < 103 || len > 104)
            return false;
if (MATCH(payload, 0xd5, ANY, ANY, ANY))
	return true;
return false;

}

static inline bool match_wechat_udp(lpi_data_t *data, lpi_module_t *mod UNUSED) {

    if (match_wechat_uplink_hb(data->payload[0], data->payload_len[0])) {
            if (match_wechat_downlink_hb(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

    if (match_wechat_uplink_hb(data->payload[1], data->payload_len[1])) {
            if (match_wechat_downlink_hb(data->payload[0],
                            data->payload_len[0]))
                    return true;
    }

/* Lots of different patterns seen when using WeChat to make a voice
 * or video call.
 */
    
    if (match_wechat_voip_a396(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_a396(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

    if (match_wechat_voip_d6200(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_d6200(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }
    
if (match_wechat_voip_d591(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_d5104(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

if (match_wechat_voip_d591(data->payload[1], data->payload_len[1])) {
            if (match_wechat_voip_d5104(data->payload[0],
                            data->payload_len[0]))
                    return true;
    }

    if (match_wechat_voip_a192(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_a175(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

    if (match_wechat_voip_a192(data->payload[1], data->payload_len[1])) {
            if (match_wechat_voip_a175(data->payload[0],
                            data->payload_len[0]))
                    return true;
    }

return false;

}

static lpi_module_t lpi_wechat_udp = {
LPI_PROTO_UDP_WECHAT,
LPI_CATEGORY_CHAT,
"WeChat_UDP",
20,
match_wechat_udp
};

void register_wechat_udp(LPIModuleMap mod_map) {
register_protocol(&lpi_wechat_udp, mod_map);
}
/

*

  • Copyright (c) 2011-2016 The University of Waikato, Hamilton, New Zealand.
  • All rights reserved.
  • This file is part of libprotoident.
  • This code has been developed by the University of Waikato WAND
  • research group. For further information please see http://www.wand.net.nz/
  • libprotoident is free software; you can redistribute it and/or modify
  • it under the terms of the GNU Lesser General Public License as published by
  • the Free Software Foundation; either version 3 of the License, or
  • (at your option) any later version.
  • libprotoident is distributed in the hope that it will be useful,
  • but WITHOUT ANY WARRANTY; without even the implied warranty of
  • MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  • GNU Lesser General Public License for more details.
  • You should have received a copy of the GNU Lesser General Public License
  • along with this program. If not, see http://www.gnu.org/licenses/.

*/

#include <string.h>

#include "libprotoident.h"
#include "proto_manager.h"
#include "proto_common.h"

/* Thanks to http://www.cse.cuhk.edu.hk/~pclee/www/pubs/iwqos15chatdissect.pdf

  • for helping confirm this rule */

static inline bool match_wechat_uplink_hb(uint32_t payload, uint32_t len) {

    /* Byte 3 appears to be a length indicator */
    if (MATCH(payload, 0xd1, 0x0a, 0x2e, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x2d, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x2c, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x1e, 0x0a))
            return true;
    if (MATCH(payload, 0xd1, 0x0a, 0x1d, 0x0a))
            return true;

    return false;

}

static inline bool match_wechat_downlink_hb(uint32_t payload, uint32_t len) {

    /* Byte 3 appears to be a length indicator */
    if (MATCHSTR(payload, "\xd1\x0a\x2b\x0a"))
            return true;
    if (MATCHSTR(payload, "\xd1\x0a\x2a\x0a"))
            return true;
    if (MATCHSTR(payload, "\xd1\x0a\x2d\x0a"))
            return true;
    if (MATCHSTR(payload, "\xd1\x0a\x29\x0a"))
            return true;

    return false;

}

static inline bool match_wechat_voip_a175(uint32_t payload, uint32_t len) {

    if (MATCH(payload, 0xa1, 0x08, ANY, ANY) && len == 75)
            return true;

    return false;

}

static inline bool match_wechat_voip_a192(uint32_t payload, uint32_t len) {

    if (MATCH(payload, 0xa1, 0x08, ANY, ANY) && len == 92)
            return true;

    return false;

}

static inline bool match_wechat_voip_a396(uint32_t payload, uint32_t len) {
if (len == 0)
return true;
if (len == 96 && MATCH(payload, 0xa3, ANY, ANY, ANY))
return true;
return false;
}

static inline bool match_wechat_voip_d6200(uint32_t payload, uint32_t len) {
if (len == 200 && MATCH(payload, 0xd6, ANY, ANY, ANY))
return true;
return false;
}

static inline bool match_wechat_voip_d591(uint32_t payload, uint32_t len) {

    if (len < 89 || len > 91)
            return false;

if (MATCH(payload, 0xd5, ANY, ANY, ANY))
	return true;
return false;

}

static inline bool match_wechat_voip_d5104(uint32_t payload, uint32_t len) {

    if (len < 103 || len > 104)
            return false;
if (MATCH(payload, 0xd5, ANY, ANY, ANY))
	return true;
return false;

}

static inline bool match_wechat_udp(lpi_data_t *data, lpi_module_t *mod UNUSED) {

    if (match_wechat_uplink_hb(data->payload[0], data->payload_len[0])) {
            if (match_wechat_downlink_hb(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

    if (match_wechat_uplink_hb(data->payload[1], data->payload_len[1])) {
            if (match_wechat_downlink_hb(data->payload[0],
                            data->payload_len[0]))
                    return true;
    }

/* Lots of different patterns seen when using WeChat to make a voice
 * or video call.
 */
    
    if (match_wechat_voip_a396(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_a396(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

    if (match_wechat_voip_d6200(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_d6200(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }
    
if (match_wechat_voip_d591(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_d5104(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

if (match_wechat_voip_d591(data->payload[1], data->payload_len[1])) {
            if (match_wechat_voip_d5104(data->payload[0],
                            data->payload_len[0]))
                    return true;
    }

    if (match_wechat_voip_a192(data->payload[0], data->payload_len[0])) {
            if (match_wechat_voip_a175(data->payload[1],
                            data->payload_len[1]))
                    return true;
    }

    if (match_wechat_voip_a192(data->payload[1], data->payload_len[1])) {
            if (match_wechat_voip_a175(data->payload[0],
                            data->payload_len[0]))
                    return true;
    }

return false;

}

static lpi_module_t lpi_wechat_udp = {
LPI_PROTO_UDP_WECHAT,
LPI_CATEGORY_CHAT,
"WeChat_UDP",
20,
match_wechat_udp
};

void register_wechat_udp(LPIModuleMap mod_map) {
register_protocol(&lpi_wechat_udp, mod_map);
}
wechat tcp/

*

  • Copyright (c) 2011-2016 The University of Waikato, Hamilton, New Zealand.
  • All rights reserved.
  • This file is part of libprotoident.
  • This code has been developed by the University of Waikato WAND
  • research group. For further information please see http://www.wand.net.nz/
  • libprotoident is free software; you can redistribute it and/or modify
  • it under the terms of the GNU Lesser General Public License as published by
  • the Free Software Foundation; either version 3 of the License, or
  • (at your option) any later version.
  • libprotoident is distributed in the hope that it will be useful,
  • but WITHOUT ANY WARRANTY; without even the implied warranty of
  • MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  • GNU Lesser General Public License for more details.
  • You should have received a copy of the GNU Lesser General Public License
  • along with this program. If not, see http://www.gnu.org/licenses/.

*/

#include <string.h>

#include "libprotoident.h"
#include "proto_manager.h"
#include "proto_common.h"

static inline bool match_wc_pair(uint32_t payloada, uint32_t lena,
uint32_t payloadb, uint32_t lenb) {

    if (lena == 16 && MATCH(payloada, 0x00, 0x00, 0x00, 0x10)) {
            if (lenb == 16 && MATCH(payloadb, 0x00, 0x00, 0x00, 0x10))
                    return true;
            if (lenb == 18 && MATCH(payloadb, 0x00, 0x00, 0x00, 0x12))
                    return true;
    }

    if (lena == 21 && MATCH(payloada, 0x00, 0x00, 0x00, 0x15)) {
            if (lenb == 25 && MATCH(payloadb, 0x00, 0x00, 0x00, 0x19))
                    return true;
    }

    return false;

}

static inline bool match_wc_ab_request(uint32_t payload, uint32_t len) {
/* This is 0xab, followed by 4 bytes of length for the first
* packet.
*/

    if (len <= 255 && MATCH(payload, 0xab, 0x00, 0x00, 0x00))
            return true;

    if (MATCH(payload, 0xab, 0x00, 0x00, 0x01))
            return true;
    return false;

}

static inline bool match_wc_ab_big02(uint32_t payload, uint32_t len) {
/* again 0xab followed by length, except this time the length is
* for the entire flow.
*/
if (len <= 255)
return false;

    /* Flows are unlikely to need a full 4 bytes for length so I'm
     * going to stick 0x00 in the top byte for now */
    if (MATCH(payload, 0xab, 0x00, ANY, ANY)) {
            return true;
    }
    return false;

}

static inline bool match_wc_ab_big01(uint32_t payload, uint32_t len) {

    if (len < 100)
            return false;
    if (len <= 255 && MATCH(payload, 0xab, 0x00, 0x00, 0x00))
            return true;
    if (len > 255 && MATCH(payload, 0xab, 0x00, 0x00, 0x01))
            return true;
    return false;

}

static inline bool match_wc_ab_reply(uint32_t payload, uint32_t len) {
/* All replies appear to be 41 bytes */

    if (len != 41)
            return false;

    if (MATCH(payload, 0xab, 0x00, 0x00, 0x00))
            return true;
    return false;

}

static inline bool match_wechat(lpi_data_t *data, lpi_module_t *mod UNUSED) {
bool valid_port = false;

/* WeChat begins with a very simple 4 byte length field.
 * This is not unique to WeChat though, so we need to be careful.
 */

/* Only observed on port 80, 443 or 8080. Because the payload 
 * signature is not entirely unique to WeChat, let's restrict matches
 * to flows using those ports unless it shows up on other ports.
 */
if (data->server_port == 80 || data->client_port == 80)
	valid_port = true;
if (data->server_port == 8080 || data->client_port == 8080)
	valid_port = true;
if (data->server_port == 443 || data->client_port == 443)
	valid_port = true;

if (!valid_port)
	return false;

if (match_wc_pair(data->payload[0], data->payload_len[0],
                    data->payload[1], data->payload_len[1])) {
	return true;
}

if (match_wc_pair(data->payload[1], data->payload_len[1],
                    data->payload[0], data->payload_len[0])) {
	return true;
}

    if (match_wc_ab_request(data->payload[0], data->payload_len[0])) {
            if (match_wc_ab_reply(data->payload[1], data->payload_len[1]))
                    return true;
    }

    if (match_wc_ab_request(data->payload[1], data->payload_len[1])) {
            if (match_wc_ab_reply(data->payload[0], data->payload_len[0]))
                    return true;
    }

    if (match_wc_ab_big01(data->payload[0], data->payload_len[0])) {
            if (match_wc_ab_big02(data->payload[1], data->payload_len[1]))
                    return true;
    }

    if (match_wc_ab_big01(data->payload[1], data->payload_len[1])) {
            if (match_wc_ab_big02(data->payload[0], data->payload_len[0]))
                    return true;
    }

return false;

}

static lpi_module_t lpi_wechat = {
LPI_PROTO_WECHAT,
LPI_CATEGORY_CHAT,
"WeChat",
10,
match_wechat
};

void register_wechat(LPIModuleMap *mod_map) {
register_protocol(&lpi_wechat, mod_map);
}
`

compile fails against ndpi-git

compiling on arch linux with ndpi git I found it didn't want to compile
attached is a patch that got it to compile and load
do still see the below in dmesg tho

[NDPI] ndpi_init_protocol_defaults(missing protoId=191) INTERNAL ERROR: not all protocols have been initialized
[NDPI] ndpi_init_protocol_defaults(missing protoId=192) INTERNAL ERROR: not all protocols have been initialized
[NDPI] ndpi_init_protocol_defaults(missing protoId=215) INTERNAL ERROR: not all protocols have been initialized
[NDPI] ndpi_init_protocol_defaults(missing protoId=216) INTERNAL ERROR: not all protocols have been initialized
[NDPI] ndpi_init_protocol_defaults(missing protoId=217) INTERNAL ERROR: not all protocols have been initialized

ndpi-netfilter-ndpi-git.txt

ssh matching not working

I tried the latest release, compiled and installed successfully.

I am able to set the following rule (want to filter all the forwarding SSH packets)
iptables -A FORWARD -m ndpi --ssh -j DROP

However, this does not work.

I tried with following method, it works!
iptables -A FORWARD -p tcp --dport 22 -j DROP

How to debug this and make sure the protocols defined under /src/lib/protocols/ works?

cat /proc/version
Linux version 3.19.0-25-generic (buildd@lgw01-20) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015

Incorrect [Host] Filed in ndpireader output

Hi,

I see host names in [Host] filed of ndpireader output which are totally unrelated to destination or protocol classification. Please see some examples below. These are frequently seen in SSL and Bittorrent traffic.

2016-02-29 12:36:53 INFO | Src: UDP 192.168.200.17:54451 | Dest: 213.199.179.147:40014 | [proto: 125/Skype] | [1 pkts/192 bytes] | [Host: mail.google.com] | [SSL client: ] | [SSL server: ]

2016-02-29 12:36:59 INFO | Src: TCP 192.168.200.37:39516 | Dest: 222.165.168.219:443 | [proto: 91.119/SSL.Facebook] | [proto: 119/Facebook] | [4 pkts/763 bytes] | [Host: statsfe2.update.microsoft.com] | [SSL client: fbcdn-profile-a.akamaihd.net] | [SSL server: ]

2016-02-29 12:37:04 INFO | Src: TCP 192.168.200.11:51149 | Dest: 17.167.140.123:993 | [proto: 140/Apple] | [11 pkts/1315 bytes] | [Host: statsfe2.update.microsoft.com] | [SSL client: ] | [SSL server: ]

$ sudo ./ndpiReader -r
ndpiReader - nDPI (1.7.1--0-)

Any idea on this?

Detection of SSL traffic

Hi everyone

I have tried to detect SSL traffic but ndpi detect nothing when I go on https website. Yet nDPI in userland detect it. I have tried to debug the SSL files (whith some pr_debug) but I have nothing in my kern.log whereas I have the debug that i added in /src/main.c
I have tired to understand how ndpi work to launch the function in each protocol dissector but I don't find.
But, the specific SSL like Skype, Microsoft, Youtube, etc ... work.
Do you have any ideas to debug the SSL detection (end so to have the host name server) ?

Moreover, do we need to compil and install nDPI ? Because we have to re-compile it for kernel module.

Scheduling while atomic

Under Scientific Linux 6.8 I'm getting "scheduling while atomic" errors and occasional kernel panics. At the moment, ndpi_malloc() calls kmalloc(size, GFP_KERNEL) - presumably changing GFP_KERNEL to GFP_ATOMIC would fix this problem, but I'm not sure if that's a good idea since ndpi_malloc() is probably called from non-atomic parts of code too?

:Pid: 0, comm: swapper Not tainted 2.6.32-642.3.1.el6.x86_64 #1
:Call Trace:
: [] ? __schedule_bug+0x44/0x50
:[] ? schedule+0xa4c/0xb70
:[] ? pvclock_clocksource_read+0x58/0xd0
:[] ? sched_clock_local+0x25/0x90
:[] ? ndpi_malloc+0x2a/0x30 [xt_ndpi]
:[] ? __cond_resched+0x2a/0x40
:[] ? _cond_resched+0x30/0x40
:[] ? __kmalloc+0x138/0x230
:[] ? ndpi_malloc+0x2a/0x30 [xt_ndpi]
:[] ? check_content_type_and_change_protocol+0x5ea/0x920 [xt_ndpi]
:[] ? _read_unlock_bh+0x15/0x20
:[] ? xfrm_policy_lookup_bytype+0x131/0x240
:[] ? mod_timer_pending+0x126/0x200
:[] ? mod_timer+0x144/0x220
:[] ? ndpi_parse_packet_line_info+0x5fe/0x7e0 [xt_ndpi]
:[] ? ndpi_search_http_tcp+0xbf/0x3f0 [xt_ndpi]
:[] ? check_ndpi_tcp_flow_func+0x22d/0x740 [xt_ndpi]
:[] ? dev_hard_start_xmit+0x21c/0x490
:[] ? sch_direct_xmit+0x78/0x1c0
:[] ? pvclock_clocksource_read+0x58/0xd0
:[] ? check_ndpi_flow_func+0x18/0x40 [xt_ndpi]
:[] ? ndpi_detection_process_packet+0x2fa/0x4c0 [xt_ndpi]
:[] ? kmem_cache_alloc+0x18a/0x190
:[] ? ndpi_mt+0x263/0x880 [xt_ndpi]
:[] ? ipt_do_table+0x34a/0x678 [ip_tables]
:[] ? ipt_hook+0x23/0x30 [iptable_filter]
:[] ? nf_iterate+0x69/0xb0
:[] ? ip_forward_finish+0x0/0x60
:[] ? nf_hook_slow+0x76/0x120
:[] ? ip_forward_finish+0x0/0x60
:[] ? sch_direct_xmit+0x78/0x1c0
:[] ? ip_forward+0x2b3/0x460
:[] ? ip_rcv_finish+0x12d/0x440
:[] ? ip_rcv+0x275/0x350
:[] ? __netif_receive_skb+0x201/0x590
:[] ? netif_receive_skb+0x58/0x60
:[] ? cp_rx_poll+0x3b8/0x4f0 [8139cp]
:[] ? swiotlb_map_page+0x0/0x100
:[] ? net_rx_action+0x103/0x300
:[] ? __do_softirq+0xe5/0x230
:[] ? call_softirq+0x1c/0x30
:[] ? do_softirq+0x65/0xa0
:[] ? irq_exit+0x85/0x90
:[] ? do_IRQ+0x75/0xf0
:[] ? ret_from_intr+0x0/0x11
: [] ? native_safe_halt+0xb/0x10
:[] ? default_idle+0x4d/0xb0
:[] ? cpu_idle+0xb6/0x110
:[] ? start_secondary+0x2c0/0x316

libxt_ndpi.so

It's possible to copy the library to another router (same debian version and same hardware) ?
When I do this, I cant load xt_ndpi.so.

Thank

Adrien

Add protocols to netfilter

Not all protocols are listed on netfilter when you run
'''iptables -m ndpi --help'''

I've tracked down the problem and it seems that on .h there are some mapped, not all.

nDPI 1.7

Hi

I have compile nDPI with your last release, but when I want to use it, I get this error message :

root@iwilive:~# iptables -I OUTPUT -m ndpi --facebook -j DROP
iptables: Invalid argument. Run `dmesg' for more information.

dmesg : x_tables: ip_tables: ndpi.0 match: invalid size 28 (kernel) != (user) 32

Have you a solution ?

The state of BitTorrent detection?

Hello! Please forgive me for using your issue tracker in this way. I'm trying to figure out how feasible it is at this point in time to identify all BitTorrent traffic on a network, both encrypted or unencrypted, using F/OSS.

I'm aware of and have read both Velan et al. (2014) and Carvalho et al. (2009) and various online fora, etc., mostly focusing on Snort and nDPI. In your GitHub issue tracker, there are half a dozen threads on the topic, (the youngest of which I was able to find) not containing any helpful information, except perhaps for a post by Vitaly Lavrov (vel21ripn) mentioning a "branch [that] has [a?] parser [for?] DHT messages" which he says can do encrypted BT detection, but no further information is given and I'm not sure whether he's a contributor to nDPI. Neither you nor the Snort people seem to be using supervised machine learning methods at the moment.

What, in your opinion, is the state of reasonably reliable (encrypted esp.) BitTorrent detection, and what does the foreseeable future look like?

kernel panic and build issue w/ current

I've built ndpi and ndpi-netfilter both from trunk on ubuntu 14.04 x64, kernel 3.13.0-55-generic, ndpi and ndpi-netfilter checked out from trunk 2015/6/16

I'm unable to load the module,
modprobe: ERROR: could not insert 'xt_ndpi': Unknown symbol in module, or unknown parameter (see dmesg)
@syadnom

I do see a couple warnings in the build:
WARNING: "ndpi_search_kakaotalk_voice" [/usr/src/ndpi-netfilter/trunk/src/xt_ndpi.ko] undefined!
WARNING: "ndpi_search_eaq" [/usr/src/ndpi-netfilter/trunk/src/xt_ndpi.ko] undefined!
LD [M] /usr/src/ndpi-netfilter/trunk/src/xt_ndpi.ko
make[2]: Leaving directory /usr/src/linux-headers-3.13.0-55-generic' rm -r ndpi_cpy make[1]: Leaving directory/usr/src/ndpi-netfilter/trunk/src'

here is the dmesg output:
[ 381.317982] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 381.319381] xt_ndpi: module verification failed: signature and/or required key missing - tainting kernel
[ 1000.082087] xt_ndpi: Unknown symbol ndpi_search_eaq (err 0)
[ 1000.082122] xt_ndpi: Unknown symbol ndpi_search_kakaotalk_voice (err 0)

I removed kakaotalk and eaq from ndpi which allowed my to build and load xt_ndpi, but now I get a kernel panic whenever iptables loads the ndpi module.

dmesg shows this when loading the module

[ 2689.841269] xt_ndpi 2.0 (nDPI wrapper module).
[ 2689.849190] [NDPI] ndpi_string_to_automa(protoId=193): INTERNAL ERROR
[ 2689.849195] [NDPI] ndpi_string_to_automa(protoId=195): INTERNAL ERROR
[ 2689.849260] [NDPI] ndpi_string_to_automa(protoId=195): INTERNAL ERROR
[ 2689.849496] [NDPI] ndpi_init_protocol_defaults(missing protoId=194) INTERNAL ERROR: not all protocols have been initialized

now:

iptables -m ndpi --help segfaults

ndpi match options:
--ftp Match for FTP_CONTROL protocol packets.
--pop Match for MAIL_POP protocol packets.
<-- truncated list -->
--whatsapp_voice Match for WHATSAPP_VOICE protocol packets.
--dpi_check Match for CHECK protocol packets.
Segmentation fault (core dumped)

what revision of ndpi is ndpi-netfilter built against? I'd like to try again with whatever revision that is.

Thanks.

Kernel panic on high traffic

Hi guys.
I was using this solution with iptables, to classify traffic and assign it so specific qdiscs (htb or hfsc)
It worked ok for test traffic, but as soon as i tried to apply a single rule to the whole network (about 1gbps traffic, trying to shape quic protocol) the system lasted about 5-10 minutes, and then kernel panic.

This was done in a Fedora 20 64bits server, kernel 3.16.6-200, using everthing cloned from this git.

I wont be able to reproduce it soon as this is a production server (and the crash doesnt appear in other low-traffic servers). But i can test some times in the night.

image
image

block https website

Hi

How can i block website with https ? I have no problems without https ?

Thank

Adrien

accepting only one service is not working

Hi,
I have install ndpi-netfilter on ubuntu 14.04LTS and it's working fine for blocking any listed services.
But i need to Accept one or two services and Drop all the other traffic.
I'm going to use this machine as a gateway and control the traffic to my local LAN. It has two interface cards as eth0 outside network and eth1 inside network. i used ip_forward and NAT to forward all the traffic through the machine.
Here is my iptables setup.

echo 1 > /proc/sys/net/ipv4/ip_forward

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -P FORWARD DROP #default policy drop
sudo iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A FORWARD -m conntrack -j ACCEPT --ctstate ESTABLISHED,RELATED
sudo iptables -A FORWARD -p udp --dport 53 -j ACCEPT
sudo iptables -A FORWARD -m ndpi --youtube -j ACCEPT
sudo iptables -A FORWARD -m ndpi --facebook -j ACCEPT

this setup should have allow only facebook and youtube blocking all the other services. But this block all the traffic including youtube and facebook. i need to allow only facebook and youtube. But i could block youtube and facebook allowing all the traffic. it works fine, but i need the other way around.

Regards,
Shamin.

Error compile ndpi-netfilter

Hi,
I've installed ntopng/ndpi on a Debian Jessie and it works fine.
I need to associate with Netfilter. I tried to compile ndpi-netfilter with your instructions.
I have these errors. Do you have an idea ? Can you help me please ?
Thanks a lot

root@firewall:/usr/src/ndpi-netfilter# NDPI_PATH=/usr/src/ndpi-netfilter/nDPI make
make -C ipt
make[1]: Entering directory '/usr/src/ndpi-netfilter/ipt'
if test -d ndpi_cpy; then
cp /usr/src/ndpi-netfilter/nDPI/src/* ndpi_cpy -R;
else
mkdir ndpi_cpy;
cp /usr/src/ndpi-netfilter/nDPI/src/* ndpi_cpy -R;
fi
make libxt_ndpi.so
make[2]: Entering directory '/usr/src/ndpi-netfilter/ipt'
cc -fPIC -Indpi_cpy/include -Indpi_cpy/lib -I../src -DOPENDPI_NETFILTER_MODULE -O2 -Wall -D_INIT=libxt_ndpi_init -c -o libxt_ndpi.o libxt_ndpi.c;
libxt_ndpi.c:33:34: error: 'NDPI_PROTOCOL_LONG_STRING' undeclared here (not in a function)
static char prot_long_str[] = { NDPI_PROTOCOL_LONG_STRING };
^
libxt_ndpi.c:34:35: error: 'NDPI_PROTOCOL_SHORT_STRING' undeclared here (not in a function)
static char prot_short_str[] = { NDPI_PROTOCOL_SHORT_STRING };
^
libxt_ndpi.c:34:1: error: initializer element is not constant
static char *prot_short_str[] = { NDPI_PROTOCOL_SHORT_STRING };
^
libxt_ndpi.c:34:1: error: (near initialization for 'prot_short_str[0]')
libxt_ndpi.c: In function 'ndpi_mt_init':
libxt_ndpi.c:116:25: warning: unused variable 'info' [-Wunused-variable]
struct xt_ndpi_mtinfo *info = (void *)match->data;
^
Makefile:19: recipe for target 'libxt_ndpi.o' failed
make[2]: *
[libxt_ndpi.o] Error 1
make[2]: Leaving directory '/usr/src/ndpi-netfilter/ipt'
Makefile:8: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/usr/src/ndpi-netfilter/ipt'
Makefile:2: recipe for target 'all' failed
make: *** [all] Error 2

kernel Module install SSL error

Hi,

I'm getting below error while module install
My linux version is : Linux infini 4.4.0-38-generic #57-Ubuntu SMP Tue Sep 6 15:42:33 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Error details:

/opt/ndpi-netfilter/ndpi-netfilter# make modules_install
make -C src modules_install
make[1]: Entering directory '/opt/ndpi-netfilter/ndpi-netfilter/src'
make -C /lib/modules/4.4.0-38-generic/build M=$PWD modules_install;
make[2]: Entering directory '/usr/src/linux-headers-4.4.0-38-generic'
INSTALL /opt/ndpi-netfilter/ndpi-netfilter/src/xt_ndpi.ko
At main.c:222:

  • SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
  • SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
    sign-file: certs/signing_key.pem: No such file or directory
    DEPMOD 4.4.0-38-generic
    make[2]: Leaving directory '/usr/src/linux-headers-4.4.0-38-generic'
    depmod -a;

The state of BitTorrent detection?

Hello! Please forgive me for using your issue tracker in this way. I'm trying to figure out how feasible it is at this point in time to identify all BitTorrent traffic on a network, both encrypted or unencrypted, using F/OSS. I'm aware of and have read both Velan et al. (2014) and Carvalho et al. (2009) and various online fora, etc., mostly focusing on Snort and nDPI. In your GitHub issue tracker, there are half a dozen threads on the topic, (the youngest of which I was able to find) not containing any helpful information, except perhaps for a post by Vitaly Lavrov (vel21ripn) mentioning a "branch [that] has parser DHT messages" which he says can do encrypted BT detection, but no further information is given and I'm not sure whether he's a contributor to nDPI. What, in your opinion, is the state of reasonably reliable (encrypted esp.) BitTorrent detection, and what does the foreseeable future look like?

Matching not correct.

I think I found that protocols don't match the protocol name with the actual protocol being described.
I guess you fixed ewildgoose version by adding to xt_ndpi.h
NDPI_PROTOCOL_LONG_STRING
NDPI_PROTOCOL_SHORT_STRING

which does the matching of the protocol names in iptables -m ndpi --[protocol name]
This was usualy done in older versions of the nDPI library. But those constants are no longer part of the nDPI library.
I will try with an older nDPI to get it to work. Otherwise it should be fixed to work with newer versions of the nDPI library since they don't match protocols with NDPI_PROTOCOL_LONG_STRING
NDPI_PROTOCOL_SHORT_STRING any more.

Kernel panic on 3.14.19

Hi,

I am seeing kernel panics while traffic is flowing through the system. I am using nDPI on a linux router with ip_forward enabled. Matches in the FORWARD chain on iptables.

Kernel: 3.14.19
IPtables: 1.4.21

Using nDPI source in git to compile
Compiled via yocto

On the kernel panic is see the nDPI function getSSLcertificate.

-A PREROUTING -s 172.16.0.0/16 -j USShaping
-A POSTROUTING -d 172.16.0.0/16 -j DSShaping
-A DSShaping -m ndpi--dpi_check
-A DSShaping -m ndpi--dhcp -j MARK --set-xmark 0xa/0xffffffff
-A DSShaping -m ndpi--dns -j MARK --set-xmark 0xa/0xffffffff
-A DSShaping -m ndpi--mdns -j MARK --set-xmark 0xa/0xffffffff
-A DSShaping -m ndpi--ntp -j MARK --set-xmark 0xa/0xffffffff
-A DSShaping -m ndpi--h323 -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--irc -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--rtp -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--sip -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--whatsapp -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--whatsapp_voice -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--skype -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--megaco -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--teamspeak -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--vnc -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--rdp -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--teamviewer -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--pcanywhere -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--ssh -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--xbox -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--battlefield -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--quake -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--steam -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--halflife2 -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--worldofwarcraft -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--warcraft3 -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--armagetron -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--crossfire -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--guildwars -j MARK --set-xmark 0x14/0xffffffff
-A DSShaping -m ndpi--rtsp -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--ftp -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--nfs -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--facebook -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--twitter -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--gmail -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--google_maps -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--youtube -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--google -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--netflix -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--apple_itunes -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--amazon -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--ebay -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--spotify -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--avi -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--flash -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--ogg -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--mpeg -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--mpeg -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--realmedia -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--windowsmedia -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--http -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--ssl -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--ipsec -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--openvpn -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--tor -j MARK --set-xmark 0x1e/0xffffffff
-A DSShaping -m ndpi--windows_update -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--imap -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--imaps -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--pop -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--pops -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--smtp -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--smtps -j MARK --set-xmark 0x28/0xffffffff
-A DSShaping -m ndpi--dropbox -j MARK --set-xmark 0x28/0xffffffff

-A USShaping -m ndpi--dhcp -j MARK --set-xmark 0xa/0xffffffff
-A USShaping -m ndpi--dns -j MARK --set-xmark 0xa/0xffffffff
-A USShaping -m ndpi--mdns -j MARK --set-xmark 0xa/0xffffffff
-A USShaping -m ndpi--ntp -j MARK --set-xmark 0xa/0xffffffff
-A USShaping -m ndpi--h323 -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--irc -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--rtp -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--sip -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--whatsapp -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--whatsapp_voice -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--skype -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--megaco -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--teamspeak -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--vnc -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--rdp -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--teamviewer -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--pcanywhere -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--ssh -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--xbox -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--battlefield -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--quake -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--steam -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--halflife2 -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--worldofwarcraft -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--warcraft3 -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--armagetron -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--crossfire -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--guildwars -j MARK --set-xmark 0x14/0xffffffff
-A USShaping -m ndpi--rtsp -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--ftp -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--nfs -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--facebook -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--twitter -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--gmail -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--google_maps -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--youtube -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--google -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--netflix -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--apple_itunes -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--amazon -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--ebay -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--spotify -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--avi -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--flash -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--ogg -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--mpeg -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--mpeg -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--realmedia -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--windowsmedia -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--http -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--ssl -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--ipsec -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--openvpn -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--tor -j MARK --set-xmark 0x1e/0xffffffff
-A USShaping -m ndpi--windows_update -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--imap -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--imaps -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--pop -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--pops -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--smtp -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--smtps -j MARK --set-xmark 0x28/0xffffffff
-A USShaping -m ndpi--dropbox -j MARK --set-xmark 0x28/0xffffffff

modules not loading on nDPI >r8323

I've pulled in nDPI release 8323 which does build and load, but it does nothing, no inspection. I can get 5761 built with the mainline nDPI sources and they work.

(debian 7)

Unable to compile on Debian 8.2 3.16.0-4-amd64

root@bobcat:~/ndpi-netfilter# NDPI_PATH=/root/ndpi-netfilter/nDPI make
make -C ipt
make[1]: Entering directory '/root/ndpi-netfilter/ipt'
if test -d ndpi_cpy; then \
        cp /root/ndpi-netfilter/nDPI/src/* ndpi_cpy -R; \
else \
        mkdir ndpi_cpy; \
        cp /root/ndpi-netfilter/nDPI/src/* ndpi_cpy -R; \
fi
make libxt_ndpi.so
make[2]: Entering directory '/root/ndpi-netfilter/ipt'
cc -fPIC -Indpi_cpy/include -Indpi_cpy/lib -I../src -DOPENDPI_NETFILTER_MODULE -O2 -Wall -DNDPI_IPTABLES_EXT -D_INIT=libxt_ndpi_init -c -o libxt_ndpi.o libxt_ndpi.c;
libxt_ndpi.c: In function ‘ndpi_mt_init’:
libxt_ndpi.c:112:25: warning: unused variable ‘info’ [-Wunused-variable]
  struct xt_ndpi_mtinfo *info = (void *)match->data;
                         ^
cc -shared -o libxt_ndpi.so libxt_ndpi.o;
rm libxt_ndpi.o
make[2]: Leaving directory '/root/ndpi-netfilter/ipt'
rm -r ndpi_cpy
make[1]: Leaving directory '/root/ndpi-netfilter/ipt'
make -C src
make[1]: Entering directory '/root/ndpi-netfilter/src'
if test -d ndpi_cpy; then \
        cp /root/ndpi-netfilter/nDPI/src/* ndpi_cpy -R; \
else \
        mkdir ndpi_cpy; \
        cp /root/ndpi-netfilter/nDPI/src/* ndpi_cpy -R; \
fi
cp ndpi_cpy/../../nDPI-patch/src/* ndpi_cpy/ -R;
cp ndpi_cpy/lib/third_party/src/*.c ndpi_cpy/lib -R;
cp ndpi_cpy/lib/third_party/include/*.h ndpi_cpy/lib -R;
cp ndpi_cpy/lib/third_party/include/*.h ndpi_cpy/include -R;
sed -i "s/^\s*void ndpi_free_flow/\/\/void ndpi_free_flow/" ndpi_cpy/include/ndpi_api.h;
make -C /lib/modules/3.16.0-4-amd64/build M=$PWD;
make[2]: Entering directory '/lib/modules/3.16.0-4-amd64/build'
make[2]: *** No targets specified and no makefile found.  Stop.
make[2]: Leaving directory '/lib/modules/3.16.0-4-amd64/build'
Makefile:156: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/root/ndpi-netfilter/src'
Makefile:5: recipe for target 'all' failed
make: *** [all] Error 2

Followed exactly as the instruction in ndpi.install.
Any suggestion?

Add a new filter

Hi

How I can add from the compilation a new nDPI filter ?
What is the difference between 'NDPI_PROTOCOL_LONG_STRING' and 'NDPI_PROTOCOL_SHORT_STRING' ?

Thank

Adrien

[NDPI] Internal error: protocol SSH/92 has been already registered

Hi,

I encountered an issue when I port ndpi-netfilter to ubuntu cloud image,
uname -a
Linux 3f6adc8e-6fd3-432a-ba83-19065e4eb00f 3.13.0-67-generic #110-Ubuntu SMP Fri Oct 23 13:24:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

The issue is,
for whatever protocol I add , for example, iptables -A FORWARD -m ndpi --ssh -j DROP
the kernel (dmesg) always complains "[NDPI] Internal error: protocol SSH/92 has been already registered"
I tried couple protocols, all with the same error.

Anyone encounter this issue before? Thanks.

Anything with the kernel module?
lsmod | grep table

iptable_nat 13011 1
nf_nat_ipv4 13263 1 iptable_nat
nf_nat 21841 4 ipt_MASQUERADE,nf_nat_ipv4,xt_nat,iptable_nat
nf_conntrack 97202 7 xt_ndpi,ipt_MASQUERADE,nf_nat,nf_nat_ipv4,xt_conntrack,iptable_nat,nf_conntrack_ipv4
iptable_filter 12810 1
ip_tables 27239 2 iptable_filter,iptable_nat
x_tables 34059 8 xt_ndpi,ip_tables,xt_tcpudp,ipt_MASQUERADE,xt_conntrack,xt_LOG,xt_nat,iptable_filter

compilation fails for gcc-4.4.6

ndpi-netfilter is failing compilation in kernel-4.7.

make -C /lib/modules/4.7.0-0.1gbu.tos3_0/build M=$PWD;
make[2]: Entering directory `/usr/src/kernels/4.7.0-0.1gbu.tos3_0'
  LD      /root/TOS-2482/xt_ndpi-3.0/src/built-in.o
  CC [M]  /root/TOS-2482/xt_ndpi-3.0/src/main.o
In file included from /root/TOS-2482/xt_ndpi-3.0/src/ndpi_cpy/include/ndpi_main.h:55,
                 from /root/TOS-2482/xt_ndpi-3.0/src/main.c:41:
/root/TOS-2482/xt_ndpi-3.0/src/ndpi_cpy/include/ndpi_define.h:299:2: error: #error "__BYTE_ORDER MUST BE DEFINED !"
In file included from /root/TOS-2482/xt_ndpi-3.0/src/ndpi_cpy/include/ndpi_main.h:57,
                 from /root/TOS-2482/xt_ndpi-3.0/src/main.c:41:
/root/TOS-2482/xt_ndpi-3.0/src/ndpi_cpy/include/ndpi_typedefs.h:195:3: error: #error "Byte order must be defined"
/root/TOS-2482/xt_ndpi-3.0/src/ndpi_cpy/include/ndpi_typedefs.h:254:3: error: #error "Byte order must be defined"
make[3]: *** [/root/TOS-2482/xt_ndpi-3.0/src/main.o] Error 1
make[2]: *** [_module_/root/TOS-2482/xt_ndpi-3.0/src] Error 2
make[2]: Leaving directory `/usr/src/kernels/4.7.0-0.1gbu.tos3_0'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/root/TOS-2482/xt_ndpi-3.0/src'
make: *** [all] Error 2
[root@localhost xt_ndpi-3.0]# 

bittorrent maching is not working properly

Hi,
I need to mark or classify all bittorrent traffic and apply a tc class to shape the traffic.
At least, i couldn't drop all the bittorrent packets.
Some packets are identified and Dropped, but most of them are cannot be identified and torrent downloads continuously. my rule is as follows.

suddo iptables -A FORWARD -m ndpi --bittorrent -j DROP

kernel patch doesn't apply

Would you mind telling me the exact kernel version your fork of the ndpi-netfilter was last built against as working? I can't get the patch to apply to 2.6.32 which is the stock deb6 kernel (maybe I'm patching wrong?) so though I can get ndpi-netfilter compiled and loaded, it wont match anything. I did recompile the kernel as instruct but without the patch.
Thanks

Compile on Centos 6.x?

Hello, is it possible to compile the ndpi-netfilter to make it work on Centos 6.x? I've look around but haven't found anything to make it work on Centos 6.x

increasing performance with PF_RING

Hi,
we need to handle at least 5Gbps traffic with ndpi-netfilter blocking and shaping rules. If we use PF_RING, will it increase the performance of ndpi-netfilter and reduce the server load?

Regards,
Shamin.

--dpi_check doesn't exist?

[dave@lake ~]$ iptables -t mangle -A PREROUTING -m ndpi --dpi_check
iptables v1.6.0: unknown option "--dpi_check"
Try `iptables -h' or 'iptables --help' for more information.

ndpi-netfilter kernel panics

Hi. You mention that to compile netfilter-ndpi I need iptables-dev >= version 1.4.21-1ubuntu1.

I have a Debian 7:
kernel 3.18.36
iptables 1.4.14-3.1
iptables-dev 1.4.14-3.1
conntrack 1:1.2.1-1+deb7u1 enabled (defaults from official debian repo).

When compiled I managed to filter traffic etc but I've been getting some random kernel panics (reporting out of memory), especially when I put filtering rules at INCOMING and OUTGOING chains of the filter table. (kernel panics are more frequent when filtering rules are at INPUT, OUTGOING, FORWARD chains, while panics are less frequent when filtering only at FORWARD, indicating that the problem is exacerbated when more traffic is processed ).

Have you tested with Debian 7? Going to iptables-dev >= 1.4.21 is the only option?
Thanx

Building with nDPI 1.7

Hello,

I'm trying to build ndpi-netfilter with nDPI 1.7 version and I'm having some issues with the "inet_pton" function.

Is there any work in progress to run with the nDPI 1.7?

Thanks in advance.

Debug of NDPI

Hi,

Do you know activate the macro NDPI_LOG in the kernel ?
I searched it , but I didn't find.

Thanks in advance

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.