ndpi-netfilter kernel panics about ndpi-netfilter

[betolj]

Maybe.. I thought in this possibility too, but i dont know what's the compatibility with "project API".
But, only in vel21ripn fork exist a /proc/net/xt_ndpi/ directory.

And in my fork, DNS queries for facebook and youtube will fail if you filter these protocols.

[betolj]

Hi, are you building the ndpi-netfilter with lastest version? And with the embedded ndpi library?
I have done tests with Ubuntu 14.04 and 16.04. As soon possible, i'll try with Debian 7 too.

[rightkick]

I cloned the latest repo from github and followed steps from ndpi.install.
Let me know if you need any other clarification.

[elico]

@rightkick can you test something in iptables?

iptables -t mangle -N NDPIRET  
iptables -t mangle -A NDPIRET  -m ndpi --proto all -j RETURN
iptables -t mangle -I FORWARD -j NDPIRET
iptables -t mangle -I PREROUTING -j NDPIRET
iptables -t mangle -I POSTROUTING -j NDPIRET

The above would make all traffic passing in the machine to be inspected by the NDPI module.
It might not be related but it is a load "test" on the module.

Also, I want to rebuild on the same machine, is it a 64 bit or 32 bit?

[rightkick]

Hi @elico. I was testing on x64 with below firewall (a simple statefull one that allows only HTTP, HTTPS and DNS for browsing). All packets are passed from ndpi filtering before their state is examined.

*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
*filter
:INPUT DROP [105:7220]
:FORWARD DROP [0:0]
:OUTPUT DROP [62:6776]
:filter-in-con - [0:0]
:filter-out-con - [0:0]
:filter-fw-con - [0:0]
:state-tracking - [0:0]
:ndpi-filter - [0:0]
-A INPUT -j ndpi-filter
-A INPUT -j state-tracking
-A INPUT -j filter-in-con
-A FORWARD -j ndpi-filter
-A FORWARD -j state-tracking
-A FORWARD -j filter-fw-con
-A OUTPUT -j ndpi-filter
-A OUTPUT -j state-tracking
-A OUTPUT -j filter-out-con
-A filter-in-con -p tcp -m multiport --dports 22,80 -j ACCEPT
-A filter-out-con -p tcp -m multiport --dports 80,443 -j ACCEPT
-A filter-out-con -p udp -m udp --dport 53 -j ACCEPT
-A filter-out-con -p tcp -m tcp --dport 53 -j ACCEPT
-A filter-fw-con -p tcp -m multiport --dports 80,443 -j ACCEPT
-A state-tracking -m state --state RELATED,ESTABLISHED -j ACCEPT
-A state-tracking -m state --state INVALID -j DROP
-A ndpi-filter -m ndpi --youtube -j DROP
-A ndpi-filter -m ndpi --facebook -j DROP
COMMIT
*mangle
:PREROUTING ACCEPT [4667:1352684]
:INPUT ACCEPT [4521:1336042]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [3264:521477]
:POSTROUTING ACCEPT [3264:521477]
-A PREROUTING -m ndpi --dpi_check
-A POSTROUTING -m ndpi --dpi_check
COMMIT

I enabled ndpi checks on all filter chains for the same purpose of stressing the ndpi module.
The difference is that I filter in filter table, instead of mangle.

I will test with your rules and let you know.

[elico]

@rightkick your rules are almost the same as what I proposed.
Just as a side note, you should put the INVALID state at the top of the chain before RELATED and ESTABLISHED if the CPU is not really 100mhz one.

[rightkick]

One question for the installation:
in ndpi.install you have commented out the ./configure –with-pic.
In my build, I issue simply ./configure, then proceed with make, make install.
Do I need to omit ./configure?

@elico, when defining the rules you suggested, I receive the following error:
Applying new ruleset... iptables-restore v1.4.14: xt_ndpi: You need to specify at least one protocol
Error occurred at line: 260
Try `iptables-restore -h' or 'iptables-restore --help' for more information.
failed.
E: unknown error applying new iptables ruleset.

It is the line with the rule:
-A NDPIRET -m ndpi --proto all -j RETURN

[elico]

@rightkick I'm not the developer but ./configure should be OK.
And I had couple minutes to adopt my OpenSUSE Docker build script into a debian one:
https://gist.github.com/elico/45c87420850803a4837366e728885575

which works for the wheezy backports kernel.
I have not tested it to run but just to build without any errors.

[rightkick]

Thanx @elico. I also had no issues during build on Debian7. It is only through testing that I observed issues.

I have transferred now all ndpi filtering rules at mangle table, after loopback ACCEPT. I am suspecting that putting ndpi check rule before looback ACCEPT rules might be causing loops.

Below is the revised portion of firewall where ndpi rules reside, with few rules for accounting only:

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:ndpi-filter - [0:0]
-A PREROUTING -s 127.0.0.0/16 -j ACCEPT
-A PREROUTING -j ndpi-filter
-A INPUT -s 127.0.0.0/16 -j ACCEPT
-A FORWARD -s 127.0.0.0/16 -j ACCEPT
-A FORWARD -j ndpi-filter
-A OUTPUT -d 127.0.0.0/16 -j ACCEPT
-A POSTROUTING -d 127.0.0.0/16 -j ACCEPT
-A POSTROUTING -j ndpi-filter
-A ndpi-filter -m ndpi --dpi_check
-A ndpi-filter -m ndpi --http
-A ndpi-filter -m ndpi --facebook
-A ndpi-filter -m ndpi --youtube
-A ndpi-filter -m ndpi --dns
-A ndpi-filter -m ndpi --skype
COMMIT

It is not clear to me what --dpi_check does. When printing with iptables -t mangle -nvL ndpi-filter it is showing as protocol CHECK with zero matching packets. Other traffic (e.g HTTP) is matched and counters are increasing, as below:

 0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0           protocol CHECK
19746   15M            all  --  *      *       0.0.0.0/0            0.0.0.0/0           protocol HTTP
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0           protocol FACEBOOK
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0           protocol YOUTUBE
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0           protocol DNS
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0           protocol SKYPE

any idea?

[elico]

@rightkick It's pretty simple.
The HTTP protocol will match any HTTP request including FACEBOOK and YOUTUBE.
Due to this you need to order the rules in a way that the more "wide" rule will come last.
The ndpi_check I do not know but you should try something like:

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:ndpi-filter - [0:0]
-A PREROUTING -s 127.0.0.0/16 -j ACCEPT
-A PREROUTING -j ndpi-filter
-A INPUT -s 127.0.0.0/16 -j ACCEPT
-A FORWARD -s 127.0.0.0/16 -j ACCEPT
-A FORWARD -j ndpi-filter
-A OUTPUT -d 127.0.0.0/16 -j ACCEPT
-A POSTROUTING -d 127.0.0.0/16 -j ACCEPT
-A POSTROUTING -j ndpi-filter -j RETURN
-A ndpi-filter -m ndpi --dns -j RETURN
-A ndpi-filter -m ndpi --facebook -j RETURN
-A ndpi-filter -m ndpi --youtube -j RETURN
-A ndpi-filter -m ndpi --skype -j RETURN
-A ndpi-filter -m ndpi --http -j RETURN
-A ndpi-filter -m ndpi --all -j RETURN
COMMIT

The concept is that the all will be a catch all while the others will match if possible.
Leave the ndi_check out of the picture since if the bug exist it will appear with the above rules.
I do not know the if -m ndpi --all rule matches every known protocol or also GRE and other protocols which should be blocked much before the above.

[rightkick]

Hi @elico. Ok amended the firewall accordingly, but -A ndpi-filter -m ndpi --all -j RETURN is not loading. Error given: Applying new ruleset... iptables-restore v1.4.14: unknown option "--all".
I removed that line and will be testing with this setup. For the last hours it has been stable.

[elico]

@rightkick This is a dump from my iptables but I do not know why it's not working on your side.

*mangle
:PREROUTING ACCEPT [1259123:1094220271]
:INPUT ACCEPT [121605:260288694]
:FORWARD ACCEPT [1137159:833504803]
:OUTPUT ACCEPT [173688:172770777]
:POSTROUTING ACCEPT [1307335:1006051018]
:MINIUPNPD - [0:0]
:NDPIRET - [0:0]
-A PREROUTING -j NDPIRET
-A FORWARD -j NDPIRET
-A POSTROUTING -j NDPIRET
-A NDPIRET -m ndpi  --all  -j RETURN
COMMIT

I can try to run a VM and see how it all glued together.
You can contact me on my email directly.

[betolj]

Hi Elico,
You are probably using the vel21ripn fork.
http://devel.aanet.ru/ndpi/

[elico]

@betolj The vel21ripn is not working for me...
how can I know? modinfo? dmesg?
Maybe something got slipped into my Docker builds that I don't know about..

I feel so stupid.
It seems to work with a patch I did but forgot about it and was sure I built and used the better one.
You where right!!!
OK the Docker build run in seconds.

[betolj]

You can use modinfo:
modinfo xt_ndpi | grep betolj
author: Humberto Juca [email protected]

The dpi_check option can be used to map all packets to the ndpi-flow.
And "-m ndpi --proto ..." its a vel21ripn sintaxe.

[elico]

@betolj Can it be that I have used the xtables module of one with the lib_xtndpi of another?
The simple answer is no..
But it crashes for me with @betolj version and no the @vel21ripn .

[elico]

The crash snapshot:
http://pasteboard.co/81ZUPUmS7.png

[betolj]

So, are you receiving kernel panics in this fork? After rebuild?

[elico]

Yes.
http://pasteboard.co/81ZUPUmS7.png

[betolj]

Ok. I will review the code.

In my tests it has been stable.

root@humberto-XPS-8300:~# cat /etc/issue
Ubuntu 16.04 LTS \n \l

root@humberto-XPS-8300:~# uname -r
4.4.0-28-generic

*mangle
:PREROUTING ACCEPT [58385:68275345]
:INPUT ACCEPT [58314:68272529]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [58131:4858716]
:POSTROUTING ACCEPT [58168:4859982]
-A PREROUTING -m ndpi --dpi_check
-A POSTROUTING -m ndpi --dpi_check
COMMIT
*filter
:INPUT ACCEPT [5998:2359081]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [5815:752563]
-A INPUT -m ndpi --facebook -j DROP
-A INPUT -m ndpi --gmail
-A INPUT -m ndpi --bittorrent
-A INPUT -m ndpi --ssl
-A INPUT -m ndpi --http
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [51:3243]
:POSTROUTING ACCEPT [51:3243]
COMMIT

[elico]

leap-router:~ # cat /etc/issue
Welcome to openSUSE Leap 42.1 - Kernel \r (\l).

leap-router:~ # uname -r
4.1.21-14-default

[rsweerarathna]

Hi,
Anyone knows whether ndpi-netfilter can compile with pfring and improve the performance?
Is there any possibility to do it?

[rightkick]

I've been testing for more then 24 hours, and seems stable. No kernel panics until now with the new firewall setup.

I would be interested to pfring if it can be plugged with ndpi-netfilter, although I have not seen performance issues with libpcap.

Additionally, is there a way to define and load custom protocols with ndpi-netfilter? I see an example file included at ndpi-netfilter/nDPI/example/protos.txt

[elico]

@betolj I managed to build and install the ndpi-netfilter modules on OpenSUSE leap and it seems to work as expected without any kernel crashes for the last hour.
I have also updated the Dockerfile and build scripts for OpenSUSE leap.
The issue I was having was that I didn't copied the libxt_ndpi.so and the module was "trying" to use the old one which.. it couldn't.

Thanks!!

[elico]

@betolj Sorry but it crashed again the machine.
I tried over and over but every time something is getting wrong.

[rightkick]

Testing several days with Debian 7 and kernel 3.18.36, and with the above mentioned firewall, has yielded stable results.

[sduponch]

Hi, i have kernel panics on Debian 7 with 3.18.26 mptcp patched kernel.

[11755.110596]  [<ffffffff8157bde8>] ? async_page_fault+0x28/0x30
[11755.110596]  [<ffffffffa06285d3>] ? ndpi_detection_process_packet+0x123/0x4d0 [xt_ndpi]
[11755.110596]  [<ffffffffa06285c5>] ? ndpi_detection_process_packet+0x115/0x4d0 [xt_ndpi]
[11755.110596]  [<ffffffffa061a3b2>] ? ndpi_alloc_id+0xb2/0xe0 [xt_ndpi]
[11755.110596]  [<ffffffffa061a749>] ? ndpi_mt+0x289/0x8f0 [xt_ndpi]
[11755.110596]  [<ffffffff81053c00>] ? kvm_clock_get_cycles+0x10/0x30
[11755.110596]  [<ffffffffa0546e76>] ? ipt_do_table+0x2a6/0x708 [ip_tables]
[11755.110596]  [<ffffffff814a1440>] ? ip_fragment+0xa90/0xa90
[11755.110596]  [<ffffffff81489585>] ? nf_iterate+0x65/0xa0
[11755.110596]  [<ffffffff814a1440>] ? ip_fragment+0xa90/0xa90
[11755.110596]  [<ffffffff81489636>] ? nf_hook_slow+0x76/0x130
[11755.110596]  [<ffffffff814a1440>] ? ip_fragment+0xa90/0xa90
[11755.110596]  [<ffffffff814a356a>] ? ip_output+0x8a/0xa0
[11755.110596]  [<ffffffff814a3088>] ? ip_queue_xmit+0x138/0x3b0
[11755.110596]  [<ffffffff814bb045>] ? tcp_transmit_skb+0x3f5/0x890
[11755.110596]  [<ffffffff81446242>] ? __kmalloc_reserve.isra.25+0x32/0x90
[11755.110596]  [<ffffffff814ac48b>] ? tcp_recvmsg+0x37b/0xc20
[11755.110596]  [<ffffffff814dab8a>] ? inet_recvmsg+0x6a/0x80
[11755.110596]  [<ffffffff8143e272>] ? sock_recvmsg+0x92/0xd0
[11755.110596]  [<ffffffff812025d0>] ? ep_read_events_proc+0xc0/0xc0
[11755.110596]  [<ffffffff81202eeb>] ? ep_scan_ready_list+0x1bb/0x1d0
[11755.110596]  [<ffffffff8143e3cd>] ? SYSC_recvfrom+0xcd/0x140
[11755.110596]  [<ffffffff81092ff0>] ? wake_up_state+0x20/0x20
[11755.110596]  [<ffffffff81053c0f>] ? kvm_clock_get_cycles+0x1f/0x30
[11755.110596]  [<ffffffff810cfb33>] ? ktime_get_ts64+0x43/0x100
[11755.110596]  [<ffffffff81579d8d>] ? system_call_fastpath+0x16/0x1b
[11755.110596] Code: d3 e2 4d 89 d5 e9 0f ff ff ff 90 48 89 ef e8 08 96 01 00 48 83 c4 18 31 c0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 80 00 00 00 00 <0f> 0b 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 
[11755.110596] RIP  [<ffffffff81188060>] __get_vm_area_node+0x150/0x160
[11755.110596]  RSP <ffff8801df896e20>
[11755.110596] ---[ end trace 45d3ee13796e9454 ]---
[11755.110596] Kernel panic - not syncing: Fatal exception in interrupt
[11755.110596] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
[11755.110596] drm_kms_helper: panic occurred, switching back to text console
[11755.110596] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

admin@docker-host-dev-2 ~/ndpi-netfilter $ uname -r
3.18.26-mptcp-martinseb-ip-tos

admin@docker-host-dev-2 ~/ndpi-netfilter $ sudo modinfo ./src/xt_ndpi.ko
filename: /home/admin/ndpi-netfilter/./src/xt_ndpi.ko
alias: ipt_ndpi
description: nDPI wrapper
author: Humberto Juca [email protected]
author: G. Elian Gidoni (main code)
license: GPL
depends: x_tables
vermagic: 3.18.26-mptcp SMP mod_unload modversions
parm: debug_dpi:Enable syslog debug (int)

[wirelessmundi]

I get a very similar panic ndpi_detection_process_packet with a vanilla kernel 4.4.15 on debian jessie.

What information can I provide that might help resolve it?

[gklipel]

rightclick, I have some random crashes too, running on arm64, arm, x86 and x86_64, leading me to the conclusion that it has nothing to do with architecture, but with my firewall rules... Interesting your experience, mind to share some more information? How many days the system is running fine? How many machines are running through your firewall?

With vel21ripn fork the kernel crashes on restoring iptables rules, this betolj fork randomly crashes like, in 1 hour on up system, runs 2 days straight and then crashes. Not quite getting any logic path to reproduce these errors...

betolj, congratulations for the effort!

[wirelessmundi]

Indeed, thanks a lot to betolj! I love this port that doesn't require any kernel patching!

To reproduce this issue on-demand, I have a script that loads all ndpi protocols in the filter table like so:
iptables -A INPUT -m ndpi --proto1 -j ACCEPT
iptables -A INPUT -m ndpi --proto2 -j ACCEPT
...

Then after a while (0-5 minutes), I guess because of traffic, kernel just panics, usually it mentions ndpi_detection_process_packet, at least once I saw a mention about "ftp_control".

Any idea of how could we further hunt this issue? (I'm using kernel 4.4.X btw)

[rightkick]

Hi, following more than 20 days of stress tests I deem my setup stable. My systemdetails and firewall rules have been provided to a previous post. Seems that you need to accept traffic on local interface before you do any filtering with ndpi to avoid panics.

On July 28, 2016 7:48:49 AM EEST, graucio [email protected] wrote:

rightclick, I have some random crashes too, running on arm64, arm, x86
and x86_64, leading me to the conclusion that it has nothing to do with
architecture, but with my firewall rules... Interesting your
experience, mind to share some more information? How many days the
system is running fine? How many machines are running through your
firewall?

With vel21ripn fork the kernel crashes on restoring iptables rules,
this betolj fork randomly crashes like, in 1 hour on up system, runs 2
days straight and then crashes. Not quite getting any logic path to
reproduce these errors...

betolj, congratulations for the effort!

---

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#29 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[wirelessmundi]

There is still a panic tho (see @sduponch's post), I'm trying to identify what causes it...

@rightkick, from what you say, it seems that you have traffic coming from loopback that was causing the kernel panic (that's why filtering it out resolves it for you). Maybe you could try to identify it to help with the issue! You could capture loopback traffic, and try filtering parts of it until you find out the source of the panic!

[gklipel]

@wirelessmundi for me, catching telegram's packets was causing the most crashes...
@rightkick, thanks for the feed...

[elico]

@rightkick I am not sure I understood what you wrote.
By local traffic do you mean loop back interface ie:

iptables -t mangle -I PREROUTING -i lo -j ACCEPT
iptables -t mangle -I POSTROUTING -i lo -j ACCEPT
iptables -t mangle -I INPUT -i lo -j ACCEPT
iptables -t mangle -I OUTPUT -i lo -j ACCEPT
iptables -t mangle -I FORWARD -i lo -j ACCEPT
iptables -t filter -I INPUT -i lo -j ACCEPT
iptables -t filter -I OUTPUT -i lo -j ACCEPT
iptables -t filter -I FORWARD -i lo -j ACCEPT

[rightkick]

Correct. I meant traffic on lo interface. In my case I have quite some traffic from zabbix monitoring system. I might be able to capture some and provide a breakdown shortly.

On August 4, 2016 9:27:17 AM EEST, Eliezer Croitoru [email protected] wrote:

@rightkick I am not sure I understood what you wrote.
By local traffic do you mean loop back interface ie:

iptables -t mangle -I PREROUTING -i lo -j ACCEPT
iptables -t mangle -I POSTROUTING -i lo -j ACCEPT
iptables -t mangle -I INPUT -i lo -j ACCEPT
iptables -t mangle -I OUTPUT -i lo -j ACCEPT
iptables -t mangle -I FORWARD -i lo -j ACCEPT
iptables -t filter -I INPUT -i lo -j ACCEPT
iptables -t filter -I OUTPUT -i lo -j ACCEPT
iptables -t filter -I FORWARD -i lo -j ACCEPT

---

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#29 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[rightkick]

Indeed, it seems that on lo interface I have a lot of Zabbix traffic on TCP port 10050 and some traffic on mysql port TCP 3306. I see also some ports I use for redirections from squid3. So my lo port is somewhat busy.

[gklipel]

@rightkick , I've done what you did and seems that I have no lucky as you did hehehehe, it mostly crashes on SSL traffic it seems... Using Skype leads to kernel panics, I'll look further to confirm this.

Any news, please share with us!

Thanks all!

[rightkick]

@graucio, I have tested Skype filetring with success. Can you paste the firewall rules to look into?

On August 5, 2016 7:53:54 PM EEST, graucio [email protected] wrote:

@rightkick , I've done what you did and seems that I have no lucky as
you did hehehehe, it mostly crashes on SSL traffic it seems... Using
Skype leads to kernel panics, I'll look further to confirm this.

Any news, please share with us!

Thanks all!

---

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#29 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[wirelessmundi]

Not sure if it can help, but latest version from vel21ripn does not panic, see:
https://github.com/vel21ripn/nDPI
Also apparently it does not need that ugly kernel patch anymore (using connlabel now).

[rightkick]

I've been testing https://github.com/vel21ripn/nDPI and weirdly it does panic sometimes when issuing apt-get update. Especially immediately after iptables have been loaded. With other traffic it does not panic.
Box used for testing: Debian 7 64 bit with kernel 3.16.43.

[elico]

@rightkick you need to exclude OUTPUT and INPUT traffic from the "lo" one.
..You should not check lo traffic at all using NDPI.

[rightkick]

@elico, I have ACCEPT rules in "lo" in place, as per below and I was seeing kernel panics:

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:custom-mpre-con - [0:0]
:custom-min-con - [0:0]
:custom-mfw-con - [0:0]
:custom-mout-con - [0:0]
:custom-mpost-con - [0:0]
:ndpi-filter - [0:0]
-A PREROUTING -i lo -j ACCEPT
-A PREROUTING -j custom-mpre-con
-A INPUT -i lo -j ACCEPT
-A INPUT -j custom-min-con
-A FORWARD -i lo -j ACCEPT
-A FORWARD -j custom-mfw-con
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -j custom-mout-con
-A POSTROUTING -o lo -j ACCEPT
-A POSTROUTING -j ndpi-filter
-A POSTROUTING -j custom-mpost-con

I am putting the DPI filtering on mangel PREROUTING, to avoid altogether processing of locally generated traffic. I will test with the new firewall and will revert with outcome. I was thinking also if it is better to move the DPI filtering at filter table, before state check, instead of mangle.

[elico]

@rightkick it's really an issue.
I hope it will get resolved soon.

[rightkick]

@elico, putting the DPI filter rules at mangle PREROUTING I don't receive kernel panics.
I will be testing this setup for some days and will get back. Wost case will be to have the rules at mangle forward and filter only forwarded traffic. I am playing now with mangle PREROUTING since I am interested to filter also redirected traffic (redirected to squid at TCP port 80).

[elico]

@rightkick if something goes into squid don't touch it with iptables and let squid what it does best.

A side note:
I am the maintainer of the unofficial pacakges of squid-cache http://wiki.squid-cache.org/KnowledgeBase/CentOS
If you are trying squid give a try also to RedWood which I am also packaging in the same repository.
https://github.com/andybalholm/redwood

if you need filtering and not caching then RedWood is much more efficient then Squid-Cache in the sense of memory and CPU.

[rightkick]

@elico filtering at mangle PREROUTING will drop any traffic before it is redirected at squid. In case I filter only at mangle FORWARD isn't there possible applications to use tcp port 80 through squid to go out (like teamviewer, skype, etc)? I will try to test this and see.

Testing for more then 24 hours with rules at mangle PREROUTING has not given any kernel panics.

I have a squid setup and I do not intend to replace it soon as I am interested on the caching option. Thanx though for pointing out RedWood.

[rightkick]

During testing I experienced kernel panic even when filtering at mangle PREROUTING.
What is left is to test filtering only of forwarded traffic at mangle FORWARD.
I will be satisfied with this if it is stable. I will revert with outcome.

[rightkick]

I have been tested for several days with 40Mbit traffic with filtering rules (DROP) at mangle FORWARD.
The setup has been stable.

[halvors]

With abort 1000 Mbit/s and 150 000 packet per second i get kernel panics exactly the same as mentioned above, this with @betolj's fork.

Hope this can be worked out seems to happend more often the more traffic. I was using this to drop torrent packets in FORWARD chain.

[halvors]

@rightkick The latest commit still have this problem, with higher speeds as 1Gbit/s it crashes after about 2-3 hours.