smcho-kr / kpcre

PCRE Linux Kernel Module & PCRE Text Search Engine

PCRE linux kernel module & PCRE/REGEX text search engine

Keywords: Netfilter iptables PCRE REGEX Linux Kernel Module

PCRE library kernel module (libpcre2-X.ko)

The PCRE library is a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. PCRE has its own native API, as well as a set of wrapper functions that correspond to the POSIX regular expression API.

This is a PCRE2 library ported to linux kernel. It is a full-fledged PCRE kernel module which supports JIT(Just-in-time) compilation.

Original sources: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre

PCRE text search engine (ts_pcre.ko)

This is a new text search engine based on the PCRE kernel module.

Getting the source code

To get the source code from the git repository

git clone https://github.com/smcho-kr/kpcre.git

Installing

To install the modules follow these steps, always from the modules package root (i.e. where this file is located)

To compile the modules first type:

make modules

Then, as root type:

make modules_install

That would install the pcre & ts_pcre modules for the given Linux kernel.

For more detailed instructions on how to build and install the kernel modules, refer to the installation guide.

Usage

Once you have installed both modules ("libpcre2-X.ko" and "ts_pcre.ko") you should type (as root):

modprobe ts_pcre

If the module has been successfully loaded you shouldn't see any message. After loading the kernel module you can use iptables to add a rule.

An example rule would be (as root):

iptables -A INPUT -m string --string "/\x7C\x7C.+[a-z]/i" --algo pcre -j DROP

This wouldn't allow any incoming traffic that has the content matching the given PCRE in the payload.

In case you want to stop using the ts_pcre kernel module, first remove every iptables rule for ts_pcre and then type (as root):

modprobe -r ts_pcre

There is also a REGEX text search engine. (ts_regex.ko)

kpcre's Issues

Error compiling on CentOS 6.8 i686

Same problem as described before but this time only on 32-bit CentOS 6.8
Compiling on x64 CentOS 6.8 with same setup works flawlessly.
Would be great if this problem could be fixed. Excellent module.

WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_substring_get_bynumber_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_substring_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_context_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_compile_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_context_create_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_get_ovector_pointer_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_assign_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_data_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_compile_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_data_create_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_code_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_create_8
make[2]: Leaving directory /usr/src/kernels/2.6.32-642.15.1.el6.i686' depmod -a; make[1]: Leaving directory /root/kpcre/ts_pcre'
make[1]: Entering directory /root/kpcre/ts_regex' make -C /lib/modules/2.6.32-642.15.1.el6.i686/build M=$PWD modules_install; make[2]: Entering directory /usr/src/kernels/2.6.32-642.15.1.el6.i686'
INSTALL /root/kpcre/ts_regex/ts_regex.ko
DEPMOD 2.6.32-642.15.1.el6.i686
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol pcre2_substring_get_bynumber_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol pcre2_substring_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol pcre2_compile_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol regcomp
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol regexec
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol pcre2_match_data_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol pcre2_match_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol pcre2_match_data_create_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol pcre2_code_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_regex.ko needs unknown symbol regfree
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_substring_get_bynumber_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_substring_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_context_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_compile_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_context_create_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_get_ovector_pointer_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_assign_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_data_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_compile_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_match_data_create_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_code_free_8
WARNING: /lib/modules/2.6.32-642.15.1.el6.i686/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_create_8
make[2]: Leaving directory /usr/src/kernels/2.6.32-642.15.1.el6.i686' depmod -a; make[1]: Leaving directory /root/kpcre/ts_regex'

Problems seems to be segmentation fault when compiling pcre2_jit_compile.c.
Tried on both Intel and AMD CPUs with same result.

CC [M] /root/kpcre/pcre2/pcre2_jit_compile.o
/bin/sh: line 1: 3950 Done(2) gcc -E -D__GENKSYMS__ -Wp,-MD,/root/kpcre/pcre2/.pcre2_jit_compile.o.d -nostdinc -isystem /usr/lib/gcc/i686-redhat-linux/4.4.7/include -Iinclude -I/usr/src/kernels/2.6.32-642.15.1.el6.i686/include/uapi -I/usr/src/kernels/2.6.32-642.15.1.el6.i686/arch/x86/include -Iarch/include/generated -Iinclude -include /usr/src/kernels/2.6.32-642.15.1.el6.i686/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -O2 -m32 -msoft-float -mregparm=3 -freg-struct-return -mpreferred-stack-boundary=2 -march=i686 -mtune=generic -Wa,-mtune=generic32 -ffreestanding -fstack-protector -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -Wframe-larger-than=1024 -Wno-unused-but-set-variable -fomit-frame-pointer -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fno-dwarf2-cfi-asm -fconserve-stack -O2 -s -I/root/kpcre/pcre2/../libc/ -DHAVE_CONFIG_H -DPCRE2_CODE_UNIT_WIDTH=8 -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(pcre2_jit_compile)" -D"KBUILD_MODNAME=KBUILD_STR(libpcre2_8)" -D"DEBUG_HASH=31" -D"DEBUG_HASH2=1" /root/kpcre/pcre2/pcre2_jit_compile.c
3951 Segmentation fault | scripts/genksyms/genksyms -a i386 -r /dev/null > /root/kpcre/pcre2/.tmp_pcre2_jit_compile.ver

Cannot compile on CentOS 7.2

[root@fern1 kpcre]# make modules
CODE_WIDTH is 8
make[1]: Entering directory `/root/kpcre/libc'
make -C /lib/modules/3.10.0-514.2.2.el7.x86_64/build M=$PWD modules;
make: Entering an unknown directory
make: *** /lib/modules/3.10.0-514.2.2.el7.x86_64/build: No such file or directory.  Stop.
make: Leaving an unknown directory
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/root/kpcre/libc'
make[1]: Entering directory `/root/kpcre/pcre2'
make -C /lib/modules/3.10.0-514.2.2.el7.x86_64/build M=$PWD modules;
make: Entering an unknown directory
make: *** /lib/modules/3.10.0-514.2.2.el7.x86_64/build: No such file or directory.  Stop.
make: Leaving an unknown directory
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/root/kpcre/pcre2'
make[1]: Entering directory `/root/kpcre/ts_pcre'
make -C /lib/modules/3.10.0-514.2.2.el7.x86_64/build M=$PWD modules;
make: Entering an unknown directory
make: *** /lib/modules/3.10.0-514.2.2.el7.x86_64/build: No such file or directory.  Stop.
make: Leaving an unknown directory
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/root/kpcre/ts_pcre'
make[1]: Entering directory `/root/kpcre/ts_regex'
make -C /lib/modules/3.10.0-514.2.2.el7.x86_64/build M=$PWD modules;
make: Entering an unknown directory
make: *** /lib/modules/3.10.0-514.2.2.el7.x86_64/build: No such file or directory.  Stop.
make: Leaving an unknown directory
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/root/kpcre/ts_regex'
make: *** [modules] Error 2

dot as dot

Hi there,
many thank's for your code
i was install your code to my linux machine ( ubuntu 16.04 over KVM ) and running but i have 1 issue. where in regex dot not as dot. i wan to know hot to get dot as dot. i try like this

iptables -A INPUT -m string --string "/\bm[0-9]{1,2}.[a-z]{7}.[a-z]{2,3}(.[a-z]{2,3}|)\b/i" --algo regex -j DROP
or
iptables -A INPUT -m string --string "/\bm[0-9]{1,2}.[a-z]{7}.[a-z]{2,3}(.[a-z]{2,3}|)\b/i" --algo regex -j DROP
or
iptables -A INPUT -m string --string "/\bm[0-9]{1,2}.[a-z]{7}.[a-z]{2,3}(.[a-z]{2,3}|)\b/si" --algo regex -j DROP

but no luck. it's dot still as any character

i implement it to block dns request like this m12.aaaaaaz.com and success. but when dns request like this m12daaaaaazdcom.com or m12daaaaaaz.com blocked too.

can you tell me how to make dot as dot in KPCRE

sorry if my English so bad.

Kpcre not build and not compile in Debian 11 (Bullseye)

Hello, friend!

I hope that the letter has been sent by me to the correct address and it will reach you. I have been using your development kpcre for iptables for several years now. Thank you very much for your work. But now there is a problem: in the latest version of Debian 11 (Bullseye), your development refuses to build and compile. Please, if you have the time and opportunity, update your project code at https://github.com/xnsystems/kpcre.git so that your development can work in the latest version of Debian 11 (Bullseye). I understand that you are most likely a very busy person, but I am not a programmer, but a system administrator and I have no one else to turn to except you. I hope that you will respond to my request and can take the time to solve the problem with kpcre with the latest Linux kernels. I would be glad to talk with you on other topics as well, I have many-sided interests in the field of information technology. I would be glad to receive your answer to my letter.

The license?

Any chance to change the license from GPL to BSD?

Can not install on centos 6.8

Can not install what shoud I do ? thanks

WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_regex.ko needs unknown symbol pcre2_substring_get_bynumber_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_regex.ko needs unknown symbol pcre2_substring_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_regex.ko needs unknown symbol pcre2_compile_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_regex.ko needs unknown symbol pcre2_match_data_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_regex.ko needs unknown symbol pcre2_match_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_regex.ko needs unknown symbol pcre2_match_data_create_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_regex.ko needs unknown symbol pcre2_code_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_substring_get_bynumber_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_substring_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_match_context_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_compile_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_match_context_create_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_get_ovector_pointer_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_assign_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_match_data_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_match_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_jit_compile_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_match_data_create_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_code_free_8
WARNING: /lib/modules/2.6.32-642.3.1.el6.x86_64/extra/ts_pcre.ko needs unknown symbol pcre2_jit_stack_create_8
make[2]: Leaving directory `/usr/src/kernels/2.6.32-642.3.1.el6.x86_64'
depmod -a;
make[1]: Leaving directory `/home/kpcre/ts_regex'

dkms.conf

I have created the dkms.conf

PACKAGE_NAME="kpcre"
PACKAGE_VERSION="0.0.1"
MAKE[0]="make KERNEL_DIR=/lib/modules/$kernelver/build modules"
CLEAN="make clean"
BUILT_MODULE_NAME[0]="libpcre2-8"
BUILT_MODULE_NAME[1]="ts_pcre"
BUILT_MODULE_NAME[2]="ts_regex"
BUILT_MODULE_NAME[3]="libc"
BUILT_MODULE_LOCATION[0]="pcre2/"
BUILT_MODULE_LOCATION[1]="ts_pcre/"
BUILT_MODULE_LOCATION[2]="ts_regex/"
BUILT_MODULE_LOCATION[3]="libc/"
DEST_MODULE_LOCATION[0]="/kernel/drivers/misc/kpcre"
DEST_MODULE_LOCATION[1]="/kernel/drivers/misc/kpcre"
DEST_MODULE_LOCATION[2]="/kernel/drivers/misc/kpcre"
DEST_MODULE_LOCATION[3]="/kernel/drivers/misc/kpcre"
AUTOINSTALL="yes"
REMAKE_INITRD="yes"

Works fine on Debian Jessie