metagriffin / modseclogc

A tool to manipulate and analyze ModSecurity audit log files.

ModSecurity Log Compiler

The modseclogc is a ModSecurity audit log file manipulation and analysis tool, command-line or python module based.

Project

• Homepage: https://github.com/metagriffin/modseclogc
• Bugs: https://github.com/metagriffin/modseclogc/issues

Installation

$ pip install modseclogc

Examples

View a request by unique ID:

$ modseclogc --match-id VtU2o38AAQEAAEV6AuwAAAAE modsec.log
--fc565b0b-A--
[01/Mar/2016:06:28:51 +0000] VtU2o38AAQEAAEV6AuwAAAAE 127.0.0.1 34882 127.0.0.1 80

[...snip...]

--fc565b0b-Z--

Display request IDs that match a path glob:

$ modseclogc --match-path /path/to/resource/** --show-id modsec.log
VtU2o38AAQEAAEV6Au0AAAAE
VtU2o38AAQEAAEV6AuwAAAAE
VtU2o38AAQEAAEV5BIgAAAAK

Display the request line and the request payload (modsec audit part "C") of each audit record:

$ modseclogc --show-request-line --show-parts C modsec.log
OPTIONS /path/to/resource HTTP/1.1
GET /path/to/resource HTTP/1.1
POST /path/to/resource HTTP/1.1
--40382b65-C--
query=foo+bar&page=1

GET /path/to/resource HTTP/1.1

Generate a copy of the audit excluding the payloads (modsec audit part "C") for a specific path glob, and compress the output:

$ zcat modsec_audit.log.gz \
  | modseclogc \
    --match-path /path/to/resource/** --hide-parts C \
    --unmatched keep \
  | gzip -9 > clean-modsec_audit.log.gz

Details

• By default, all input records are matched. This is modified via the --match-* and --inverse arguments.
• By default, all unmatched records are dropped. This is modified via the --unmatched argument.
• Output operations (show, hide, etc) only apply to matched records (note that the --inverse argument inverts the matching algorithm, not this rule).
• The audit log must be in Native format (see the SecAuditLogFormat modsec option).

modseclogc's Issues

support for libmodsecurity3 version 3

is this available? i'm using libmodsecurity3 with haproxy 2.4

File passed as argument generates "IOError: [Errno 2] No such file or directory"

Hello,

It seems modseclogc is not recognizing the file passed as an argument, at least for me:

jota@security-audits:~/modsecurity/ModSecurity-log-utilities$ modseclogc --show-request-line --show-parts C modsec.log 
Traceback (most recent call last):
  File "/usr/local/bin/modseclogc", line 11, in <module>
    sys.exit(main())
  File "/home/jota/.local/lib/python2.7/site-packages/modseclogc/cli.py", line 163, in main
    src = source.Source(open(fname, 'rb'), size=None, name=fname)
IOError: [Errno 2] No such file or directory: 'm'

jota@security-audits:~/modsecurity/ModSecurity-log-utilities$ modseclogc --show-request-line --show-parts C odsec.log 
Traceback (most recent call last):
  File "/usr/local/bin/modseclogc", line 11, in <module>
    sys.exit(main())
  File "/home/jota/.local/lib/python2.7/site-packages/modseclogc/cli.py", line 163, in main
    src = source.Source(open(fname, 'rb'), size=None, name=fname)
IOError: [Errno 2] No such file or directory: 'o'

Regards,
Jota

Add function show chosen parameters for each PARTS

Hi metagriffin

Is it possible to add a function to display only the "parameters" that we chose for each PARTS:
Example :
modseclogc --match-path /ajs.php --show-request-line --show-params Referer,X-Forwarded-For --hide-parts FHKZA modsec_audit.log

expected result :
GET /ajs.php?zid=13013 HTTP/1.1
--5a1c1823-B--
X-Forwarded-For: XXX.XXX.XXX.XX
Referer: http://www.test.fr/choserien/

thanks

undisplay parts with no data

hi Metagriffin
Thanks for this tool .

Is it possible to add a parameter that hide the "PARTS" when it's empty ? For example the command below display a lot of empty line because i have no "rule" modsec triggrered in my log file .

modseclogc --show-parts K modsec.logsecu

.......
--d86e6b12-K--

--95c52a3c-K--

--e7eb404f-K--

--cd51e70c-K--

......

Thanks