kurobeats / fail2ban-grok-log-pattern

Fail2ban GROK Log Pattern

This is just a Fail2ban log GROK pattern to extract useful infor from fail2ban logs.

todo aka please help

I don't completely understand the fail2ban log format and you'll see two variables that are (imo) absolute rubbish. Pull requests welcome.

Updated on 3/20/19

Can now be used as a filter.conf file that you can put into an already in development .conf file, or as a seperate filter file depending on your conf.d architecture.

fail2ban-grok-log-pattern's Issues

Better pattern

Hi,

I used your pattern as starting point. Because fail2ban in v0.10+ can handle IPv6 you might change the pattern. Furthermore on "service" (the name of the triggered fail2ban jail) might contain "-" or other chars, the grok pattern "WORD" might not work then.

OLD version:
"pattern" : "%{TIMESTAMP_ISO8601:timestamp},%{INT:fail2ban_digit} fail2ban.%{WORD:fail2ban_action} *\[%{INT:fail2ban_digit2}\]: %{LOGLEVEL:level} *\[%{WORD:service}\] %{WORD:ban_status} %{IPV4:clientip}"

NEW version:
"pattern" : "%{TIMESTAMP_ISO8601:timestamp},%{INT:fail2ban_digit} fail2ban.%{WORD:fail2ban_action} *\[%{INT:fail2ban_digit2}\]: %{LOGLEVEL:level} *\[%{GREEDYDATA:service}\] %{WORD:ban_status} %{IP:clientip}"

You want it even better? In logstash I'm namespacing the output, to have a cleaner access (e.g. in Kibana Discover tab), here's my very own, namespaced pattern:
%{TIMESTAMP_ISO8601:[f2b][timestamp]},%{INT:[f2b][fail2ban_digit]} fail2ban.%{WORD:[f2b][fail2ban_action]} *\[%{INT:[f2b][fail2ban_digit2]}\]: %{LOGLEVEL:[f2b][loglevel]} *\[%{GREEDYDATA:[f2b][service]}\] %{WORD:[f2b][ban_status]} %{IP:[f2b][clientip]}

The output will then be:

{
"[f2b][timestamp]": "2018-01-01 12:00:00",
"[f2b][service]": "my-own-jail",
"[f2b][clientip]": "1.2.3.4",
"[f2b][fail2ban_action]": "actions",
"[f2b][ban_status]": "Ban",
"[f2b][fail2ban_digit]": "123",
"[f2b][fail2ban_digit2]": "4567",
"[f2b][loglevel]": "NOTICE"
}

Regarding your "To Do/Help Wanted"

The first "random" number (,INT) is actually part of the timestamp. It is ISO8601 timestamp format's representation of the fraction of the second. So it is not a separate field.