Fail2ban jails not working about nginx-ultimate-bad-bot-blocker

[mitchellkrogza]

Try using in jail.local

backend = auto

I see you are using pynotify ??? did you change that for a reason?

[mitchellkrogza]

Here's my very stripped down jail.local

# Local Jail.conf File
[INCLUDES]
before = paths-debian.conf
[DEFAULT]
ignoreip = 127.0.0.1/8
ignorecommand =
bantime  = 600
findtime  = 600
maxretry = 6
backend = auto
usedns = yes
logencoding = auto
enabled = false
filter = %(__name__)s
destemail = [email protected]
sender = [email protected]
sendername = Fail2Ban
mta = mail
protocol = tcp
chain = INPUT
port = 0:65535
banaction = iptables-multiport

action_ = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
action_mw = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
            %(mta)s-whois[name=%(__name__)s, dest="%(destemail)s", sender="%(sender)s", protocol="%(protocol)s", chain="%(chain)s"]
action_mwl = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
             %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", sender="%(sender)s", logpath=%(logpath)s, chain="%(chain)s"]
action_xarf = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
             xarf-login-attack[service=%(__name__)s, sender="%(sender)s", logpath=%(logpath)s, port="%(port)s"]
action_cf_mwl = cloudflare[cfuser="%(cfemail)s", cftoken="%(cfapikey)s"]
                %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]
action_blocklist_de  = blocklist_de[email="%(sender)s", service=%(filter)s, apikey="%(blocklist_de_apikey)s"]
action_badips = badips.py[category="%(name)s", banaction="%(banaction)s"]
action_mybadips = badips[category="badbots", key="5c7dda713d5defa1994af8b3ee6270e8561566a5"]

action = %(action_mwl)s

# JAILS
[ssh]
enabled  = true
port     = ssh
filter   = sshd
logpath  = /var/log/auth.*
maxretry = 6

[ssh-ddos]
enabled  = true
port     = ssh
filter   = sshd-ddos
logpath  = /var/log/auth.*
maxretry = 6

[nginx-http-auth]
enabled = true
port    = http,https
filter = nginx-http-auth
logpath = %(nginx_access_log)s
maxretry = 3

[nginx-botsearch]
enabled = true
port     = http,https
filter = nginx-botsearch
logpath = %(nginx_access_log)s
maxretry = 6

[nginx-noscript]
enabled = false
port     = http,https
filter = nginx-noscript
logpath = %(nginx_access_log)s
maxretry = 6

[nginx-nohome]
enabled  = true
port     = http,https
filter   = nginx-nohome
logpath = %(nginx_access_log)s
maxretry = 6

[nginx-limit-req]
enabled = true
port    = http,https
filter = nginx-limit-req
logpath = %(nginx_error_log)s
maxretry = 1

[recidive]
enabled = false
logpath  = /var/log/fail2ban.*
banaction = iptables-allports
action = %(action_mwl)s
bantime  = 31536000   ; forever
findtime = 31536000   ; 3 months
maxretry = 10

[nginxrepeatoffender]
enabled = true
logpath = %(nginx_access_log)s
filter = nginxrepeatoffender
banaction = nginxrepeatoffender
bantime  = 86400   ; 1 day
findtime = 604800   ; 1 week
maxretry = 20

[cesar-castro]

I used to have backend = auto set on my default jail already, but I even tried to include it in every jail, and I still get this annoying message telling me that I'm using pynotify.

I also tried using your settings the way they are, I just removed nginx-noscript and nginx-nohome, because I don't have these filters. But I still get told that I'm using pynotify. I didn't touch my jail.conf. Is there any other place I can tell fail2ban to keep backend to auto?

[mitchellkrogza]

are you sure you have enough 403 or 444 errors to make it block someone?

try

grep '444' /var/log/nginx/mysite.com.error.log
grep '403' /var/log/nginx/mysite.com.error.log

and see how many occurences there are

[cesar-castro]

My logs just rotated recently, but I usually get tons of 444 requests from Seznam, DotBot and Ahrefs in a weekly period of time, and those requests are spotted on mysite.com.access.log, and not on the mysite.com.error.log, which is empty.

I'm experiencing this issue for two weeks or so, and ever since I never got a repeat offender banned so far, and also never figured out what's going on.

My ssh, and ssh-ddos jails are also using pynotify, but they are working. I can't image why the heck Fail2ban keeps picking up pynotify.

[cesar-castro]

Oh, it's also worth mentioning that back in the day I used to lower the maxretries to something like 2, to test if it would work without waiting for 20 444 requests.

[mitchellkrogza]

What I do is I don't gzip any of my Nginx logs so that they stay in plain text format. And the I keep them for 13 months. I do the same with my fail2ban.log files too I only rotate them once a month and do not gzip them and keep all logs for 13 months. This way when I tell a jail to do a findtime anytime within a year it does it with ease.

…

On 10 Apr 2017 17:48, "Cesar Castro" ***@***.***> wrote: My logs just rotated recently, but I usually get tons of 444 requests from Seznam, DotBot and Ahrefs in a weekly period of time, and those requests are spotted on mysite.com.access.log, and not on the mysite.com.error.log, which is empty. I'm experiencing this issue for two weeks or so, and ever since I never got a repeat offender banned so far, and also never figured out what's going on. My ssh, and ssh-ddos jails are also using pynotify, but they are working. I can't image why the heck Fail2ban keeps picking up pynotify. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#23 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJgARX2Q0TRGNEaesdhQk8nHDpQYuYEsks5ruk9mgaJpZM4M43SF> .

[cesar-castro]

Ohh, that's a good one. How can I reflect your settings? Also, which backend do your jails pick up? I'm not forcing pynotify, it happens that it's being randomly picked up when my backend is set to auto.

Thank you once again!

[mitchellkrogza]

Will put the changes you need to make in log rotate here for you tomorrow

…

On 10 Apr 2017 20:07, "Cesar Castro" ***@***.***> wrote: Ohh, that's a good one. How can I reflect your settings? Also, which backend do your jails pick up? I'm not forcing pynotify, it happens that it's being randomly picked up when my backend is set to auto. Thank you once again! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#23 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJgART4BktyDc-HKo3jaIOrF1SV9nrn8ks5rum_fgaJpZM4M43SF> .

[cesar-castro]

Thank you so much, bro!

:-)

[mitchellkrogza]

Edit the following 2 files and replace with these contents.

nano /etc/logrotate.d/fail2ban

/var/log/fail2ban.log {
    missingok
    notifempty
    monthly
    rotate 13
    create 640 root adm
    postrotate
        /usr/bin/fail2ban-client set logtarget /var/log/fail2ban.log 2> /dev/null || true
    endscript
}

nano /etc/logrotate.d/nginx

/var/log/nginx/*.log {
	monthly
	missingok
	rotate 13
	notifempty
	create 0640 www-data adm
	sharedscripts
	prerotate
		if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
			run-parts /etc/logrotate.d/httpd-prerotate; \
		fi \
	endscript
	postrotate
		invoke-rc.d nginx rotate >/dev/null 2>&1
	endscript
}