stklcode / statify-blacklist Goto Github PK

The current filter strips sld.tld from referer URLs. The idea was to simplify wildcard filters for spammers using multiple subdomains as server1.example.com, server2.example.com , etc.

This filter rule obviously does not work for country-code secold-level-domains as example.co.uk.

Filter has to be expanded to at least 3 parts or dropped completely.

Originally reported in WP support forums (German):
https://wordpress.org/support/topic/probleme-mit-blacklisteintrag/

International Domain Names (IDNs, more precise IDNA2008) are not handled correctly. Besides Statify apparently not resolving those entries nicely, the blacklist does not accept such entires for filtering.

Should be fixed and documented.

To not mess up on sites with incompatible WP and PHP versions add a simple check before activating any plugin functionality and disable the plugin if already active.

With the changes in v1.4.1 it is now possible to filter target pages and IPs. Unfortunately the appender still checks for the availability of referer live filtering.

This check has to be corrected to add the filter if any (live) blacklist is activated.

Add the ability to filter by regular expressions instead of (or in addition to?) domain in_array check.

Performance is far worse than the simple filter, so not recommended for live filtering, but using cron or manual clean-up is should be fine.

Matomo is maintaining a referrer spam list:
https://github.com/matomo-org/referrer-spam-blacklist

Maybe you can use it or add a feature to set up an URL for using it.

Feature suggested in the WP support forums (https://wordpress.org/support/topic/blacklisten-bitte-zur-auswahl-stellen/#post-10381313 - German).

IP addresses of blocked visitors (by any metric except IP-blacklist itself) should be collected in a local blacklist (similar to the approach of AntispamBee for comments) and automatically excluded from tracking.

This feature has to be an opt-in and store addresses anonymized (at least a salted hash) to not break GDPR compliance.

Feature request from WP support forums: https://wordpress.org/support/topic/great-feature-request/

Introduce a feature to white- or blacklist requests from certain countries from tracking.

To achieve this, a geolocation service has to be integrated and processed during live filter - cron cleanup is not possible here because the IP is not saved.

Such service might be MaxMind GeoIP2 which provides a nice PHP API¹. Keep in mind the latest changes for database usage².

[1] https://maxmind.github.io/GeoIP2-php/
[2] https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/

In 1.1.x the filter hook splits referer domain by dot and runs the last 2 parts (sld.tld) against a blacklist using in_array(). It's fast, but it can be faster, as the hook is exeuted on every visit.

Some evaluation hast to be made to compare different methods in a real-world scenario (about a dozen blacklisted domains with a small percentage of hits).

First improvement in develop branch is the switch from in_array() to isset() by flipping the array.

Received a bug report with reproducible misbehavior in v1.7.0.

Seems the "CleanUp Database" feature is broken after rewriting the settings page.

There's a problem with the plugin, as of 1.7 the 'CleanUp Database' is not working any more.
After roll back to 1.6.3 it's working again.
Verified on WP 5.0.x and WP 6.4.x, both with Statify 1.8.4.

The setting file is completely rewritten and it's difficult to compare with the former one. But it looks like you forgot the call of the cleanup function.

Inspired by this WP forums discuisson (German): https://wordpress.org/support/topic/ip-blacklist-fur-subnetze/#post-12634053

Implement a filter for user agents, analogous to the existing referer filter mechanism.

Statify itself already has some checks to exclude bots. Might be helpful anyway for flexible reaction on new bots or specific strings of spammers which do not identify themselves as bots/crawlers.

As part of v1.4.1 parse_url() has been changed to wp_parse_url() which requires at least WP 4.4 (second parameter as of 4.7 is not used here).

This change has already been published without community complains (and Statify itself is requires 4.7 as of 1.5.2), so just change the README file here and don't revert any code if not necessary.

In addition to the existing referer blacklist, an optional IP address filter should be implemented.

Requirements

• efficient matching on single addresses and subnets
• compatibility with both IPv4 and IPv6 addresses
• comfortable input (e.g. CIDR notation and/or ranges)

Limitations
Since addresses are not stored, this filter can only be applied during live filtering.
For a consistent UI experience, the input form has to be adapted to that.

Reported on German WordPress support forums (https://wordpress.org/support/topic/seit-einsatz-des-plugin-werden-admin-besuche-mitgezahlt/):

When using the blacklist plugin, admin logings are tracked. Seems that statify's hook returns the filter value and does not continue with it's own filters after execution.

Due to an added check in a previous commit that should disable the live filter completely if not activated, the condition is inverted.

if ( self::$_options['active_referer'] != 1 ) {
    add_filter( 'statify_skip_tracking', array( 'StatifyBlacklist', 'apply_blacklist_filter' ) );
}

Obviously it should be != 0, seems to be copy&paste mistake from the filter method itself.

On multisite installations the activation hook for new sites references a non-existing method StatifyBlacklist_Install::init_site().

Second issue is triggered on network wide activation. For compatibility with WP < 4.6 a fallback to wp_get_sites() is called, which generates an array of associative arrays, while get_sites() returns an array of WP_Site objects, hence $site['blog_id'] and $site->blog_id both fail in one case.

When using regular expressions, the filter strings are simply saved to database. This raises a warning from preg_match() when an expression is invalid or clashes with delimiters /.

These expresions should be validated before saving or - if possible - sanitized (analog to URL sanitization for non-regex referrer filter) to avoid such warnings.

If live-filtering of referrers or targets with regular expressions is enabled and does not match, the filter function returns null and the following filters (target and/or IP), if enabled, are skipped.

Implement an optional cron function to execute DB cleanup as time-based action instead of live filtering.

Extension suggested in the WP support forums (https://wordpress.org/support/topic/blacklisten-bitte-zur-auswahl-stellen/#post-10381313 - German).

The referrer filter currently has 3 operational modes: (sub)domain, regular expression, regular expression case-insensitive.

As suggested, extend this option to allow a simple keyword filter that excludes a visitor from tracking if any keywords is contained in the provided list.

Analogous to the referer filter, a target filter might be a handy feature to exclude certain pages or posts from tracking.

Skimming over the table there are always a couple of requests to suspicious pages in the admin area that usually end up in a 404 state and thus not recognized as admin pages (that are filtered out by Statify itself) and also a handful of requests to /.git/ or /.well-known/ locations.

The filter mechanisms can be the same as for referer (live, cron, regexp, manual cleanup).