force_consent bad for crawler bots? about cookieconsent

[orestbida]

Hi @PWaddict,

as far as I know, web crawlers directly inspect/download the entire markup of the page. Thus, they completely ignore the fact that there's a modal or not. Instead, it is important that you have proper sitemap and robots.txt files.

You can block crawlers using the following methods:

• noIndex tag: <meta name="robots" content="noindex">
• robots.txt
• ad-hoc client/server side code

You can also check if your website can be analyzed/crawled with online tools like Sureoak Seo Crawler.

[lamtonylam]

I had a problem with it.
I've implemented on my cookie notice, what if the device is mobile force_consent is off, and on desktop it is on.

const isMobile = window.matchMedia("only screen and (min-width: 760px)").matches; if (isMobile) { _addClass(html_dom, 'force--consent'); }

https://github.com/lamtonylam/opkh/blob/08b7f7f480084841fd420b8f6bbde8ea99cf3d46/static/keksi.js#L156

This fixed the issue, because google indexes using mobile crawler.

[orestbida]

as of v2.6.0, a new option hide_from_bots was added.

If enabled, the plugin will detect if the current client is a bot/crawler; if it is a bot then it will prevent itself from running, to avoid the indexing of it's own text content and also any possible issue with modals.