Hello everyone...
i want ask about how to block robots ahrefs/majestic/semrush/moz ?
My questions is what script to hide/block in .htaccess ?
thankyou
Hello everyone...
i want ask about how to block robots ahrefs/majestic/semrush/moz ?
My questions is what script to hide/block in .htaccess ?
thankyou
try something like this in robots.txt:
User-agent: AhrefsBot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: SemrushBot
Disallow: /
Or
via htaccess:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} AhrefsBot [OR]
RewriteCond %{HTTP_USER_AGENT} MJ12bot [OR]
RewriteCond %{HTTP_USER_AGENT} SemrushBot [OR]
RewriteRule . - [F,L]
I don't know what bot use Moz, but i am pretty sure that you can find it
Hope that helps a little.
Jonui (17 October 2020)
Hope this will help - https://gorilla.marketing/seo-news/h...m-competitors/
IRON WITHIN IRON WITHOUT
www.slots43.com | www.slots43.com/ru/ | Cosmolot24 | aussielegitcasinos.com | toponlinecasinolist.com | ua.toponlinecasinolist.com
Jonui (17 October 2020)
What I find ironic is that gorilla marketing claims to do something they do not themselves do.
What they claimed they did in their own robots text file is not what is is in their own robots text file.
Rick
Universal4
robyroy (18 October 2020)
Here is a little trick I've tried in .htaccess
RewriteCond %{HTTP_USER_AGENT} !(Google|msnbot|bingbot|facebook|duckduck) [NC]
RewriteCond %{HTTP_USER_AGENT} (Bot|Spider|Crawl) [NC]
RewriteCond %{REQUEST_URI} !^bots\.htm
RewriteRule .* /bots.htm [L]
That basically says :
- if it's not Google, msn, bing etc
- and if it has the words bot or spider or crawl
- and it's not already looking for a file called bots.htm
- then send them to a file called bots.htm
This doesn't give them an error code - just all they ever see is a file called bots.htm no matter what they ask for.
That hopefully screws up their plans and they don't notice that they've been directed to a dummy file.
casinoportal (18 October 2020), Michael Martinez (18 October 2020), universal4 (17 October 2020)
Ha....and if you wanna be mean, make the bots.htm for certain useragents an infinite loop....or at least one that goes on for a few days....lol
Rick
Universal4