bots, bots, and more bots

[kwblue]

Anyone else have a problem with so many bots hitting their site? I've been troubleshooting some slowness on my site - used to be blazing fast - and seeing a lot of different bots just eating up time. A few of the big ones:

ahrefsbot
semrushbot
mj12bot
backlinksextendedbot
amz-searchbot
meta-webindexer
petalsearch

And then the other crap I have to block... things like:
slot-data-scraper
slots-scraper

I mean... c'mon, be f'n original people.

Anyway - I've been a week troubleshooting all sorts of stuff. It has severely impacted my site because I've turned off a lot of features that I had thinking maybe my queries were too 'expensive'. Hopefully I remember everything I turned off because I was just starting to gain a lot of traction in the SERPs.

It's just all so frustrating - so I'm venting. And these scraper people that just want to duplicate our content - **** off.

[invinciblepoker]

"bot fight mode" on cloudflare is free and useful icymi fwiw, Toggle "Block AI Bots": specifically blocks AI scrapers and crawlers from OpenAI, Anthropic, and others. also WAF mode is useful

#free

[kwblue]

"bot fight mode" on cloudflare is free and useful icymi fwiw, Toggle "Block AI Bots": specifically blocks AI scrapers and crawlers from OpenAI, Anthropic, and others. also WAF mode is useful

#free

I was going to show a screenshot because I had turned on Bot Fight Mode, but apparently I turned it back off at some point I do have some rules to block some things (like scraper bots, now semrush and ahrefs, and a few others) that I just created. That will help for sure. Bot fight mode is back on.

I will keep the AI bots for now and look in the future at this one - thanks!