Activate all of them. If a bot shows up and is not activated it's treated as guest. This leads to two issues, they will get issued new SID with each request since they don't accept cookies creating an infinite amount of URL's to crawl. The second issue this will bloat the sessions table which can cause sessions table if it's very aggressive bot.
As side note if you see a lot of online users check the guests to see if any of them are identifying as bots with their user agent. If you find any add the bot to list.
If you want to block bots those that identify themselves will almost always obey robots.txt.
https://developers.google.com/search/do ... bots/intro
As side note if you see a lot of online users check the guests to see if any of them are identifying as bots with their user agent. If you find any add the bot to list.
If you want to block bots those that identify themselves will almost always obey robots.txt.
https://developers.google.com/search/do ... bots/intro
Statistics: Posted by thecoalman — Wed Apr 09, 2025 10:44 am
