- Joined
- Mar 4, 2019
I've mitigated things somewhat by blocking most of the far east and india, but I'm currently getting raped by dutch IP addresses. I'm probably going to shut off http for a while until I can figure it out.How do you guys deal with AI crawlers on your home server? I got anubis, but some other bots are bypassing it. I had to block windows and macos user agents, but the other day I saw some using android user agent instead. Soon I'll have to block everyone except linux desktop users from accessing my server to make it usable.
The AI crawlers kill my server, making all cores run at 100% which results in the server timing out requests.