If you're visiting this site on the clear-web, you may have noticed a brief landing page featuring an anime cat girl.
That's the mascot of the Anubis project, created by Techaro.
The purpose of that project is block overly aggressive scrapper bots. This blog is fairly unknown and receives less than 100 legitimate views a day. My tiny little webserver is plenty capable of sharing my thoughts with those few people who stumble onto this site, however, it also sees hundreds of thousands of visits from scrapper bots. These bots might not harm massive CDNs like Google or Cloudflare, but can easily overwhelm tiny, self-run webservers like mine. I've been using rate-limiting and blocking connections based on user-agents and IP addresses, but the scrappers continue to become more malicious, using residential IPs, dynamic user-agents other tricks to continue to hammer this tiny server.
Anubis is an interesting solution to the problem. It introduces a proof-of-work requirement, where it essentially sends a math problem to your browser to complete. Your browser solves the problem and sends it back, then you're admitted into the site. At the moment, the scrapper bots aren't designed to account for this challenge, so they can't get through to the site, drastically reducing the load on my server.
Chances are, as Anibus continues to build in popularity, the scrappers will eventually be updated to take this challenge into account and circumvent this protection... But until then, lets enjoy the temporary calm.
