I used fail2ban as an anti-scraper strategy for a few months. It helped with the really basic scrapers that only use one or a few IP addresses.
But then they adapted and started using hundreds or thousands of different IP addresses, each one only making a couple of requests. fail2ban became counter-productive as it slowed my server down - the log files it was monitoring became massive under high load and it needs to scan the log files often to check if an IP has been seen before.
I’m not saying don’t try it, I’m saying it’s only effective in some situations.




I did not, it seemed complicated and rabbit-hole-y.
In the end I just added an Anubis-like feature into PieFed. It works great.