I have issues with bad traffic (mostly bad actors and bad robots) and I waste a lot of time banning proxies, but it seems like there is no end to it. I am looking for alternate solutions such as IP filters or phone verification. Huge IP filters can have a dramatic impacts on performance so it would have to be an external service. ipqualityscore has a proxy filtering service but it's extremely expensive (500$ for 500K queries) I haven't found much info about phone verification service and its implementation or price. I would appreciate any info or experience about the topic. Thank you.
I used to have that problem but I solved it with some plugins. If you use WordPress you can install Blackhole for Bad Bots or Cloudflare. Cloudflare offers security against vulnerability and bot traffic.
Separate from rate-limiting and direct engineer intervention, the easiest and most effective way to stop bad bot traffic is with a bot management solution. A bot management solution can leverage intelligence and use behavioral analysis to stop malicious bots before they ever reach a website. Instead, use the suggestions and tips below to learn what you can do to avoid dangerous sites and guard your safety online. Use a Web Filter. Don't Guess the Address of a Website. Check the URL for Issues. Never Open Questionable Sites. If It Looks Unreal, It Probably Is. Choose Your Searches Carefully. Use a Link Scanner.
It's not just a bad robot problem, it's also a bad actors problem, and bad actors can be as much a threat as bad robots for bigger sites. And the one thing bad actors and bad robots share is the use of open proxies/vpn. Anyway, I have found an interesting link that might help. A list of bad ASN created by people who clearly had similar problem to mine. https://github.com/brianhama/bad-asn-list The list is 3 years old sadly, so I recommend to verify it carefully before using it. You can then use a firewall to filter bad ASN. Blocking proxies at the ASN level is the most efficient protection I can think of, sadly I can't think of a more simple and manageable way.
Once again I must stress the point here is not to block only robots, but also human bad actors. Bad actors can be a bigger threat to a website than bad robots. I've heard a famous streaming website blocks all VPN, so there must be solutions out there to block all VPN. I know it's not possible to block 100% of VPNs because new VPN are created every day, including in private residences, but if I could block at least 95% that would be good enough. My goal is to make it a chore to find an IP that works. So please share any anti VPN solution you know about, I personally haven't had much luck with a google search.