I want to bann all netseer.com crawlers IP's from spidering my website, from linux firewall (iptables).. There are over 500 different IP's from where they are access my server.. few IP's are: 67.202.22.253 67.202.23.133 67.202.24.156 67.202.24.170 how can i ban this entire IP range? and what is the range that should be banned? it is 67.202.0.0-67.202.255.255 or ..?
You'll have to block it using CIDR notation as far as I know, if I'm not completely wasted it should be 67.202.0.0/16 to block the entire 67.202.xxx.xxx range iptables -I INPUT -s 67.202.0.0/16 -j DROP will drop all the packets that come to your machine from 67.202.0.0-67.202.255.255 But keep in mind this will block 65k ips so I'd rather try seeing if I could block a bit closer.. and see if they only come from like 67.202.20.xxx-67.202.25.xxx
If you have a firewall installed it would be more easier to block multiple IPs. You can install APF for this, it works excellent.
I had a look at the interactive map of the internet and it looks like the range you want to block belongs to Amazon. The range they own that includes all of the IP addresses you mentioned is denoted by 67.202.0.0/18. I have noticed that well-behaved bots stop requesting pages if you start sending them 403 HTTP responses. This might be useful if you want to block based on User-Agent rather than IP address.