I was looking at AWSTATS for my site and it showed me that this host 64.124.85.79.become.com has downloaded 33470 pages from my site and chewed up over 1 gig in bandwidth in the last few days. Is this site worth it or should I block em? AWSTATS did not recognize them as a search engine since there was no hit on robots.txt.
Depends on whether you think getting listed in a search engine no one's ever heard of is worth whatever expense it's incurring. One gig over a few days doesn't sound like would really cost anything, to tell the truth, but I don't know what your hosting arrangements are.
Argh tell me about it, BecomeBot has been spamming up the logs of both my sites now for the last week under the ip 64.124.85.80, noxtrum.com is another nuisance bot...quite annoying crawlers they are.
Has anyone got any half decent number of hits from become? I am nearing my peak in bandwidth before I have to use the next package which is twice as much money per month. It's not going to break the bank or anything but I rather not pay it if I do not half too.
become is the worst bot ever...ban their class C and use the robots.txt as well..I hate that bot..totally useless and can suck up a ton of bw.