JesterMagic
Jan 25th 2006, 2:49 pm
I posted this in search engines but was wondering if anyone here has had become.com tear through there site?
I was looking at AWSTATS for my amazon site and it showed me that this host
64.124.85.79.become.com
has downloaded 33470 pages from my site and chewed up over 1 gig in bandwidth in the last few days. Is this site worth it or should I block em?
AWSTATS did not recognize them as a search engine since there was no hit on robots.txt.
I was looking at AWSTATS for my amazon site and it showed me that this host
64.124.85.79.become.com
has downloaded 33470 pages from my site and chewed up over 1 gig in bandwidth in the last few days. Is this site worth it or should I block em?
AWSTATS did not recognize them as a search engine since there was no hit on robots.txt.