It happened. whois info IP Information for 72.30.81.159 crawl.yahoo.net United States New York Inktomi Corporation llf531218.crawl.yahoo.net I am disappoint that yahoo keep doing this kind foolish behavior. I have stopped it and made it redirect to itself. The question is why yahoo doing this? Do you have any good solutions? really like to know Thanks
use crawl delay in your robots txt... see http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-03.html as far as i can tell tho, the time in seconds is per bot, so if you set it for 10 seconds and they have 10 bots on your site then altogether yahoo will make a request every second.
glad to be of some help. make sure you also block in robots.txt any duplicate content, things you don't need it to see etc and that will help reduce the load on the server. One of my sub domains has crawl delay set to 300 for yahoo otherwise it goes totally mad. like your serps checker by the way http://www.allno1.com/google_serp/multi_country_rank_checker.asp 1st time i have seen a site that does that and is a tool i will use often.