I have a script plug in of 20 or so internet radio stations for my readers to access, however, I have noticed an increase in Bandwidth use that doesn't match the traffic. Is it possible that this is the bandwidth hog? I assumed since I was not the radio station that the burden was on the host, and I was just establishing a link. Is this correct?
well, I can't be sure. I have a lot of bandwidth coming from 3 IP addresses, that are registered to Yahoo, 216.109.121.44 11594 11594 400.13 MB 19 May 2007 - 23:13 216.109.121.41 11560 11561 405.01 MB 19 May 2007 - 23:12 216.109.121.42 11374 11374 398.89 MB 19 May 2007 - 23:13 and I 'm just trying to track it down.
It's possible that the traffic is actually Yahoo's spiders. They could be traversing your site. A good 'robot' ruleset might be a good thing to implement to reduce them from accessing images & other information that they may not really need to.
Any tips on how I should implement that, they are sucking up a lot of bandwidth, and it's freaking me out.
hmansfield, You could block the spiders temporarily with something like the following in your .htaccess: RewriteCond %{HTTP_USER_AGENT} Slurp RewriteRule ^.*$ X.html [L] Replace 'Slurp' with the agent's name from your apache logs. Slurp is 'inktomi's' horrible little robot. Note that blocking a search engine that provides you with a lot of traffic would be a bad idea, obviously even from an SEO standpoint. This following article could explain it better than I'd ever be able to: http://www.thesitewizard.com/archive/robotstxt.shtml It also has some tips/tricks.