Yahoo is killing me with favicon.ico requests - immediate help please!

[jim]

Yahoo has just cause my web hosting company to shut me down:
We're getting a ton of individual requests for your favicon.ico file on the site from what appears to be the default home page for SBC and some other ISPs via Yahoo. I've moved this file out of the way, but the requests keep pouring in for it at a rate of about 10-15 per second. And while serving up individual files is fast, it's the sheer magnitude that causes the load to go up. Can you contact Yahoo or SBC to see about getting that icon removed from their pages? I find it kind of odd that they don't store it locally and rely on your version instead. If we can get it to stop, that's the majority of the load right there.
Does anyone know how I should proceed? Hod do I get Yahoo or SBC (who is SBC anyway?) to stop using my favicon.ico file?

[chachi]

Are you sure that it is actually Yahoo and/or SBC and not someone pretending like they are Yahoo/SBC? Do you know the IP addresses?

[Austars]

Well I just typed in sbc.com and got http://www.att.com/gen/landing-pages?pid=3308 and I don't see any favicon.ico files on that page...

[jim]

I don't but I suppose my web host does. I'll ask them if they could have been fooled. Can I just delete the file or block all requests to it or something?Are you sure that it is actually Yahoo and/or SBC and not someone pretending like they are Yahoo/SBC? Do you know the IP addresses?

[Cypherus]

How about mod_rewrite, hotlink protection?

[alexdale]

How about mod_rewrite, hotlink protection?

i agree, you can activate hotlink protextion on your host's cpanel.

omg that should've consumed tons of bandwidth :eek:

[jim]

I set up hotlink protection and added .ico files so hopefully this will do the trick. I'm not too confident though because all the sites on my account went down a few hours after I got the site back up. No word yet, on whether that had anything to do with my site or not.

As for bandwidth, my one site gobbled up about 50 GB in a few days so it was pretty serious.

Can I just add how irritating it is to have a webhost with no phone support in cases like this. A few people here recommended asmallorange but they've don'e very little to help me as far as I can tell. When I asked them if hotlink protection would do the trick they never even answered so who knows?

i agree, you can activate hotlink protextion on your host's cpanel.

omg that should've consumed tons of bandwidth :eek:

[vic_msn]

SBC is yahoo's DSL service Yahoo won't be doing it. must be something else.

[chachi]

Even with that stuff turned on I am sure the request will still get through to the web server...they will just come back 403 Forbidden or something. I have had people hotlink images on my site from MySpace...that will bring your webserver to its knees quickly. :)

I am curious what the IP addresses of the Yahoo bots are. You can check them at ARIN to see if they are in fact registered to Yahoo!

[jim]

This is exactly what the host said. Hotlink protection is working but there are still os many requests. Their exact words: While the hotlink protection does stop some of the bandwidth usage, there is still some used when a request is made to the server for the various response headers sent back to the requesting client. Because of the sheer number of requests coming in, it builds up rather quick.

What did you do Chachi? Do I need a dedicated server or will Yahoo still bring me down?

Regarding the IPs, they are mostly in the 65... to 75... range. A few of them are
71.166.24.141
70.106.165.99
71.132.130.218
76.211.180.213
75.51.93.104
69.213.142.254
75.40.166.47
69.86.69.143
75.0.146.48

There are tons more of course... What is ARIN and how do i use it?

Even with that stuff turned on I am sure the request will still get through to the web server...they will just come back 403 Forbidden or something. I have had people hotlink images on my site from MySpace...that will bring your webserver to its knees quickly. :)

I am curious what the IP addresses of the Yahoo bots are. You can check them at ARIN to see if they are in fact registered to Yahoo!

[chachi]

ARIN is in control of who gets assigned what IP addresses. You can check who owns what IP address. I checked a bunch of those and they come back to Verizon and SBC ownership. Why do you think it is a Yahoo bot that is killing you?

If the useragent is Yahoo's Slurp, it sounds like someone is simply spoofing the useragent (very easy to do) and launching a small attack on your site for some reason.

[jim]

I don't think it's a bot at all. I think I got a top 5 ranking in My Yahoo for a very popular keyword and now whenever hundreds of thousands of people go to their My Yahoo they try to load up my favicon.ico.

It looks like I'll need a dedicated server to handle this extra traffic. I'm shopping now but I've always done shared before so this is very new territory.

[hans]

a new territory, an exciting territory and a challenging territory
but above all a rewarding territory - to have your own root server with full direct control of all

if you have been using your own linux machine locally for the past years - it is very little new to remotely run a root server as well.

i did the same move after 9 years of shared hosting a few months back and now i regret having waited a few years too much. all is far up, traffic, google adsense, server security, control, comfort of having a full professional support within seconds 24/7/365 and the pleasure to have access to absolutely all features of your machine and installed OS is far more than worth the step up to a root server. I am glad I never even considered having a managed dedicated server but went directly to root server!

each site has a size when it will be time to step up and get full control of all on your own.

[TechEvangelist]

Yahoo shouldn't request a favicon. I believe it is the browsers that request the favicon image.

Something is not right. Could it be a denial of service attack focusing on the favicon image? If so. just block all of the IPs making the requests.

[psyberweb]

I believe its from a client, i checked my recent logs and its from a client requesting the file..I dont know the reason for this but there must be some kind of browser that is doing this. Do tell me the result of activating your hotlink..

[NetworkTown.Net]

Use robot.txt ;)