Yahoo spiders hungry

[Fishing Forum]

Yahoo spiders hungry - Well its day four and I still have 180 plus Yahoo spiders on my site its been sold for four days

Is it just me or is Yahoo spiders hungry as hell . THey are always there but not in such numbers and for so long

Is this a sign of something happening ? as the last time it happened like this ( about 6 mouths ago ) and I got in yahoo with good listings and site: index

Newish site then with only one paged indexed

Just had a quick look at DPs online members and 180 ish are Yahoo which is 99% of spiders at this moment in time

Or should I not be having beer on my cornflakes :p

[ephricon]

I have noticed a larger than normal amt. of traffic from their spiders lately. Also I've seen a nice increase in number of pages indexed over a previously stagnant number...

[noppid]

Yahoo ran out my bandwidth over night. hehe

The presense on the bot has been like 10 fold the norm over the past about 4 days.

[dct]

I've been slurped rotten continually for the past few days. I'm not getting my hopes up as it also happened a month or so ago with no real effect :(

[noppid]

I've been slurped rotten continually for the past few days. I'm not getting my hopes up as it also happened a month or so ago with no real effect :(


Agreed, they don't show results that seem to match their thirst to crawl.

If it's so interesting that you stay and go deep so often, please help me pay the bandwidth bill by listing more pages to pay for it!

[dct]

My last 2 months slurping are:
Inktomi Slurp 4060+773 33.06 MB
Inktomi Slurp 6637+636 63.22 MB
Pages in URL is 89 now WTF is that all about, but the pages they have indexed rank really well, stupid search engine.

[Ian]

My last 2 months slurping are:
Inktomi Slurp 4060+773 33.06 MB
Inktomi Slurp 6637+636 63.22 MB
Pages in URL is 89 now WTF is that all about, but the pages they have indexed rank really well, stupid search engine.


Is this them: 65.54.188.81 ?

I've had hundreds of page requests from this I.P. so i'm curious if anyone can tell if this is them.

Ian

[dct]

My stats were from what AWStats reported, which I think is done on user agent (or something like that)

A Whois on the IP 65.54.188.81 comes out as "Microsoft Corp" so likely to be the MSN bot.

[Ian]

My stats were from what AWStats reported, which I think is done on user agent (or something like that)

A Whois on the IP 65.54.188.81 comes out as "Microsoft Corp" so likely to be the MSN bot.

Thanks DCT. I was trying to figure out who that was. They've been all over the site the past two days. Yahoo has just one page indexed from the site, and hasn't indexed the rest of it yet and it's been three weeks. Hopefully that changes soon.

[SportsOutlaw]

my site is getting hit heavily by yahoo spiders as well, and has been for the past 4-5 days. On top of that, google is also making an increased showing.

Maybe something positive will come of this.

I know I broke my "most users online" record about 50 times in the past couple of days.

[joeychgo]

Yahoo has been on all three of my forums for several days.

Hope they index a few of them.

[dsr771]

I saw some increase by Yahoo, but not a big amount.

[Fishing Forum]

Well glade its not just me

Lets hope something good comes out of this as Yahoo is still going for it

Any ideas why yahoo spiders so much but indexs so little as it seems a waste

[ephricon]

I finally actually looked at my spider activity and I've noticed a HUGE increase in Yahoo spider traffic. It started around 4 weeks ago and really picked up in the past week or two...

In my WebTrends it also appears to be two different Yahoo spiders. My info would lead one to believe that the spider my logs are showing as "Slurp" is being replaced by just "Yahoo", which seems to want to crawl much deeper.

Anyone heard anything about this? Maybe it is a new/improved spider or the equivalent of a freshbot/deepbot??? Whatever it is I've also noticed an equally sharp increase in my total pages indexed as well, so I guess this is a good thing :) Only a modest traffic increase though from Yahoo referrals thus far...

[Dominic]

I've noticed them too. Haven't looked at my stats yet, but they have been munching away on my forums for a few days.

[digidogstudios]

i had a server crash the other day from it being blasted by yahoo/msn/google/altavista, it was my error of basically creating a loop in my mod_rewrite coding, sucks