MPAA - Flights - Babb Fest - Easy movies download - Remortgages
PDA

View Full Version : AskJeeves going nuts?

darqSHADOW
Nov 25th 2004, 8:56 am
Has anyone else noticed that AskJeeves is doing a deep crawl? I've had over 50k hits this month from them, where-as I normally only recieve a few hundred. They've also managed to eat over 10GB of bandwidth in that time.

DS
darksat
Nov 25th 2004, 9:00 am
How many pages have they crawled?
Thewormman
Nov 25th 2004, 9:33 am
Has anyone else noticed that AskJeeves is doing a deep crawl? I've had over 50k hits this month from them, where-as I normally only recieve a few hundred. They've also managed to eat over 10GB of bandwidth in that time.

DS

Yep I have had 130,000 pages of a new site crawled in last week or two, a constant 5 pages per minute

Only seen them take 1 page at a time before :eek:

Go figure??
ResearchTechs
Dec 4th 2004, 7:52 am
I had 30+ sites hit in my first week online, but my site name doesn't even show up on their search? (Check it for ResearchTechs). Any idea what's going on there?
melfan
Dec 4th 2004, 10:00 am
Same experience on one of my content sites.
Thewormman
Dec 4th 2004, 10:33 pm
It is still on my site having had 225000 pages and I can't find more than the index page on their search.
If it doesn't show soon I think there will be a new addition to my robots txt file, I don't need them to use all my bandwidth. :mad: And give nothing in return.