Digital Point Forums
westernunion

Go Back   Digital Point Forums > Search Engines > Yahoo
Google Analytics
Log In to view
your analytics

Reply
 
Thread Tools
  #1  
Old Oct 27th 2004, 7:39 am
aleco aleco is offline
Grunt
 
Join Date: Mar 2004
Location: UK
Posts: 51
aleco is on a distinguished road
Question Odd Yahoo crawling?

I've been checking through my logs etc and noticed the Yahoo spider has been accessing some urls that either don't exist, or have been written with mod_rewrite. The last one I found was particularly strange as it uses the .html extension to a page that doesn't exist - if I rewrite the url or make a normal static url, I either end them in .php or .htm

It's also tried to access my main url but added PREMIAIRSESSID/5bc9f0744126394912fb7/p and /h/9198rgd.html onto the end of it!

The weirdest one is /index.phtml?command=further-details&resultRow=2004-08-04|JMC||BRS|BRISTOL|KGS|Kos/Kalymnos|Self+Catering|7|25

Actually, as I write this I notice even more!
/portal/templates/ms_daydream2blue/css/template_css.css
/resources/books/index.asp
/PCL-740-rackmount-computer-8011.html
...etc.!

Any ideas what's going on here? In some ways it's as if the spider can read the non-static urls rather than the rewritten ones - I didn't think that was possible...is it? It's a brand new site so there's no old pages linking to these non-existant urls etc if that helps!

It's from the ip range 66.196.91.xxx which I presume is the Slurp range, but I've not double checked.

Reply With Quote
  #2  
Old Oct 28th 2004, 12:34 am
Redleg Redleg is offline
Raider
 
Join Date: Jul 2004
Location: Norway
Posts: 360
Redleg is on a distinguished road
Have you just registered the domain name??

If you have, then it's possible that it's been used in the past, and Yahoo is crawling old links..

Have you checked on www.archive.org ??
__________________
| VoIP USA | Analyze and Geolocate your IP Address | |
Reply With Quote
  #3  
Old Oct 29th 2004, 4:41 am
aleco aleco is offline
Grunt
 
Join Date: Mar 2004
Location: UK
Posts: 51
aleco is on a distinguished road
Thanks for the reply and suggestion - I've just checked that site now but it's not come up with any results for my new domain!

Any other possibilities? Has no-one else noticed this on their sites?
Reply With Quote
Reply

Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Yahoo Directory Adding NOT Yahoo Web misohoni Directories 3 Apr 21st 2009 12:24 am
Yahoo US shopping - joke of the year? expat Yahoo 12 Dec 19th 2007 3:40 pm
Yahoo Search Using Google Again? digitalpoint Yahoo 34 Apr 5th 2005 9:05 am
Yahoo WebRank prof Yahoo 14 Dec 10th 2004 6:04 pm
Do you Yahoo? Do you seig heil? Owlcroft General Chat 9 Aug 24th 2004 7:59 pm


All times are GMT -8. The time now is 8:00 pm.