![]() |
|
|
#1
|
|||
|
|||
|
I've been checking through my logs etc and noticed the Yahoo spider has been accessing some urls that either don't exist, or have been written with mod_rewrite. The last one I found was particularly strange as it uses the .html extension to a page that doesn't exist - if I rewrite the url or make a normal static url, I either end them in .php or .htm
It's also tried to access my main url but added PREMIAIRSESSID/5bc9f0744126394912fb7/p and /h/9198rgd.html onto the end of it! The weirdest one is /index.phtml?command=further-details&resultRow=2004-08-04|JMC||BRS|BRISTOL|KGS|Kos/Kalymnos|Self+Catering|7|25 Actually, as I write this I notice even more! /portal/templates/ms_daydream2blue/css/template_css.css /resources/books/index.asp /PCL-740-rackmount-computer-8011.html ...etc.! Any ideas what's going on here? In some ways it's as if the spider can read the non-static urls rather than the rewritten ones - I didn't think that was possible...is it? It's a brand new site so there's no old pages linking to these non-existant urls etc if that helps! It's from the ip range 66.196.91.xxx which I presume is the Slurp range, but I've not double checked.
|
|
#2
|
|||
|
|||
|
Have you just registered the domain name??
If you have, then it's possible that it's been used in the past, and Yahoo is crawling old links.. Have you checked on www.archive.org ?? |
|
#3
|
|||
|
|||
|
Thanks for the reply and suggestion - I've just checked that site now but it's not come up with any results for my new domain!
![]() Any other possibilities? Has no-one else noticed this on their sites? |
![]() |
| Bookmarks |
| Thread Tools | |
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Yahoo Directory Adding NOT Yahoo Web | misohoni | Directories | 3 | Apr 21st 2009 12:24 am |
| Yahoo US shopping - joke of the year? | expat | Yahoo | 12 | Dec 19th 2007 3:40 pm |
| Yahoo Search Using Google Again? | digitalpoint | Yahoo | 34 | Apr 5th 2005 9:05 am |
| Yahoo WebRank | prof | Yahoo | 14 | Dec 10th 2004 6:04 pm |
| Do you Yahoo? Do you seig heil? | Owlcroft | General Chat | 9 | Aug 24th 2004 7:59 pm |