Hello, I've noticed that google shows some of my urls, in a search query, that are dissabled by robots.txt. They don't show any description, but in fact they came up in the query. Anyone geting the same results? Nuno
In some URLs yes (but already about 6 months ago), but in other I had always a robots.txt disallowing indexing every file on that subdomain.
Google seems to put pages that it sees, but isn't allowed to spider. But they always seem to drop out of the serps after being there a day or two. At least that's been my experience... - Shawn
For example, look at this query: http://www.google.pt/search?hl=pt-PT&ie=UTF-8&oe=UTF-8&q=ip+aborla.net+-mysql&meta= That domain is disabled since the begining and it still returns the http://testes.aborla.net/ip.php URL. bad behaviour!...
I have experienced something similar to this, also brings up the question thats being asked about no follow still being followed by Google bot.
That was one hell of a bump. You probably should have just started a new thread rather than resurrecting a 3 year old one. The answer is still the same 3 years later though. Google will record the existence of URLs that have links pointing to them. If the only links that point to the page are nofollowed, or if the page in question is blocked by either robots.txt or a noindex meta directive, then the only thing that will show up in the serps is the url itself. Google does not cache the actual page. <sidenote> You can almost tell how old the damn thread is just by who posted in it. Shawn hasn't posted in this forum since last year. </sidenote> -Michael
can give more specific information pls ? if you urls like name.php?id=4566 , you must add name.php$ to your robots.txt . which urls are you talking about? i dont think google can make a mistake