Hi, I have seen recently robots.txt for my site & one from my blog (Blogger custom domain) is appearing in SERPs (snapshots attached). I have no idea how did this happen. I have never placed the robots.txt link to any forum or other sites. Whereas I often use these free online website analysis tools like websitegrader.com/woorank.com which may checks robots file while analyzing. I am not sure, but I hope using url removal tool in Google Webmaster Tools won't be a good idea. What I can do to fix this? Do I really need to care about this? Thanks, Jim
It is very weird and first time I see it, maybe you have somehow gotten link to it. Perhaps you can use robots.txt and block it. Anyway, don't see a way how it could harm the website.
Actually when I thought about it - and found it rather funny - what happens when you block robots.txt - I mean then the robots cannot go there to see what you have blocked, so does it create some sort of loop or something. Maybe some link partner of yours got mad that you are using robots.txt to block link exchange page and linked to your robots.txt so anyone who would exchange links with you, would see it.
Yes, i can see the same for my site,but disallow folder or url has not been indexed in SERP. So, no need to worry. You can also see this one http://forums.digitalpoint.com/robots.txt at SERP.
It seems strange that the Robots.txt file is coming in the search results. Try deleting the old one and uploading again. But even if it is coming in the results, there is no need to worry.
Where have you placed the robots.txt file?At any rate the robots.txt file should not be showing in the search results, the browsers are intelligent enough to know that like the search engines.