Hello friends, I just found out today that Search engines are indexing sub folder of a site. Its acutaly a subdomain. but now subfolder and subdomains are beging indexed now. How can I stop subfolder from being indexed? here is my site structure. http://dir.mysite.com and search engines have indexed more than 1000 pages for http://www.mysite.com/dir how should i stop it? Robots.txt or there is any other way?
Create a Robots.txt file in your root and put this in the txt file. User-agent: * Disallow: Disallow: /dir/ I also wrote a Robots.txt How-To a couple of hours ago on my site. http://www.talk-seo.com/2007/08/seo-tip-1-robotstxt/
Edit robot.txt files. This can help u. Do it fast ; bcoz if ur folders have some valuable docs this can help hackers!