i understand that the robots.txt file must be placed in the root directory, i.e. www.yourdomain.com/robots.txt and won't work if u place it in a sub directory, i.e. www.yourdomain.com/subdirectory/robots.txt ------------------------------------------------ But I want my site to be indexed, i just have a sub directory that i use to host verious unrelated files that i do not want indexed. How can I accomplish this? thanks
You put the robots.txt file in your root directory. Then specify within that file which pages/folders you don't want indexed. *EDIT* Looks like you already figured it out. Glad to hear that.