Hey, I want to disallow everything in a directory except my /images/ directory. I am new with robots.txt syntax. adbox
Hi Adbox, You want to block all the pages in a directory except few pages, correct? if this is what you want to do, then here is the simple solution to do it. You can simply do it using following robots.txt: user-agent: * allow: /directory/page.html disallow: /directory/ if you have any further query, feel free to let me know.
No, you shouldn't put robots.txt into a sub directory, it should always placed in web root. In case of subdomains, you can put the robots.txt in their root folder.