Hello On my site, I have a printable version of each article at: mysite.com/articlename/print To get these print views not viewed by search engines (to prevent duplicate content penalty), my robots.txt file looks like this: User-agent: * Disallow:/*/print/ Also, I'm thinking of adding my admin directory to my robots.txt file, and i had a question. Couldn't people who want to hack your site find all the important directories by just looking at the robots.txt file? Should I add my admin directory to the robots file or not?
You will see the files in your ftp because your logged in and have privilidgesm however if someone without tries to browse they won't/ shouldn't see anything
i dont think that people can use use your print urls to hack. google and big the other big boys are suggesting to use robots.txt
User-agent: * Disallow:/*/print/ I think this will not work To disallow the print directory and it's subdirectory you need to put like User-agent: * Disallow: /print/ That's it.
User-agent: * Disallow:/*/print/ I think this will not work To disallow the print directory and it's subdirectory you need to put like User-agent: * Disallow: /print/ Ah, but does this work for /private/print too? As I think that's what he needs ( I might be wrong though )
If print is into a root directory than he has to give /print/ only he need not to give /private/print/