I have a robots.txt file in a sub directory with the following contents: User-agent: * Disallow: / However, google still indexed the subdirectory! any help? Example: "http://www.website.com/subdirectorythatshouldn'tbeindexed"
In general, this should not happen. If this is still happening, you can remove the subdirectory from Google index forcefully by using Google webmaster tool