Hi All, I have some doubt in my robots txt file. I want to confirm only one thing. What is difference between both below syntax? User-agent: * Disallow: /images User-agent: * Disallow: /images/ What will happen if we placed (/) after images? Thanks, Rock
Correctly putting the / after /images ensures that no filenames starting with "images" would be excluded. For example, disallow: /images would exclude not only the directory /images/ but also files like /images_viewer.php etc should you have any such files.