I was trying to fix duplicate title tag using robot.txt. I logged to GWT, then to block Url to use the robot.txt to block one of the Url but on the page there is this ''Check to see that your robots.txt is working as expected. (Any changes you make to the robots.txt content below will not be saved.)'' As much as I tried to edit the context to block the unwanted Url it would not be saved. Pls what is it that I was I am doing wrong? Is there another way to go about this? thanks
If your site is using Wordpress, there are some SEO plugins that will take care of setting things like archives etc to noindex, nofollow. Plugins like All in one SEO and Yoast SEO. Do a search for wordpress duplicate noindex plugin and you will find plugins that take care of this as well. If you aren't using Wordpress then I suggest you start with a bare robots.txt and do something obvious like add a disallow block for archives and see if it works then add on slowly from there. Look at guides specific to robots.txt for duplicate content like this one http://www.curvve.com/blog/guides/2...-an-seo-problem-with-robots-txt/#.Ui6Z-n8_VFI