I just saw 190 links restricted by robots.txt out of which 180 are my feed links. How do I fix it? I use wordpress My KB robots page shows this- Disallow: */feed
Just remove that line? You probably have an SEO plugin that is causing this. There should be a setting to turn this option on/off.
I'm not sure what you're trying to say. Yes, feeds and comments can be indexed by Google, as long as they are allowed by your robots.txt. Google won't ban you for having comments or feeds.
This is what I can see inside the plugin- User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /trackback Disallow: /feed Disallow: /comments Disallow: /category/*/* Disallow: */trackback Disallow: */feed Disallow: */comments Disallow: /*?* Disallow: /*? Allow: /wp-content/uploads # Google Image User-agent: Googlebot-Image Disallow: Allow: /* # Google AdSense User-agent: Mediapartners-Google* Disallow: Allow: /* # Internet Archiver Wayback Machine User-agent: ia_archiver Disallow: / # digg mirror User-agent: duggmirror Disallow: / So do I remove these 2 lines? Disallow: /feed Disallow: /comments
Did you add these settings manually? As I said earlier, you probably have a plugin (not KB Robots.txt) that is causing this. When you find the plugin, there should be settings to turn these options on/off.
I am using a plugin called Robots.txt Editor, I never changed anything. I don't see any setting to turn it off
Log into WordPress, and click Plugins. Make a list of all of the plugins that you have, and paste them here. Did you add the contents of robots.txt yourself? (did you copy it from somewhere else, and paste it into KB Robots.txt)
In the plugin list it says- KB Robots.txt Manage your robots.txt file from within Wordpress. Especially useful for WP-MU installations. Deactivate | Edit Version 1.0.1 | By Adam R. Brown | Visit plugin site But when I click on it it shows- Robots.txt Editor