Hi folks, This week I made the transfer from blogger to wordpress. I'm glad I finally did, but there was alot more work involved than I previously thought. I have my permalink structure setup as /%category%/%postname%/ I read that in my robots.txt file I should have "Disallow: /category/" I have not included that in my robots because it seems like that wouldn't allow access to my posts. If the category is in the url before my postname, wouldn't that mean the dissallow wouldn't allow access for search engines to that? I noticed google is indexing both http://mywebsite.com/category(whatever the category is)/name-of-posted-article, and also http://mywebsite.com/category(literally just the word category)/categoryname. What should I do to maintain spider access to my posts, but not to the direct category area? One last thing. I'm also using the all in one SEO pack. I have selected the feature for it to add meta keywords to each post by it making the category names the meta keywords. For each new post do I really need to tag my posts too if I'm already selecting the category names? After posting the pages contain specific meta keywords from the categories I chose, so should I just leave tagging out if I'm already using the All in one SEO pack? I hope you guys can help, Thanks, Mike
Well you could simply give a noindex tag to categories in your header.php file, it should look something like this <?php if(is_archive()) { ?><meta name="robots" content="noindex" /><?php } ?> or you could simply install all in one seo pack which has these features to noindex categories, archives or even tags.
So if I put "Disallow: /category/" in my robots.txt file that would still keep search engines spidering my posts, despite their permalink structure having the category name in them?