Hi i've never used a Robots.txt, as i heard Wordpress has some issues i am looking into this matter. Do you think i should set up one and how would i go about doing this, what should i block? I'm thinking of maybe blocking my tags, how do i write this in robots.txt? Also anything else that i should add to my robots.txt?
Robots.txt are good, but why not try all in one seo pack plugin for WP? You can set rules from the admin itself. I believe this is a wonderful plugin. For writing robots.txt or seo for wordpress, you can check this thread at sitepoint, Wordpress SEO Checklist, where Dan Schulz explained it beautifully!
Yes i have All in One installed already, from the post it seems he advises to block tags and archives i think i'll set it up to do that for now. Many Thanks.