wordpress optimized robots.txt

Hi, many bloggers use wordperss as their primary platform, so it will be nice if someone will post wordpress specific optimized robots.txt

can u please share ur file and some knowledge how it works

thx.

+reps from me.

 

You´ll find a few tips here: http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html

 

Here's my wordpress blog's robots.txt:

User-agent: *
Disallow: /cgi-bin
Disallow: /includes
Disallow: /content/plugins
Disallow: /content/cache
Disallow: /content/themes
Disallow: /category
Disallow: /tag
Disallow: /author
Disallow: /trackback
Disallow: /*trackback
Disallow: /*trackback*
Disallow: /*/trackback
Disallow: /*feed*

# Internet Archiver Wayback Machine
User-agent: ia_archiver
Disallow: /

# disable duggmirror
User-agent: duggmirror
Disallow: /

Code (markup):

 

looks good, why do u disable categorys and tags? they are good for traffic?

 

Categories/tags are duplicate content - so there's no need. I only want the articles themselves, and the archives included in the index. For some reason though google still insists on indexing my tags/categories. Not sure why...

 

which directory do you place your robot.txt into.

I put mine in the root directory

www.example.com/robot.txt

and i dont think it works for me

 

You are missing one s at the end of robot:

www.example.com/robot[B]s[/B].txt