View Full Version : how to install a robots.txt for a wordpress blog ?
pradeep.balua
Jan 22nd 2008, 1:47 pm
I wanted to add an robots.txt file to my wordpress blog http://balumusings.com/
Is it possible to write a robots.txt file to prevent search engines from indexing specific ,not all, posts in my blog and how it would look like ?
thank you
your help will be greatly appreciated
myp
Jan 22nd 2008, 7:33 pm
Yes it is possible and it would look something like this: User-agent: *
Disallow: /10/tips-to-popularise-your-blog/
Disallow: /date/2007/11/
or whatever you want. If you just simply put disallow: / it will keep it from indexing all pages. If you want specific pages you can put it like i did in the first disallow or if you want to keep the bots from accessing archive pages (or category pages) it would look something like the second disallow I put in the example.
pradeep.balua
Jan 23rd 2008, 8:44 am
is there any plugin available for it ?
Valve-Hosting
Jan 23rd 2008, 8:49 am
For wordpress, yes.
http://wordpress.org/extend/plugins/google-sitemap-generator/
pradeep.balua
Jan 23rd 2008, 9:33 pm
k . thanks a lot I will check it out ...
Dan Schulz
Feb 8th 2008, 9:08 pm
Valve-Hosting, was this the link you meant to type? http://adambrown.info/b/widgets/kb-robots-txt/
vBulletin® v3.6.8, Copyright ©2000-2008, Jelsoft Enterprises Ltd.