Query about robots.txt function in wordpress?

Discussion in 'robots.txt' started by aniket123, Feb 1, 2012.

0
  1. #1
    Hello,

    I have a wordpress theme site. I want tohave disallow a post which appear on home page. I have declare on robots.txt disallow the post link. but it appear on sitemap.xml. So, is the crawler skip this page index or follow??

    --
    Thanks & Regards,
     
    aniket123, Feb 1, 2012 IP
  2. Richie_Ni

    Richie_Ni Illustrious Member

    Messages:
    10,721
    Likes Received:
    1,175
    Best Answers:
    0
    Trophy Points:
    410
    #2
    Richie_Ni, Feb 1, 2012 IP
  3. webgnomes

    webgnomes Peon

    Messages:
    7
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Just to take this a step further, here's a link to Yoast's wordpress plugin: http://yoast.com/wordpress/seo/

    Install it, and you'll be able to set the robots meta tags for each individual post on your blog. It will also automatically update your XML Sitemap for you.
     
    webgnomes, Feb 6, 2012 IP
  4. tiffanywilliams12i2

    tiffanywilliams12i2 Peon

    Messages:
    164
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    yoast is a nce plugin
     
    tiffanywilliams12i2, Apr 10, 2012 IP