I use this robots.txt file in all my blogs, is it correct?

Discussion in 'robots.txt' started by domainloco.net, Feb 18, 2009.

  1. #1
    Not sure if I'm missing something or have something extra in there. Anyone care to revise please?


    User-agent: *
    Disallow: /wp-content/
    Disallow: /wp-includes/
    Disallow: /wp-admin/
    Disallow: /wp-login.php
    Disallow: /wp-register.php
    Disallow: /xmlrpc.php
    Code (markup):
    Dan
     
    domainloco.net, Feb 18, 2009 IP
  2. shailendra

    shailendra Peon

    Messages:
    1,225
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    0
    #2
    yes it is absolutely correct if you are trying to block all these things from getting indexed
     
    shailendra, Feb 18, 2009 IP
  3. domainloco.net

    domainloco.net Notable Member

    Messages:
    3,085
    Likes Received:
    59
    Best Answers:
    1
    Trophy Points:
    225
    #3
    Thanks, I was just making sure I wasn't missing anything.
     
    domainloco.net, Feb 18, 2009 IP
  4. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #4
    If you are using blogs, you can also block tags, category and author area like:

    User-agent: *
    Disallow: /wp-content/
    Disallow: /wp-includes/
    Disallow: /wp-admin/
    Disallow: /wp-login.php
    Disallow: /wp-register.php
    Disallow: /xmlrpc.php
    Disallow: /tag/
    Disallow: /author/
    ...........
     
    manish.chauhan, Feb 22, 2009 IP
  5. domainloco.net

    domainloco.net Notable Member

    Messages:
    3,085
    Likes Received:
    59
    Best Answers:
    1
    Trophy Points:
    225
    #5
    Thanks, I didnt include it in this file because the all in one seo plugin ads it to the pages for me.

     
    domainloco.net, Feb 23, 2009 IP