How to Restrict all the unnecessary stuff from being indexed.

Discussion in 'HTML & Website Design' started by 2point2, Nov 19, 2011.

  1. #1
    Hi
    I am tring to do what I can to get my ranking back in google. The below was one of the things suggested. Can anyone tell me how to do this or steer me in the direction to learn how to do this. I have a blog and use wordpress.

    "Restrict all the unnecessary stuff from being indexed. Do this by tweaking the Robots.txt file of your website.
    we suggest you to use the following code in it
    User-agent: *
    Disallow:/page/
    Disallow:/tag/
    Disallow:/category/
    Disallow:/wp-admin/
    Disallow:/wp-includes/
    Disallow:/wp-content/
    Disallow:/wp-*
    Disallow:/author/
    Disallow:/comments/feed/
     
    2point2, Nov 19, 2011 IP
  2. dcristo

    dcristo Illustrious Member

    Messages:
    19,776
    Likes Received:
    1,200
    Best Answers:
    7
    Trophy Points:
    470
    Articles:
    7
    #2
    I think it's a mistake to restrict spiders from indexing your category pages. The category pages are a fundamental part of your navigation structure.
     
    dcristo, Nov 19, 2011 IP