block certain pages

Discussion in 'Site & Server Administration' started by choice, Oct 27, 2006.

  1. #1
    is there a way to stop certain pages being indexed by search engines
     
    choice, Oct 27, 2006 IP
  2. mad4

    mad4 Peon

    Messages:
    6,986
    Likes Received:
    493
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Yes. Use robots.txt and the noindex tag.
     
    mad4, Oct 27, 2006 IP
  3. choice

    choice Prominent Member

    Messages:
    5,444
    Likes Received:
    490
    Best Answers:
    0
    Trophy Points:
    350
    Digital Goods:
    2
    #3
    is ther a program i can use to create a robot file

    as there is about 300 pages i do not want indexing
     
    choice, Oct 27, 2006 IP
  4. mad4

    mad4 Peon

    Messages:
    6,986
    Likes Received:
    493
    Best Answers:
    0
    Trophy Points:
    0
  5. Estevan

    Estevan Peon

    Messages:
    120
    Likes Received:
    8
    Best Answers:
    1
    Trophy Points:
    0
    #5
    caution robot.txt are a plain txt anyone read this !
     
    Estevan, Oct 28, 2006 IP
  6. Monty

    Monty Peon

    Messages:
    1,363
    Likes Received:
    132
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Monty, Oct 28, 2006 IP
  7. Estevan

    Estevan Peon

    Messages:
    120
    Likes Received:
    8
    Best Answers:
    1
    Trophy Points:
    0
    #7
    if you have a time please look your server logs and look how many times no robots request your robot.txt

    if no like engine index this only no make a link , but google toolbar sent all pages visited from you to google !
    if is a dir use .htaccess
     
    Estevan, Oct 28, 2006 IP