Need a bit of help here please.

Discussion in 'robots.txt' started by Kingdom85, Oct 5, 2006.

0
  1. #1
    I want to remove some pages from Google's index and i beleive using a robots.txt command is the way to do it (i've heard it is anyway)

    The pages in my site are all .php pages. is there a format i need to put the robots.txt code in to make sure it works?

    which code do i use?

    :confused:
     
    Kingdom85, Oct 5, 2006 IP
  2. MattUK

    MattUK Notable Member

    Messages:
    6,950
    Likes Received:
    377
    Best Answers:
    0
    Trophy Points:
    275
    #2
    Just,

    User-agent: Googlebot
    Disallow: /

    You can also block access to certain directories

    User-Agent: Googlebot
    Disallow: /folder1/
    Allow: /folder1/myfile.html
     
    MattUK, Oct 5, 2006 IP
  3. Kingdom85

    Kingdom85 Peon

    Messages:
    178
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    does that exclude it from just Google?

    is it possible to exclude it from all search engines?
     
    Kingdom85, Oct 5, 2006 IP
  4. MattUK

    MattUK Notable Member

    Messages:
    6,950
    Likes Received:
    377
    Best Answers:
    0
    Trophy Points:
    275
    #4
    Just Disallow: / should block everything
     
    MattUK, Oct 5, 2006 IP
  5. Kingdom85

    Kingdom85 Peon

    Messages:
    178
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    oh i see. think i get it now. cheers matt, thanks for your help
     
    Kingdom85, Oct 5, 2006 IP