Please review my robots.txt

Discussion in 'robots.txt' started by jameskertley, Jun 4, 2008.

0
  1. #1
    Dear can you please tell me what changes i have to made in my robots.txt file for good......

    please review this

    http://www.foundok.com/robots.txt


    Thanks in advance
     
    jameskertley, Jun 4, 2008 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    Check your inbox.....:)
     
    manish.chauhan, Jun 5, 2008 IP
  3. Skullborg

    Skullborg Guest

    Messages:
    757
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    0
    #3
    You need to add Disallow: /cgi-bin
    So that nobody can access your cgi folder.. if you leave your cgi folder open, Your site will most likely get hacked.
     
    Skullborg, Jun 20, 2008 IP
  4. C.Whyte

    C.Whyte Peon

    Messages:
    802
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    0
    #4
    That's a pretty short robots.txt lol! I think you might be missing some important disallows
     
    C.Whyte, Jun 20, 2008 IP
  5. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #5
    It's fine.
     
    e-marketing guru, Jun 23, 2008 IP