Can someone check my robots.txt file

Discussion in 'robots.txt' started by st_jimi, Jul 2, 2007.

0
  1. #1
    Can someone check my robots txt file to make sure its constructed ok allowing all bots to index

    User-agent: *
    Disallow:
    Disallow: /cgi-bin/
    Disallow: /billing/admin
    Disallow: /support/admin
    Sitemap: http://www.serverdragon.com/sitemap.xml
     
    st_jimi, Jul 2, 2007 IP
  2. nusuni

    nusuni Peon

    Messages:
    129
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Looks alright, I don't really get the point of the blank disallow line though.
     
    nusuni, Jul 3, 2007 IP
  3. Costumer

    Costumer Peon

    Messages:
    50
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Your robot file is all right. Like nusuni said to you remove the blank disallow.
     
    Costumer, Jul 3, 2007 IP
  4. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #4
    i think everythink looks ok for your robots.txt file
     
    trichnosis, Aug 7, 2007 IP