Analyze These Robots.txt which one is better?

Discussion in 'robots.txt' started by e-marketing guru, Jun 23, 2008.

0
  1. #1
    Robots A:

    # Alexa
    User-agent: ia_archiver
    Disallow:

    # Ask Jeeves
    User-agent: Teoma
    Disallow:

    # Google
    User-agent: googlebot
    Disallow:

    # MSN
    User-agent: msnbot
    Disallow:

    # Yahoo!
    User-agent: Slurp
    Disallow:

    # Abacho
    User-agent: AbachoBOT
    Disallow:

    # Baidu
    User-agent: baiduspider
    Disallow:

    # Fireball
    User-agent: fireball
    Disallow:

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:

    # Szukacz
    User-agent: szukacz
    Disallow:

    # Voila.fr
    User-agent: VoilaBot
    Disallow:

    # Walhello
    User-agent: Appie
    Disallow:

    # Yandex
    User-agent: Yandex
    Disallow:

    # Others
    User-agent: *
    Disallow:

    Robots B:

    # Alexa
    User-agent: ia_archiver
    Disallow:8h4ft/

    # Ask Jeeves
    User-agent: Teoma
    Disallow:8h4ft/

    # Google
    User-agent: googlebot
    Disallow:8h4ft/

    # MSN
    User-agent: msnbot
    Disallow:8h4ft/

    # Yahoo!
    User-agent: Slurp
    Disallow:8h4ft/

    # Abacho
    User-agent: AbachoBOT
    Disallow:8h4ft/

    # Baidu
    User-agent: baiduspider
    Disallow:8h4ft/

    # Fireball
    User-agent: fireball
    Disallow:8h4ft/

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:8h4ft/

    # Szukacz
    User-agent: szukacz
    Disallow:8h4ft/

    # Voila.fr
    User-agent: VoilaBot
    Disallow:8h4ft/

    # Walhello
    User-agent: Appie
    Disallow:8h4ft/

    # Yandex
    User-agent: Yandex
    Disallow:8h4ft/

    # Others
    User-agent: *
    Disallow:8h4ft/
     
    e-marketing guru, Jun 23, 2008 IP
  2. TheSyndicate

    TheSyndicate Prominent Member

    Messages:
    5,410
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    365
    #2
    what does this mean Disallow:8h4ft/ ?
     
    TheSyndicate, Dec 21, 2011 IP