1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Analyze These Robots.txt which one is better?

Discussion in 'robots.txt' started by e-marketing guru, Jun 23, 2008.

0
  1. #1
    Robots A:

    # Alexa
    User-agent: ia_archiver
    Disallow:

    # Ask Jeeves
    User-agent: Teoma
    Disallow:

    # Google
    User-agent: googlebot
    Disallow:

    # MSN
    User-agent: msnbot
    Disallow:

    # Yahoo!
    User-agent: Slurp
    Disallow:

    # Abacho
    User-agent: AbachoBOT
    Disallow:

    # Baidu
    User-agent: baiduspider
    Disallow:

    # Fireball
    User-agent: fireball
    Disallow:

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:

    # Szukacz
    User-agent: szukacz
    Disallow:

    # Voila.fr
    User-agent: VoilaBot
    Disallow:

    # Walhello
    User-agent: Appie
    Disallow:

    # Yandex
    User-agent: Yandex
    Disallow:

    # Others
    User-agent: *
    Disallow:

    Robots B:

    # Alexa
    User-agent: ia_archiver
    Disallow:8h4ft/

    # Ask Jeeves
    User-agent: Teoma
    Disallow:8h4ft/

    # Google
    User-agent: googlebot
    Disallow:8h4ft/

    # MSN
    User-agent: msnbot
    Disallow:8h4ft/

    # Yahoo!
    User-agent: Slurp
    Disallow:8h4ft/

    # Abacho
    User-agent: AbachoBOT
    Disallow:8h4ft/

    # Baidu
    User-agent: baiduspider
    Disallow:8h4ft/

    # Fireball
    User-agent: fireball
    Disallow:8h4ft/

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:8h4ft/

    # Szukacz
    User-agent: szukacz
    Disallow:8h4ft/

    # Voila.fr
    User-agent: VoilaBot
    Disallow:8h4ft/

    # Walhello
    User-agent: Appie
    Disallow:8h4ft/

    # Yandex
    User-agent: Yandex
    Disallow:8h4ft/

    # Others
    User-agent: *
    Disallow:8h4ft/
     
    e-marketing guru, Jun 23, 2008 IP
  2. TheSyndicate

    TheSyndicate Prominent Member

    Messages:
    5,410
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    365
    #2
    what does this mean Disallow:8h4ft/ ?
     
    TheSyndicate, Dec 21, 2011 IP