Robots.txt validation problem in MSN webmaster

Discussion in 'Search Engine Optimization' started by e-marketing guru, Jun 23, 2008.

0
  1. #1
    First my Robbots.txt is

    # Alexa
    User-agent: ia_archiver
    Disallow:

    # Ask Jeeves
    User-agent: Teoma
    Disallow:

    # Google
    User-agent: googlebot
    Disallow:

    # MSN
    User-agent: msnbot
    Disallow:

    # Yahoo!
    User-agent: Slurp
    Disallow:

    # Abacho
    User-agent: AbachoBOT
    Disallow:

    # Baidu
    User-agent: baiduspider
    Disallow:

    # Fireball
    User-agent: fireball
    Disallow:

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:

    # Szukacz
    User-agent: szukacz
    Disallow:

    # Voila.fr
    User-agent: VoilaBot
    Disallow:

    # Walhello
    User-agent: Appie
    Disallow:

    # Yandex
    User-agent: Yandex
    Disallow:

    # Others
    User-agent: *
    Disallow:

    Validations Results:

    Line #55: Disallow:
    Error: Conflicting disallow tags found.
    **************************************************
    Warning: 'sitemap' - tag isn't specified.
    **************************************************

    Could you please tell me how to solve it.I will be very thankful to you all.:(
     
    e-marketing guru, Jun 23, 2008 IP
  2. munnakv

    munnakv Active Member

    Messages:
    293
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    60
    #2
    Those lines gives direction to "all" spiders to crawl everwhere in your site.
    Then why single out specific spiders to crawl everywhere in your previous lines?


    Syntax is: Sitemap: http://yourwebsite.com/sitemap.xml
    You have to give a valid reference to a valid site map. I guess this is optional.

    BTW, there is a separate sub-forum for robots.txt discussion:
    http://forums.digitalpoint.com/forumdisplay.php?f=50
     
    munnakv, Jun 23, 2008 IP
  3. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #3
    Thanks For quick reply.
     
    e-marketing guru, Jun 23, 2008 IP
  4. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #4
    Problem has solved,
    # Alexa
    User-agent: ia_archiver
    Disallow:

    # Ask Jeeves
    User-agent: Teoma
    Disallow:

    # Google
    User-agent: googlebot
    Disallow:

    # MSN
    User-agent: msnbot
    Disallow:

    # Yahoo!
    User-agent: Slurp
    Disallow:

    # Abacho
    User-agent: AbachoBOT
    Disallow:

    # Baidu
    User-agent: baiduspider
    Disallow:

    # Fireball
    User-agent: fireball
    Disallow:

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:

    # Szukacz
    User-agent: szukacz
    Disallow:

    # Voila.fr
    User-agent: VoilaBot
    Disallow:

    # Walhello
    User-agent: Appie
    Disallow:

    # Yandex
    User-agent: Yandex
    Disallow:

    # Others
    User-agent: *
    Sitemap: http://www.freebiesarchive.com/sitemap.xml

    Validation Results:
    No Problem found
     
    e-marketing guru, Jun 23, 2008 IP
  5. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #5
    Problem has solved,
    # Alexa
    User-agent: ia_archiver
    Disallow:

    # Ask Jeeves
    User-agent: Teoma
    Disallow:

    # Google
    User-agent: googlebot
    Disallow:

    # MSN
    User-agent: msnbot
    Disallow:

    # Yahoo!
    User-agent: Slurp
    Disallow:

    # Abacho
    User-agent: AbachoBOT
    Disallow:

    # Baidu
    User-agent: baiduspider
    Disallow:

    # Fireball
    User-agent: fireball
    Disallow:

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:

    # Szukacz
    User-agent: szukacz
    Disallow:

    # Voila.fr
    User-agent: VoilaBot
    Disallow:

    # Walhello
    User-agent: Appie
    Disallow:

    # Yandex
    User-agent: Yandex
    Disallow:

    # Others
    User-agent: *
    Sitemap: http://www.freebiesarchive.com/sitemap.xml
    Sitemap: http://www.freebiesarchive.com/sitemap.xml.gz

    Validation Results:
    No Problem found[/QUOTE]
     
    e-marketing guru, Jun 30, 2008 IP