How to generate robots.txt file for MSN?

Discussion in 'Bing' started by jazznchew, Feb 19, 2008.

0
  1. #1
    If we r submit our site to MSN, we required to validate the robots.txt file.
    How? where to get the tool for generation?

    Thanks in advance,
     
    jazznchew, Feb 19, 2008 IP
  2. marshalseo

    marshalseo Peon

    Messages:
    252
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #2
    robots.txt file is common for all search engine. If you want to use specially for msn then you should mention the name of msnbot in useragent row

    for example

    User-Agent: msnbot
    User-Agent: msnbot-products
    User-Agent: msnbot-news
    Disallow: /filename.ext

    etc.
    to know about names of crawlers and bots visit http://expertseogroup.blogspot.com/2007/03/crawler-spiders-and-robots-name-of.html

    or you want to know more about how to create a robots.txt visit
    http://expertseogroup.blogspot.com/2007/02/protect-your-website-using-robotstxt-or.html
     
    marshalseo, Feb 19, 2008 IP
  3. Chuchuzoo

    Chuchuzoo Banned

    Messages:
    30
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yea he's accurate in what he's saying.
     
    Chuchuzoo, Feb 20, 2008 IP