What does this mean?

Discussion in 'robots.txt' started by Dragoon, Feb 14, 2010.

  1. #1
    User-agent: *
    Disallow:
    Code (markup):
    What does this mean? No following at all? Or does it mean nothing at all?
     
    Dragoon, Feb 14, 2010 IP
  2. jibin

    jibin Peon

    Messages:
    43
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    it means to allow all robots complete access
    :)
     
    jibin, Feb 16, 2010 IP
  3. rubeena

    rubeena Peon

    Messages:
    24
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    means all robots are instructed to not index any of your pages (indicated by "/").

    I believe

    Means there is no disallow and the robots can crawl the site.
     
    rubeena, Feb 16, 2010 IP
  4. rubeena

    rubeena Peon

    Messages:
    24
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I cant find any document to support my point of view. It is better you try it out yourself and find out.
     
    rubeena, Feb 16, 2010 IP
  5. jibin

    jibin Peon

    Messages:
    43
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5

    rubeena is right. this is the proof
    http://www.robotstxt.org/robotstxt.html
    :)
     
    jibin, Feb 16, 2010 IP
  6. Jackob2916

    Jackob2916 Peon

    Messages:
    552
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Great rubeena thanks a lot for sharing this information
     
    Jackob2916, Feb 17, 2010 IP
  7. bajay

    bajay Peon

    Messages:
    48
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    :D now I know what the meaning of it thanks for sharing, you've to know that yesterday my sister asked same quest about that, thanks once again.
     
    bajay, Feb 18, 2010 IP
  8. aty24x7

    aty24x7 Greenhorn

    Messages:
    40
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #8
    User-agent: *
    Disallow: /


    This is used inside robots.txt to disallow all search engines not to index your pages
     
    aty24x7, Feb 19, 2010 IP
  9. mforconi

    mforconi Member

    Messages:
    18
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    36
    #9
    hey man...why not just type robot.txt in Google?
    you'll obtain TONS of infos :)
     
    mforconi, Feb 20, 2010 IP