What is a robot.txt file exactly?

Discussion in 'Blogging' started by deannatroupe, Sep 19, 2010.

  1. #1
    What is a robots.txt file exactly? What does that have to do with my blog's SEO ranking?
     
    deannatroupe, Sep 19, 2010 IP
  2. gift-online

    gift-online Well-Known Member

    Messages:
    2,359
    Likes Received:
    63
    Best Answers:
    0
    Trophy Points:
    115
    #2
    you can allow/disallow access from serach engines to pages of you site in robot.txt
     
    gift-online, Sep 21, 2010 IP
  3. Kerosene

    Kerosene Alpha & Omega™ Staff

    Messages:
    11,366
    Likes Received:
    575
    Best Answers:
    4
    Trophy Points:
    385
    #3
    It basically tells spiders which pages you want them to avoid crawling.

    You don't necessarily need a robots.txt file. And you can do a lot of damage if you put in a bad entry.
    Unless you have a specific reason to block spiders from parts of your site, then you don't need to worry about robots.txt too much.

    Worth reading: http://en.wikipedia.org/wiki/Robots_exclusion_standard
     
    Kerosene, Sep 21, 2010 IP
  4. Linkadoodle

    Linkadoodle Guest

    Messages:
    51
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Id stay well away from it just incase you dont google to stop crawling your sight because altered this file. As usual Wiki is prob your best resource for a quick eplanation.
     
    Linkadoodle, Sep 23, 2010 IP