Use a robot.txt file ?

Discussion in 'Search Engine Optimization' started by hemanth.freelancer, Oct 28, 2007.

0
  1. #1
    What is this robot.txt file. I am seen it many a times when reading about SEO and am never able to make any sense of it. Should we use a robot.txt file when it comes to SEO ?
     
    hemanth.freelancer, Oct 28, 2007 IP
  2. stickycarrots

    stickycarrots Peon

    Messages:
    4,513
    Likes Received:
    115
    Best Answers:
    0
    Trophy Points:
    0
    #2
    robot.txt is a waste of time
     
    stickycarrots, Oct 28, 2007 IP
  3. harshakiran

    harshakiran Notable Member

    Messages:
    4,586
    Likes Received:
    226
    Best Answers:
    0
    Trophy Points:
    225
    #3
    harshakiran, Oct 28, 2007 IP
  4. Boardwalk

    Boardwalk Well-Known Member

    Messages:
    1,651
    Likes Received:
    44
    Best Answers:
    0
    Trophy Points:
    140
    #4
    If you need help regarding robots.txt, check my sig link. :D It's a guide on how to tweak your robots.txt file.
     
    Boardwalk, Oct 28, 2007 IP
  5. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,926
    Likes Received:
    270
    Best Answers:
    0
    Trophy Points:
    280
    #5
    What would make you think that robots.txt is a waste of time? That is just plain wrong. :eek:
     
    astup1didiot, Oct 28, 2007 IP
  6. grandaddy

    grandaddy Peon

    Messages:
    242
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #6
    It can be used to keep bots from indexing content you wish to remain private.
     
    grandaddy, Oct 28, 2007 IP
  7. FreeTraffic

    FreeTraffic Active Member

    Messages:
    336
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    58
    #7
    FreeTraffic, Oct 29, 2007 IP