what is robots.txt using for?

Discussion in 'Site & Server Administration' started by johnsebastian777, Aug 4, 2009.

0
  1. #1
    what is robots.tx using for? is that like a spider??
     
    johnsebastian777, Aug 4, 2009 IP
  2. qazu

    qazu Well-Known Member

    Messages:
    1,834
    Likes Received:
    62
    Best Answers:
    0
    Trophy Points:
    165
    #2
    qazu, Aug 4, 2009 IP
  3. premiumscripts

    premiumscripts Peon

    Messages:
    1,062
    Likes Received:
    48
    Best Answers:
    0
    Trophy Points:
    0
    #3
    This question has been asked, and answered, a thousand times. It's best to first try and search google or this forum before you ask this. Just helping :)
     
    premiumscripts, Aug 5, 2009 IP
  4. Dr-AMG

    Dr-AMG Peon

    Messages:
    32
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    In essence it is a guide for spiders to crawl your site. Got it?!
     
    Dr-AMG, Aug 7, 2009 IP