Purpose of robot.txt file

Discussion in 'Search Engine Optimization' started by bradchawla, Feb 22, 2009.

0
  1. #1
    Hi can plz tell me the information about robot.txt file ,,,what is robot.txt file and whats the purpose of robot.txt file....
     
    bradchawla, Feb 22, 2009 IP
  2. Lpe04

    Lpe04 Peon

    Messages:
    579
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    0
    #2
    just do a search. www.robotstxt.org Blocks search engines and spiders from certain parts of your site.
     
    Lpe04, Feb 22, 2009 IP
  3. shailendra

    shailendra Peon

    Messages:
    1,225
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    0
    #3
    shailendra, Feb 22, 2009 IP
  4. ekajanabee4u

    ekajanabee4u Peon

    Messages:
    146
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    In short meaning robots.txt help to search engine which page or directory allow or disallow for indexing for website
     
    ekajanabee4u, Feb 22, 2009 IP
  5. kutekutta

    kutekutta Peon

    Messages:
    807
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Make your pages Keep away from SE spiders and keep near to the Spiders also.
     
    kutekutta, Feb 22, 2009 IP