robots.txt

Discussion in 'robots.txt' started by neeruneeraj, Aug 19, 2008.

0
  1. #1
    hi all, what is a robots file and what's use for robot file.
     
    neeruneeraj, Aug 19, 2008 IP
  2. ramesollin

    ramesollin Peon

    Messages:
    34
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Robot.txt file is very usable file in indexing pages in different search engines...

    You can block the Search Engine's robot from indexing it.

    for more information just find this on Google Help. I will help you
     
    ramesollin, Aug 19, 2008 IP
  3. kalinpetrovbg

    kalinpetrovbg Peon

    Messages:
    273
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3

    read this : http://forums.digitalpoint.com/showthread.php?t=423794
     
    kalinpetrovbg, Aug 20, 2008 IP
  4. catanich

    catanich Peon

    Messages:
    1,921
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #4
    A Search Engine will index everything unless told not to. The robots.txt file is the instruction that tell the SEs what not to index.
     
    catanich, Aug 21, 2008 IP
  5. ramesollin

    ramesollin Peon

    Messages:
    34
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    ramesollin, Aug 25, 2008 IP