About This robot

Discussion in 'Google' started by taha009, Mar 16, 2012.

0
  1. #1
    User-agent: 008
    Disallow: /

    what is About this robot
    008
     
    taha009, Mar 16, 2012 IP
  2. ashleyjohn2347

    ashleyjohn2347 Peon

    Messages:
    269
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I didn't get the question. Will you please elaborate?
     
    ashleyjohn2347, Mar 16, 2012 IP
  3. turkishhomes

    turkishhomes Peon

    Messages:
    299
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #3
    It is there to prevent a particular crawler (80legs) from crawling your site. For more information go to:
    http://www.80legs.com/webcrawler.html
     
    turkishhomes, Mar 17, 2012 IP