Robots.txt

Discussion in 'robots.txt' started by Jooy, Mar 16, 2011.

0
  1. #1
    How to use robot.txt for joomla sites ?
     
    Jooy, Mar 16, 2011 IP
  2. Alan Smith

    Alan Smith Active Member

    Messages:
    1,263
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    78
    #2
    Its same like that of other sites. Just add and post it in root folder.
     
    Alan Smith, Mar 21, 2011 IP
  3. livandos

    livandos Member

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    36
    #3
    User-agent: *
    Disallow: /administrator/
    Disallow: /cache/
    Disallow: /components/
    Disallow: /editor/
    Disallow: /help/
    Disallow: /images/
    Disallow: /includes/
    Disallow: /language/
    Disallow: /mambots/
    Disallow: /media/
    Disallow: /modules/
    Disallow: /templates/
    Disallow: /installation/
     
    livandos, Mar 22, 2011 IP
  4. vijay01

    vijay01 Peon

    Messages:
    13
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    robot.txt file are add in root directory. it is benifical for crawler.
     
    vijay01, Mar 22, 2011 IP
  5. srikanth.akella

    srikanth.akella Peon

    Messages:
    138
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Upload it to the root folder. The way to upload robots.txt is always unique.
     
    Last edited: Mar 22, 2011
    srikanth.akella, Mar 22, 2011 IP
  6. vijay01

    vijay01 Peon

    Messages:
    13
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Thanks dear
     
    vijay01, Mar 22, 2011 IP