Robot txt

Discussion in 'Search Engine Optimization' started by gooseandluba, Dec 11, 2008.

  1. #1
    Here is what my robot txt. file looks like.

    User-agent: *
    Disallow: /administrator/
    Disallow: /cache/
    Disallow: /components/
    Disallow: /editor/
    Disallow: /help/
    Disallow: /images/
    Disallow: /includes/
    Disallow: /language/
    Disallow: /mambots/
    Disallow: /media/
    Disallow: /modules/
    Disallow: /templates/
    Disallow: /installation/

    Is it configured properly or should it be changed?
     
    gooseandluba, Dec 11, 2008 IP
  2. vansterdam

    vansterdam Notable Member

    Messages:
    3,145
    Likes Received:
    120
    Best Answers:
    0
    Trophy Points:
    245
    #2
    It looks fine. As long as it is robots.txt and not robot.txt
     
    vansterdam, Dec 11, 2008 IP
  3. jamesplato

    jamesplato Active Member

    Messages:
    359
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    60
    #3
    jamesplato, Dec 11, 2008 IP
  4. robertsmith

    robertsmith Peon

    Messages:
    92
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Hi;
    i am agree with vansterdam file name should be robots.txt not robot.txt.
     
    robertsmith, Dec 11, 2008 IP
  5. Blinked

    Blinked Guest

    Messages:
    74
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    are there other robots? analogical to it ^_^
     
    Blinked, Dec 11, 2008 IP
  6. seoservicesvs

    seoservicesvs Member

    Messages:
    310
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    26
    #6
    It's Ok, I agree with vansterdam. you can use file name robots.txt
     
    seoservicesvs, Dec 11, 2008 IP
  7. safi_982@hotmail.com

    safi_982@hotmail.com Peon

    Messages:
    30
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    thanks for this information..I was new and only not getting the right answer to what to put in my robot.txt file..that is why I never made one ..Now I think I can and can test it too..I hope this is all to put in this file nothing else to put in this file..
     
    safi_982@hotmail.com, Dec 27, 2008 IP
  8. hardkoramerikan

    hardkoramerikan Member

    Messages:
    54
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    43
    #8
    noob question.
    What is the robot txt thing about anyway?
    what is it?
    laugh if you want. Im a noob but Im here too learn.:)
     
    hardkoramerikan, Dec 27, 2008 IP
  9. jamesplato

    jamesplato Active Member

    Messages:
    359
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    60
    #9
    No laughing--its better to ask then guess. I recommend you check out my blog, it answers a lot of noob questions among other things. In short, a robots.txt controls what files a "robot" (or spider) from a search engine will index. So for example, if you have a website, and you place all your pictures in an "images" folder, but you don't want Google to index these Photos in Google Images, you write in the Robots text to not allow Google to Index this folder.

    read more here
    http://www.mark8t.com/2008/09/03/why-you-need-a-robotstxt/
     
    jamesplato, Jan 12, 2009 IP