1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

keywords in robots.txt

Discussion in 'robots.txt' started by schabier, Oct 7, 2004.

0
  1. #1
    recently someone told me that it was possible to put keywords inside the robots.txt file... I haven't found any reference in speciallysed pages...
    Does anybody know anything about??? :confused:
     
    schabier, Oct 7, 2004 IP
  2. Spyware Remover

    Spyware Remover Peon

    Messages:
    126
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I have not heard of this. It would not seem to do any good because that file is for SE spiders and it is used to give them instructions.

    I am looking forward to other peoples thoughts on this as well.
     
    Spyware Remover, Oct 7, 2004 IP
  3. caroline

    caroline Peon

    Messages:
    376
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #3
    One word: Spam

    I don't know if Google penalizes it, but I am certain it doesn't help in the SERPS at all.
     
    caroline, Oct 7, 2004 IP
  4. schabier

    schabier Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Have you ever done it???
    I have been searching everywhere and I did not find anything about... only found common and well known uses of it.
     
    schabier, Oct 7, 2004 IP
  5. caroline

    caroline Peon

    Messages:
    376
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Let me ask you the other way around. Why would this help?

    And yes, I've read in a specialized robots.txt site (can't remember which one) that stuffing your robots.txt file with keywords is not going to help you.
     
    caroline, Oct 7, 2004 IP
  6. john_loch

    john_loch Rodent Slayer

    Messages:
    1,294
    Likes Received:
    66
    Best Answers:
    0
    Trophy Points:
    138
    #6
    It's definitely not the intended use for the file, and I'd be quite surprised if it had any bearing at all.

    Must say though, I've never tried it (or heard of it being tried) :)
     
    john_loch, Oct 7, 2004 IP
  7. Such Great Heights

    Such Great Heights Peon

    Messages:
    715
    Likes Received:
    19
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Why would Google or any other search engine put relevence on keywords in the robots.txt file?

    This would just lead to straight keyword stuffing like meta tags for keywords and descriptions. Then not only would there be only one file listing keywords for your entire domain, but how would this be intuative for users?

    Users wont even see this as something to look at.

    It doesn't make sense and if any search engine algo was to put an importance on the robots.txt file for keywords then once it is found out it would be abused and taken out of the algo. So this thread alone could ruin it if in fact it had any bearing, which I can't imagine.
     
    Such Great Heights, Oct 7, 2004 IP
  8. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #8
    Correct. It makes no sense. And the only way that I can think of at the moment to do it would be to put the keywords in comments lines which are ignored by the spiders any way.

    File under "totally bogus myths and rumours".
     
    minstrel, Oct 8, 2004 IP
  9. mykoleary

    mykoleary Peon

    Messages:
    64
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Exactly. The only thing spiders look for in robots.txt are the field:value pairs which equate to directions of what to do. Anything else is ignored.

    See http://www.robotstxt.org/wc/norobots-rfc.html
     
    mykoleary, Oct 9, 2004 IP
  10. Trance-formation

    Trance-formation Peon

    Messages:
    598
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    0
    #10
    My theory... if it looks like spam... and smells like spam...
     
    Trance-formation, Oct 9, 2004 IP