keywords in robots.txt

[schabier]

recently someone told me that it was possible to put keywords inside the robots.txt file... I haven't found any reference in speciallysed pages...
Does anybody know anything about??? :confused:

[Spyware Remover]

I have not heard of this. It would not seem to do any good because that file is for SE spiders and it is used to give them instructions.

I am looking forward to other peoples thoughts on this as well.

[caroline]

One word: Spam

I don't know if Google penalizes it, but I am certain it doesn't help in the SERPS at all.

[schabier]

One word: Spam

I don't know if Google penalizes it, but I am certain it doesn't help in the SERPS at all.
Have you ever done it???
I have been searching everywhere and I did not find anything about... only found common and well known uses of it.

[caroline]

Let me ask you the other way around. Why would this help?

And yes, I've read in a specialized robots.txt site (can't remember which one) that stuffing your robots.txt file with keywords is not going to help you.

[john_loch]

It's definitely not the intended use for the file, and I'd be quite surprised if it had any bearing at all.

Must say though, I've never tried it (or heard of it being tried) :)

[Such Great Heights]

Why would Google or any other search engine put relevence on keywords in the robots.txt file?

This would just lead to straight keyword stuffing like meta tags for keywords and descriptions. Then not only would there be only one file listing keywords for your entire domain, but how would this be intuative for users?

Users wont even see this as something to look at.

It doesn't make sense and if any search engine algo was to put an importance on the robots.txt file for keywords then once it is found out it would be abused and taken out of the algo. So this thread alone could ruin it if in fact it had any bearing, which I can't imagine.

[minstrel]

Correct. It makes no sense. And the only way that I can think of at the moment to do it would be to put the keywords in comments lines which are ignored by the spiders any way.

File under "totally bogus myths and rumours".

[mykoleary]

Correct. It makes no sense. And the only way that I can think of at the moment to do it would be to put the keywords in comments lines which are ignored by the spiders any way.

File under "totally bogus myths and rumours".
Exactly. The only thing spiders look for in robots.txt are the field:value pairs which equate to directions of what to do. Anything else is ignored.

See http://www.robotstxt.org/wc/norobots-rfc.html

[Trance-formation]

My theory... if it looks like spam... and smells like spam...