The function of robots.txt

Frens I have read lot of articles about robots in DP
So I just want to ask for a conclusion
Is the robot function only for:
- allowing and dissalowing site for being crawl by spider
- for letting spider know our sitemap, ex: for google spider to know our sitemap
Is it only for that? if there is still another fuction please let me know, thanks

 

Please Review: http://forums.digitalpoint.com/showthread.php?t=531765
And Also: http://www.robotstxt.org

 

I have reviewed them, and I just want to know the conclusion of it so I ask frens bout it again..coz I am still a newbie

 

What excatly do you want to know about it?

 

I mean the function of robots.txt besides, these:
- allowing and dissalowing site for being crawl by spider
- for letting spider know our sitemap, ex: for google spider to know our sitemap
Thanks for helping ssandecki

 

As far as i know it is for... Come on... the name itself suggests what it is for.

 

Man..I am a newbie so you can not just give me answer like that..if you dun want to help its fine but dun give such answer, thanks

 

Don't worry, I dropped him\her some -REP for that negative response which was uncalled for.

http://en.wikipedia.org/wiki/Robots.txt

The wiki entry is great for what your asking, it will explain a lot better than the last URL I gave you, I believe this should answer your basic question on what robots.txt can be used for and its real use on the internet.

 

Man... I did not mean to offend you.

 

Thanks ssandecki, you are a real helper now I know more already thanks again n God Bless