What is robots.txt file? Which rules it can have in SEO? What information it contains? Is that nesseccary to have in directory of our site. How much it can be useful in SEO discussions? Please say about this.
robots.txt is a file that is uploaded in your web server. It is used to instruct the search engines on which pages to crawl and which pages not to crawl. It is just an instruction, but most search engines follow it (though not necessary). If you have some confidential details that you don't want to appear in SERP, then you can use robots.txt (If the details are really sensitive, then this method of protection is not preferred). I don't think it has anything to do with SEO. Anyways check robotstxt.org for more information on robots.txt
In general robots.txt is not about security - because anybody can see the robot.txt file and therefor know where on your site the info is. It is about obscurity. Certain pages may be on your site, for historical reasons for instance, and there is no harm in people getting to the page, but you would prefer new people got to different pages. That's when to use robot.txt : new people will get to the other pages. One example I heard about was the personal site of someone. They wanted to keep that site just for friends and family, not potential employers for instance. So they had a robots.txt file to keep the SE's out. Worked like a charm.
I manage PPC for a couple clients. I take some pages on the website and customize it for the adwords keywords. both to increase the pages quality score and so its relevant to the users that are landing there. I use the robot.txt and tell the search engines not to spider these pages because they are all duplicates of other pages....just a few words are changed.
I have no any thing that I want to disallow robots to see them then it seems to be good to have robots.txt file with no any information Thanks for good helps
If you have no pages you want google to ignore, you don't need a robots.txt file. Don't bother with it.