Your robots.txt file is something that the search engine robots refer to when they crawl your site. This tells them what they can and cant crawl. If you want your website to be crawled by the search engines, best to use Webmaster Tools by Google, they have a robots.txt generator there, and it will make sure that when a bot visits your site, it can get everything that it needs. You can also use the robots.txt file to block potentially harmful bots as well, but that is completely at your own disgression.