How to create robots.txt file

Thanks for sharing best information about robots.txt file. It is best information for me. I like it.

 

You can use Google Webmaster Tools to create robots file.
After Webmaster tools create your robots file, you can upload to your web directory

 

1)
User-agent: * Disallow: /
2)
User-agent: Googlebot-Image Disallow: /
3)
User-agent: * Disallow: /cgi-bin/ Disallow: /privatedir/ Disallow: /tutorials/blank.htm
4)
User-agent: * Disallow: / User-agent: Googlebot Disallow: /cgi-bin/ Disallow: /privatedir/
5)
User-agent: * Disallow: / User-agent: ia_archiver Disallow:
6)
User-agent: * Disallow: / User-agent: Googlebot Allow: /