Hi, I made a robots.txt for my domain http://www.ad4business.com . Please have a look and let me know if it is right or is there a syntax error. Thanks The link to RT is http://www.ad4business.com/robots.txt Regards jeet
Yep, looks fine to me. See http://www.robotstxt.org/ for any further needs for help regarding your robots.txt
Thanks . Actually MSN is not indexing the site and I just saw a request for robots.txt in error log . Think it is MSN . Hope it does now . Thanks for checking . Regards jeet
Sure, np. I often just create a robots.txt so robots see that it exists, but there's nothing in it. Also, that way it doesn't rack up my error_log too. Good luck.
You should also create an xml sitemap for your domain. I use this one, and it seems to do well. http://www.xml-sitemaps.com/