Hello, To keep the visitors on the page i created .httaccess and placed ErrorDocument 404 redirect http://www.yoursite-name.com/sitemap.php It works fine! when i type www.yoursite-name.com/blanlabakhdfd it goest to http://www.yoursite-name.com/sitemap.php I am fine with that. Unfortunately, my IT guy, the company where i work, blocked robots.txt file, he said robots.txt file is not necessary for SEO purposes. he also said that SEearch engine robots will index the site with out it. and added by having robots.txt and accidently put the wrong information in there, you could have serious problems. I told him that you need to put the correct info in there. anyways, what happens now when robots.txt is blocked when i type www.yoursite-name.com/robotx.txt in browse it goest to http://www.yoursite-name.com/sitemap.php the question is this: Will this CONFUSE the SE robots (spiders)? Will my site still be indexed? Any sugesstions? comments?
Until you change the error page in your .htaccess to a relative path, you will not generate a 404 for the robots.txt, but a 302. A 302 is a temporary redirect, and Google will index the resulting content from the redirect under the url it comes in on.
I'd leave a totally blank file there. When the file isn't found the server has to think about what to serve up. An empty file is quicker and more efficient.