Guys I have a website thats less than 1 month, and I have been looking at my stats every day,,,, the site average around 6 visitors a day, but what is bothering me is the huge error hits I am getting.... I am not sure what to make of it Any explanation or suggestins or ideas are welcome.... takea look.. http://img162.imageshack.us/i/statse.jpg/
Since you've blanked out the filenames, it's difficult to know where the errors are coming from. But the majority seem to be from a non-existent robots.txt file, which is nothing to worry about. The errors are being caused by spiders looking for the file, and not finding it. You can either leave it as it is, create a blank one, or read up on how to create one that benefits your site.
I am mainly concerned with the robots.txt errors.....80 for the month so far. So you are saying I have nothing to worry about ok...its just spiders
Yep. It's just Googs and Yahoo etc. looking for your robots.txt file. If you don't need one, then you don't need one.
Well, if you don't want the errors, just make an empty file and save it as robots.txt in your root dir.
Ok so I did create a page and named it /robots.txt and errors have stoped It is just a blank page.....no words...no codes..nothing......will it affect traffic in anyway....or my site on google... As you can see the effect of adding the robots.txt page.....no more errors after 3 days http://img198.imageshack.us/i/stats2u.jpg/
Good job! Now, you may want to benefit from that robots.txt file. Add something like this to it (just the basics). # /robots.txt file for http://www.yourwebsite.com/ User-agent: * Disallow: /cgi-bin/ Code (markup): Just modify the domain to reflect your domain. You can add more directories if needed.
The file has a purpose, that's why the bots are looking for it. What I offered above is an example of what might be placed in that file. The first line is just a comment, but the other lines actually talk to Good bots. Bad bots use it for something else, if they use it at all. The lines above tell all Good bots to stay out of your cgi-bin (assuming you have one). I just thought I'd offer an example of what a robots file is used for.