robot.txt is necessary ?

Pls, can you help me to know that is it necessary put robot in website .. any problem if i dont use it .

 

robot.txt enable you to instruct robots (google) to not crawl and index certain files, directories within your site Eg. you may not want Google to crawl the /images directory which is meaningless. By this you can save up your site's bandwidth.

 

No problems at all.
If you would like all elements of your site to be indexed then you don't need it or you can put blank robots.txt in the root domain directory.

 

One very good reason to put up a robots.txt file even if it is blank is to prevent hundreds of 404-Not Found errors in your server log, caused by robots trying to check your robots.txt file. This reduces error log file clutter, and makes it more useful for finding real problems.

 

The robots.txt is a very simple text file that is placed on your root directory. All major search engine spiders respect this, and naturally most spambots email collectors for spammers do not. If you truly want security on your site, you will have to actually put the files in a protected directory, rather than trusting the robots.txt file to do the job.

 

You can use .htaccess for a lot of things that robots.txt does

 

robot.txt is not necessary, its depend for you, if you want your site page(some confidential information ) not cache by search engine crawler then use robots.txt.....

 

No its not like this that you must have robots.txt as it helps in crawling to the search engine.

 

Robots.txt is not necessary but it will be good if you add robots.txt.

 

can u explain in detail ?

 

I found many errors in the crawl check from google (you can do this via sitemaps).
I had a 404 redirect back to my main page and because I did not have a robots.txt file googlebot took the main index page as robot.txt file because its the 404 redirect main index page. Ofcourse this will return an error from googlebot.