Is Index file necessary in each website? If yes, then how we can hide it from the crawler as well from the web searcher? Otherwise crawler will treat that index file as the different page with same content as in the home page.
Unfortunately it is the way that webservers work - the index file is classed as the homepage. The best way to get around it is to use a canonical tag on the /index version so; <link rel="canonical" href="http://www.example.com/" /> You can find more information on how Google uses the Canonical tag here; googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html