I have created a sitemap and when I tested it in webmaster tools then its showing 404 error. I want to ask that is just submitting sitemap to GWT is enough or before that we have to add it into the root? If it has to be added then where in the index file? Is there any specific place or we can add it anywhere. Doing it first time that's why I am little bit confused.
You have to add it into the root. For example: http://www.yourdomain.com/sitemap.zip/php/ and then fetch your website using GWT. Also, submit sitemap path in GWT.
You don't need (and shouldn't) set your sitemap in index.html (unless you are talking about an HTML sitemap, in that case, yes, the top of it should be part of your http://mysite.com/index.html file). Put your XML sitemap at the root, for example: http://mysite.com/sitemap.xml and submit it to search engines. Eventually, you can define your sitemap url (i.e., location) in your robots.txt file.