XML site map for SE purpose, and normal site is for user purpose, if you done XML site map and submit to webmaster tool then it is helps crawl your pages.
Both HTML sitemaps and XML sitemaps are a good step in the right direction to improve your website's exposure.
I think you can use both. xml sitemaps are good for the search engines so that you can submit it in the webmaster tool. HTML sitemaps are good for the user point of view. Moreover, search engines like HTML pages.