We have two options which i found one of the best specially when it's about creating xml sitemap for huge websites. 1. http://gsitecrawler.com 2. http://www.xml-sitemaps.com So what you think?
I think I should ask you a few questions. What kind of site do you have? Are you using a content management system, is it a basic HTML site, forum, wordpress blog,,,, the list goes on and on. And what is your definition of a "HUGE" site? Would that be less then 100,000 urls, less then 500,000, less then 1,000,000 or greater then 1,000,000 URLs? A site map generator should match the software that your site is using. There are site map generators designed especially for forums, wordpress blogs,,,,,,,, So whatever type of site you have, you should be using a site map generator designed especially for that product. This is going to give you your best options and results. For the 2 options you asked about: xml-sitemaps.com - At the bottom of the page it says "Maximum 500 pages will be indexed in sitemap." Which is not a very good option. What if your site has 10,000 urls? gsitecrawler.com - looks like it has more options, but I did not see a maximum URL listed. And I did not see if the software will create a 2nd site map if the first one contains over 50,000 URLs. Google will only accept 50,000 URLs per sitemap. So if your site has more then that, you have to submit more then one map. On a personal note - I run a VBulletin forum. Every night my sitemap generator creates a site map with around 650,000 URLs in it - and its spread out over 12 files. And then google grabs the site map every 24 hours. Site maps should be flexible enough to update on a daily basis and be able to include all of the new urls without you having to do anything. If the site has more then 50,000 URLs, the site map generator must be able to create more then one file. From the ones you asked about, it looks like xml-sitemaps.com can not do the job. Mainly because it has a 500 URL limit.
Yahoos news page, news.yahoo.com has 12,000,000 pages indexed with google. So your talking about a site that has 3 million more pages then yahoo? Take 15,000,000 and divide that by 50,000. And that is the number of pages the site map will be. Just to be on the safe side, my site map pages only hold 49,000 URLs. So that means that you are going to need a site map generator that is specific to your content management system.