How to create sitemap for large content site.

I have article website which have 80,000+ articles. I was trying to create a sitemap with a free program called GSitecrawler; it’s been running 6 hours, still crawling the site.
I wonder how long it should take to create the sitemap.
Is there any better approach/tool to make sitemap for large site?

 

Have you tried http://enarion.net/google/phpsitemapng/?

 

For really large sites I would prefer to use a sitemap generator that is tied in to the CMS / database. It just doesn't make sense for you to crawl the site when you (should) have all the URLs in your database, just a few queries away. Of course if your site is a combination of lots of small database-based components then you won't have a single table to query .

Large sites take time to crawl, no matter which tool you use. For the GSiteCrawler you might want to try the SQL-based version (a slightly faster database makes a difference) or even use it in a network (you can have it run for the same site on several computers at the same time).

 

Just crawl them on directory based.. It is more easier..

 

I might be too late with my response, but I use Sitemap Writer Proand it's crawling takes several minutes the most, plus, it has up to 40 multi-thread crawler.

 

Here is the website: http://sitemapwriter.com/

 

For these there are so many facilities can be available for creating a large content map in a site. It is a combination of small data base components and give so many advance facilities for these content site map.

 

If it is large content site with word press then it is easy to create.Pick the google XML site map plugin and activate it.

 

nice program Sitemap Writer Pro

 

I give you the list of sites which create site maps automatically:--
http://code.google.com/p/sitemap-generators/wiki/SitemapGenerators

 

If it is a large site, it is better to create multiple sitemaps as you might hit the limit of urls that can be included in each sitemap.
Then create an index stemap file and submit it.
For more details, check http://googlewebmastercentral.blogspot.com/2006/10/multiple-sitemaps-in-same-directory.html

 

You could also try A1 Sitemap Generator, but do note that 80k URLs always going to take time unless you are connected directly to DB. i.e. crawling takes time and often the webserver can choke and be the limiting factor, e.g. if you beef up simultaneous connections/threads in programs like A1SG

(BTW, regarding the "limit of URLs in each sitemap" most sitemappers can split across multiple xml sitemap files)