Best Sitemap Generator for Large Sites

Any suggestions for a Sitemap creator that will do a large site, preferably with minimal resources and something that Google will accept.

Thanks.

 

I'm going to give GSiteCrawler a shot since the webpage was very nice I'll let you know how it goes

 

Ok, trying GSiteCrawler but it's taking forever. Seems to be a good program though. (Many options, been around for awhile, etc.)

 

If you have a large website, then it is going to take time I can't speak for GSiteCrawler, but here's a list of things you can try to speed up crawling with A1 Sitemap Generator. Some of the tips are general and apply to more sitemapper tools.

 

How long is a long time? Like a whole day? And is this bad for a server? Thanks.

 

Dpeneds on how large a website you have. You can always try to compare the speed / tinker with the options described in the article. It also discusses serverload issues. If you have a very large website, then you may want to e.g. schedule the scan at night (At least A1SG supports this. I don't know about GSC, but you could always try ask the author.)

 

Try this online Google sitemap generator - Has always worked for me on large sites and is pretty fast.

 

WebCEO produces well organized and visually attractive sitemaps for large websites.

 

xml-sitemaps.org is good but maxes out at 500 pages. Thanks though.

 

I've found that it is best to break the site up into pieces, and work from there. Has helped me tremendously, and can be done over time.

 

I have looked at the online site-map generators and was always concerned that they would add their own content as a "hidden fee", anyone ever check this out...I personally would not even know where to look

 

WE have use gSiteCrawler for years and are happy with it. Check your settings and make sure you are indexing only the web pages, not jpg, gif, etc.

Also, we have about 38,000 pages. We start it when we leave the office and it is done in the morning. Don't run it from home during the day. The band width will kill you.

 

will try gsitecrawler..hve tried xml-sitemaps.org that is good ..

 

try auditmypc.com s sitemap generator

 

I've used xml-sitemaps.org, i bought their paid version. its great for large sites and u can set a cron job so it'll update google automatically.

 

thanks a lot, this sitemap really helped

 

I would suggest phpSitemap, setup is 1 minute max.

 

Can you run a python script?

The google sitemap generator is the fastest I have seen, generates an 18000 map without issue.

https://www.google.com/webmasters/tools/docs/en/sitemap-generator.html

Setup a cron job to run the script as often as your content updates and it will send the new one off to google without any interaction.

 

No, I saw that but it seemed a little complex, may be worth it though if it is very fast. Thanks Bob.

Lance

 

So I guess the question is now, what is the fastest sitemap generator for a site?

Is it fast? Faster than GSiteCrawler? The only problem with GSiteCrawler is it seems to load each and every page, which may slow it down (at least according to stats). Thanks.