Best Sitemap Generator for more than 30000

Hi,

Which is best sitemap generator tool for more than 30000+ pages.
Anyone here using any tool for thousand of urls.
If anyone than suggest tool and your experience.

Your Prompt reply will be highly appreciated.

Thanks in advance

 

I don't think you could find one, I think FREE sitemap generator can only make for upto 500 pages.

 

Try to use this tool www.xml-sitemaps.com/

 

XML-SITEMAPS COM Only have a maximum of 500 pages will be indexed in sitemap

 

you could do it with an excel document

 

i have used xml-sitemaps it generate 500 at a time .i think there is not any site map generator for 30000 pages .

 

I agree with marketraise, xml-sitemaps is only 500 unless you pay.

 

Hands down, Surveyor. I bought it for the two big car insurance sites in my sig. It's a Dreamweaver extension though, so you need to have DW installed.

 

I do have dreamweaver, but how exactly does it work? I have dynamically rendered pages on my site...

 

It's real similar to how xml-sitemaps.com works, where you enter a website URL and it goes out and crawls the site to whatever level or number of pages you specify. You can pause and restart whenever you need, exclude certain page types, and edit the sitemap when it is complete. The best part is, there is no set limit on the number of pages it will index. And dynamic URL's are a non-issue because it crawls them too.

 

So it's not a web based application, you run it form your computer, right? I'm assuming it generates an .xml files that you can then upload on your server and have the site map. Is this correct?

How reliable is it in listing all pages, including dynamically rendered from a rather large site.

 

It's a Dreamweaver extension, so it runs from within Dreamweaver. It's also designed to create sitemaps from your local Dreamweaver site structure and HTML sitemaps in DW. But you can just put in your website URL, tell it how many pages to crawl, hit the button and watch it go. It generates the XML sitemap then you can go in and edit the page priorities and update frequencies for each page. Then if you want to automatically submit it to the search engines, you just click a button and it sends it to Google, Yahoo, Ask, Live Search and Moreover. It's pretty slick.

It's very good at crawling dynamic pages on large sites. It operates like a search engine and just follows the links in your site. If you tell it to crawl the dynamic links it will. You can also manually add or delete pages from the XML sitemap before sending to the search engines.

 

30.000 pages is not a problem with A1 Sitemap Generator. It has a 30 days fully functional trial you can try if you wish.

 

Thomas, does your evaluation download come with any limitations (for example as to how many pages it will spider)?

 

No limitations.

You may hit a physical limit at some point at number of pages. But it is much higher than 30k pages. (And much higher than most of those socalled "unlimited" alternatives that exist.)

If you want to maximize the amount of pages A1 Sitemap Generator (evaluation or not) check this article.

I know some who scan websites with nearly a million pages, but exactly how much A1SG can scan depends on many factors. Some can only successfully scan and create sitemaps of e.g. 200.000-300.000 pages.

 

I've tried the evaluation copy, it didn't work. Failed to find pages that are linked multiple times. And I just randomly checked, I haven't gone through all 10 000+ records.

 

Please provide an example website address. If you prefer to go through this in public

Usual reasons include people mixing www and non-www in their links, servers suffering from read timeout due to excessive simultaneous connections and so on. Please give me a chance to either acknowledge and fix the issue (which I am always happy to do, I love bug reports since they help improve the product) or diagnose your problem... Anything else is dreadful to an author of a software product

 

To create google sitemap, use gsite crawler. It is absolutely free and can crawl many pages.

 

In all fairness, A1 did an exceptional job building a sitemap. I was looking for a feature I didn't find, but I'm sure it's something you'd be able to implement becuase the product is rather feature rich already. I'll hit you up on PM to show you where it missed several pages that are linked form a major page. It's really weird as it dug up pages that are buried much deeper (old forum archives) and the ones that are linked form main forum pages were missed.

 

I was too quick to make this post and it proved false. Server logs prove that disruption of A1 sitemap creation was due to a momentary lag in server performance which prevented about a dozen URLs from being listed in created sitemap. A1 sitemap creator has a way to resume the analysis and include URLs that were missed due to server downtime which I was not aware of and haven't utilized.

Considering I have a huge website with thousands of database driven URLs, A1 Sitemap Creator did a superb job creating a sitemap that reliably lists all of my URLs. I was too quick to leave previous post and left it without getting myself properly familiar with the software.

One thing that totally blew me away was personal approach from Thomas Schulz dealing with me. You have not known positive shopping experience until you've dealt with these guys. What a bunch of excellent programmers and great people.

CONCLUSION:

After two year of testing various sitemap creators, each of which failed because I have a massive website with high traffic and new pages created every day, I have eventually found a reliable sitemap creator that delivers expected results even for a massive site with thousands of dynamically created URLs. After two years I have pretty much given up hope my site would have a sitemap, and then I've met Thomas from A1 and let me tell you, those guys are A1