1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Best Sitemap Generator for more than 30000

Discussion in 'Google Sitemaps' started by mikanjweusla, Jul 23, 2008.

  1. #1
    Hi,

    Which is best sitemap generator tool for more than 30000+ pages.
    Anyone here using any tool for thousand of urls.
    If anyone than suggest tool and your experience.

    Your Prompt reply will be highly appreciated.
    SEMrush
    Thanks in advance;)
     
    mikanjweusla, Jul 23, 2008 IP
    SEMrush
  2. real_skinner

    real_skinner Notable Member

    Messages:
    2,401
    Likes Received:
    61
    Best Answers:
    0
    Trophy Points:
    215
    #2
    I don't think you could find one, I think FREE sitemap generator can only make for upto 500 pages.
     
    real_skinner, Jul 23, 2008 IP
  3. Hersheys

    Hersheys Peon

    Messages:
    2,591
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Try to use this tool www.xml-sitemaps.com/
     
    Hersheys, Jul 24, 2008 IP
  4. real_skinner

    real_skinner Notable Member

    Messages:
    2,401
    Likes Received:
    61
    Best Answers:
    0
    Trophy Points:
    215
    #4
    XML-SITEMAPS COM Only have a maximum of 500 pages will be indexed in sitemap :D
     
    real_skinner, Jul 24, 2008 IP
  5. galatron

    galatron Guest

    Messages:
    65
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    you could do it with an excel document
     
    galatron, Jul 24, 2008 IP
  6. marketraise it

    marketraise it Peon

    Messages:
    23
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    i have used xml-sitemaps it generate 500 at a time .i think there is not any site map generator for 30000 pages .
     
    marketraise it, Jul 24, 2008 IP
  7. internetsuser27

    internetsuser27 Peon

    Messages:
    113
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I agree with marketraise, xml-sitemaps is only 500 unless you pay.
     
    internetsuser27, Jul 24, 2008 IP
  8. webrehash

    webrehash Peon

    Messages:
    163
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Hands down, Surveyor. I bought it for the two big car insurance sites in my sig. It's a Dreamweaver extension though, so you need to have DW installed.
     
    webrehash, Jul 28, 2008 IP
  9. muncle

    muncle Guest

    Messages:
    1,195
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I do have dreamweaver, but how exactly does it work? I have dynamically rendered pages on my site...
     
    muncle, Jul 28, 2008 IP
  10. webrehash

    webrehash Peon

    Messages:
    163
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #10
    It's real similar to how xml-sitemaps.com works, where you enter a website URL and it goes out and crawls the site to whatever level or number of pages you specify. You can pause and restart whenever you need, exclude certain page types, and edit the sitemap when it is complete. The best part is, there is no set limit on the number of pages it will index. And dynamic URL's are a non-issue because it crawls them too.
     
    webrehash, Jul 28, 2008 IP
  11. muncle

    muncle Guest

    Messages:
    1,195
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    0
    #11
    So it's not a web based application, you run it form your computer, right? I'm assuming it generates an .xml files that you can then upload on your server and have the site map. Is this correct?

    How reliable is it in listing all pages, including dynamically rendered from a rather large site.
     
    muncle, Jul 28, 2008 IP
  12. webrehash

    webrehash Peon

    Messages:
    163
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #12
    It's a Dreamweaver extension, so it runs from within Dreamweaver. It's also designed to create sitemaps from your local Dreamweaver site structure and HTML sitemaps in DW. But you can just put in your website URL, tell it how many pages to crawl, hit the button and watch it go. It generates the XML sitemap then you can go in and edit the page priorities and update frequencies for each page. Then if you want to automatically submit it to the search engines, you just click a button and it sends it to Google, Yahoo, Ask, Live Search and Moreover. It's pretty slick.

    It's very good at crawling dynamic pages on large sites. It operates like a search engine and just follows the links in your site. If you tell it to crawl the dynamic links it will. You can also manually add or delete pages from the XML sitemap before sending to the search engines.
     
    webrehash, Jul 28, 2008 IP
  13. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #13
    30.000 pages is not a problem with A1 Sitemap Generator. It has a 30 days fully functional trial you can try if you wish.
     
    websitetools, Jul 28, 2008 IP
  14. muncle

    muncle Guest

    Messages:
    1,195
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Thomas, does your evaluation download come with any limitations (for example as to how many pages it will spider)?
     
    muncle, Jul 28, 2008 IP
  15. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #15
    No limitations.

    You may hit a physical limit at some point at number of pages. But it is much higher than 30k pages. (And much higher than most of those socalled "unlimited" alternatives that exist.)

    If you want to maximize the amount of pages A1 Sitemap Generator (evaluation or not) check this article.

    I know some who scan websites with nearly a million pages, but exactly how much A1SG can scan depends on many factors. Some can only successfully scan and create sitemaps of e.g. 200.000-300.000 pages.
     
    websitetools, Jul 29, 2008 IP
  16. muncle

    muncle Guest

    Messages:
    1,195
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    0
    #16
    I've tried the evaluation copy, it didn't work. Failed to find pages that are linked multiple times. And I just randomly checked, I haven't gone through all 10 000+ records.
     
    muncle, Jul 29, 2008 IP
  17. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #17
    Please provide an example website address. If you prefer to go through this in public :)

    Usual reasons include people mixing www and non-www in their links, servers suffering from read timeout due to excessive simultaneous connections and so on. Please give me a chance to either acknowledge and fix the issue (which I am always happy to do, I love bug reports since they help improve the product) or diagnose your problem... Anything else is dreadful to an author of a software product :D
     
    websitetools, Jul 29, 2008 IP
  18. mobiplanet.in

    mobiplanet.in Peon

    Messages:
    1
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #18
    To create google sitemap, use gsite crawler. It is absolutely free and can crawl many pages.
     
    mobiplanet.in, Jul 30, 2008 IP
  19. muncle

    muncle Guest

    Messages:
    1,195
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    0
    #19
    In all fairness, A1 did an exceptional job building a sitemap. I was looking for a feature I didn't find, but I'm sure it's something you'd be able to implement becuase the product is rather feature rich already. I'll hit you up on PM to show you where it missed several pages that are linked form a major page. It's really weird as it dug up pages that are buried much deeper (old forum archives) and the ones that are linked form main forum pages were missed.
     
    muncle, Jul 30, 2008 IP
  20. muncle

    muncle Guest

    Messages:
    1,195
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    0
    #20
    I was too quick to make this post and it proved false. Server logs prove that disruption of A1 sitemap creation was due to a momentary lag in server performance which prevented about a dozen URLs from being listed in created sitemap. A1 sitemap creator has a way to resume the analysis and include URLs that were missed due to server downtime which I was not aware of and haven't utilized.

    Considering I have a huge website with thousands of database driven URLs, A1 Sitemap Creator did a superb job creating a sitemap that reliably lists all of my URLs. I was too quick to leave previous post and left it without getting myself properly familiar with the software.

    One thing that totally blew me away was personal approach from Thomas Schulz dealing with me. You have not known positive shopping experience until you've dealt with these guys. What a bunch of excellent programmers and great people.

    CONCLUSION:

    After two year of testing various sitemap creators, each of which failed because I have a massive website with high traffic and new pages created every day, I have eventually found a reliable sitemap creator that delivers expected results even for a massive site with thousands of dynamically created URLs. After two years I have pretty much given up hope my site would have a sitemap, and then I've met Thomas from A1 and let me tell you, those guys are A1
     
    muncle, Aug 2, 2008 IP