Any good Sitemap generator for big sites

Discussion in 'Google Sitemaps' started by dpsubi1, Nov 30, 2008.

  1. #1
    Hi,

    One of my client has a site (using wordpress CMS) with huge number of pages like 100000 pages ... we have tried many sitemap tools but most of them didn't work.

    Can any one suggest me a good tool to create sitemaps for such a big site.

    Thanks in advance.
     
    dpsubi1, Nov 30, 2008 IP
  2. MeetHere

    MeetHere Prominent Member

    Messages:
    15,399
    Likes Received:
    994
    Best Answers:
    0
    Trophy Points:
    330
    #2
    MeetHere, Nov 30, 2008 IP
  3. dpsubi1

    dpsubi1 Notable Member

    Messages:
    9,318
    Likes Received:
    420
    Best Answers:
    0
    Trophy Points:
    280
    #3
    yes. I know that one.

    Anyone have experience with this standalone tool ???? is it works good with big sites ?
     
    dpsubi1, Nov 30, 2008 IP
  4. websitetools

    websitetools Well-Known Member

    Messages:
    1,513
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #4
    I don't think you will have any problems with A1 Sitemap Generator, at least not if you follow this article about crawling big websites.
     
    websitetools, Nov 30, 2008 IP
  5. gbh1935

    gbh1935 Peon

    Messages:
    585
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #5
    gbh1935, Nov 30, 2008 IP
  6. jingwen

    jingwen Peon

    Messages:
    312
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #6
    jingwen, Nov 30, 2008 IP
  7. Bezzen

    Bezzen Peon

    Messages:
    244
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I've just used GSiteCrawler for a 200,000+ pages website. Worked great, eventhough it did take a small lifetime to crawl all the pages. ;)
     
    Bezzen, Dec 1, 2008 IP
  8. dpsubi1

    dpsubi1 Notable Member

    Messages:
    9,318
    Likes Received:
    420
    Best Answers:
    0
    Trophy Points:
    280
    #8
    didn't worked. it will eat server CPU and stopped after the limit. Even increasing the memory and time limit doesn't helped.
    I will give a try.

    I tried VIGOS Gsitemap 0.97a, after running around 25000 urls, it errored out and stopped working.
     
    dpsubi1, Dec 1, 2008 IP
  9. Bezzen

    Bezzen Peon

    Messages:
    244
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #9
    GSiteCrawler crashed once for me, but both the crawled pages and the queue was loaded up after the restart with no problems so nothing was lost. It did almost give me a slight heart attack though. :D
     
    Bezzen, Dec 1, 2008 IP
  10. articledirectory

    articledirectory Peon

    Messages:
    1,704
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    0
  11. dpsubi1

    dpsubi1 Notable Member

    Messages:
    9,318
    Likes Received:
    420
    Best Answers:
    0
    Trophy Points:
    280
    #11
    dpsubi1, Dec 1, 2008 IP