1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Best Sitemap Generator for Large Sites?

Discussion in 'Google Sitemaps' started by thebassman, Jan 29, 2006.

0
Page 1 of 3
  1. #1
    Anyone have a recommendation for a Sitemap Generator for sites in the 1M+ range?
     
    thebassman, Jan 29, 2006 IP
  2. thebassman

    thebassman Asleep at the Keyboard

    Messages:
    1,105
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    140
    #2
    Bueller? ;)
     
    thebassman, Feb 3, 2006 IP
  3. WhatiFind

    WhatiFind offline

    Messages:
    1,789
    Likes Received:
    257
    Best Answers:
    0
    Trophy Points:
    180
    #3
    WhatiFind, Feb 3, 2006 IP
  4. Fishing Forum

    Fishing Forum Active Member

    Messages:
    537
    Likes Received:
    21
    Best Answers:
    0
    Trophy Points:
    60
    #4
    Fishing Forum, Feb 3, 2006 IP
  5. thebassman

    thebassman Asleep at the Keyboard

    Messages:
    1,105
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    140
    #5
    That's what I've been using up until now, but my site is bigger than the database will allow. I have't tried the SQL version, but the GSiteCrawler ran out of database space before I got to 1M pages.
     
    thebassman, Feb 6, 2006 IP
  6. extreme90

    extreme90 Banned

    Messages:
    450
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Cofee cup google site map generator is a good choice
     
    extreme90, Feb 6, 2006 IP
  7. thebassman

    thebassman Asleep at the Keyboard

    Messages:
    1,105
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    140
    #7
    The Coffee Cup one won't crawl sites, though, so any large sites that reply on the dynamic creation of pages won't work w/ that SM generator.
     
    thebassman, Feb 6, 2006 IP
  8. The_FoX

    The_FoX Peon

    Messages:
    666
    Likes Received:
    21
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Bass, im looking for a similar one too - as of now i think the google python script (https://www.google.com/webmasters/tools/docs/en/sitemap-generator.html) is the best option to handle large sites. But i heard that some errors were reported running it on the live server by some webmaster here on DP - so it may be a risky one.

    Please do let me know if you found a better third party software.
     
    The_FoX, May 13, 2007 IP
  9. thebassman

    thebassman Asleep at the Keyboard

    Messages:
    1,105
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    140
    #9
    I'm now using http://gsitecrawler.com - it's been updated to allow for crawling of bigger sites.

    Best of luck.
     
    thebassman, May 14, 2007 IP
  10. axlarry

    axlarry Notable Member

    Messages:
    3,961
    Likes Received:
    121
    Best Answers:
    0
    Trophy Points:
    240
    #10
    I have tried to use google python script before. But I just can't figure out how exactly to run the script.

    I mean, what requirements do I need to run it from my server?
     
    axlarry, May 14, 2007 IP
  11. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #11
    gsitecrawler is good sitemap maker but i allways prefer the module or component if my cms has
     
    trichnosis, May 14, 2007 IP
  12. azvouchers.com

    azvouchers.com Peon

    Messages:
    24
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    I try to use GSiteCrawler
     
    azvouchers.com, May 15, 2007 IP
  13. tamilsoft

    tamilsoft Banned

    Messages:
    1,155
    Likes Received:
    78
    Best Answers:
    0
    Trophy Points:
    0
    #13
    tamilsoft, May 15, 2007 IP
  14. mobhead

    mobhead Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Trying GSiteCrawler now - is this really the best tool a site of around 100k pages?
     
    mobhead, May 22, 2007 IP
  15. spin

    spin Well-Known Member

    Messages:
    1,258
    Likes Received:
    48
    Best Answers:
    0
    Trophy Points:
    138
    #15
    I use siteexpert...And I love it. It will crawl the biggest sites so I'm told...crawl mine anyway as the free ones didn't. but i also download my site and had it crawl on my comp...was much faster
     
    spin, May 22, 2007 IP
  16. infoway

    infoway Guest

    Messages:
    145
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #16
    We developed our own google xml sitemap generator.

    Our xml sitemap generator can generate 700,000+ pages [tested on 2 sites].

    Let me know if you need any help.
     
    infoway, May 23, 2007 IP
  17. GeekMike

    GeekMike Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #17
    Hey Bassman - are you still using GsiteCrawler?

    -michael
     
    GeekMike, May 23, 2007 IP
  18. thebassman

    thebassman Asleep at the Keyboard

    Messages:
    1,105
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    140
    #18
    Yeah, I use it for all my sites pretty much - except for sites that have built-in sitemap programs.
     
    thebassman, May 23, 2007 IP
  19. GeekMike

    GeekMike Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #19
    For larger, dynamic sites?

    -mb
     
    GeekMike, May 23, 2007 IP
  20. thebassman

    thebassman Asleep at the Keyboard

    Messages:
    1,105
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    140
    #20
    I've used it for a site with almost 2 million dynamic pages.
     
    thebassman, May 23, 2007 IP
Page 1 of 3