1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Sitemap generator for large site

Discussion in 'Google Sitemaps' started by picobello, Nov 5, 2007.

  1. #1
    I was using a tool named "Vigos Gsitemap" but it is crashing after 6000 pages saying "memory problem" etc

    What tool do you use to generate sitemaps for very large site?

    I am questioning here about a flat EXE tool (a remote generation method)
    SEMrush
    Thanks
     
    picobello, Nov 5, 2007 IP
    SEMrush
  2. kosmo101

    kosmo101 Member

    Messages:
    22
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    36
    #2
    i dont think making a map of that many pages is so great. is this real content or some page generation software you used to make it. free sitempa generator in G to find such a tool
     
    kosmo101, Nov 6, 2007 IP
  3. Joelemon

    Joelemon Active Member

    Messages:
    127
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    58
    #3
    Try to use this, I think it's the best: SOFTplus GSiteCrawler
     
    Joelemon, Nov 6, 2007 IP
    picobello likes this.
  4. cosmocentral

    cosmocentral Banned

    Messages:
    349
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I am having the same issue. I ended up listing the major urls? As a result my SERP traffic from Google dropped to some extent.
     
    cosmocentral, Nov 6, 2007 IP
  5. Desception

    Desception Banned

    Messages:
    106
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    google sitemap generator, should help most of you.
     
    Desception, Nov 6, 2007 IP
  6. picobello

    picobello Active Member

    Messages:
    572
    Likes Received:
    17
    Best Answers:
    0
    Trophy Points:
    60
    #6
    I am busy in testing the SOFTplus GSiteCrawler .... it seems to work so far but it is much much more slower ..... already 1 night and still 5349 url to process
     
    picobello, Nov 6, 2007 IP
  7. litebulb1

    litebulb1 Peon

    Messages:
    151
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #7
    litebulb1, Nov 6, 2007 IP
  8. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #8
    6000 pages is a fairly low amount of pages compared to some websites I know :D

    I suggest you try A1 Sitemap Generator :)
     
    websitetools, Nov 7, 2007 IP
  9. Lood1234

    Lood1234 Banned

    Messages:
    44
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    i use google sitemap
     
    Lood1234, Nov 12, 2007 IP
  10. ~*77*~

    ~*77*~ Peon

    Messages:
    72
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #10
    I tried a bunch of things and ran into all sorts of problems. Some didn't allow enough pages, some crashed, and some seemed too slow.

    In the end I paid for this:

    http://www.xml-sitemaps.com/standalone-google-sitemap-generator.html

    Overall I am pleased, but it is not without problems. It takes about a day to complete, and can add a bit to the CPU load, so I only run it every two weeks or so.

    I have only had it a few months, so it has been updated maybe three or four times to date.

    On one site, the site map has submitted 148,848 pages to Google. As a result, the Google "site:" statistics have climbed from 18,000 to 49,000 - and are still rising.
     
    ~*77*~, Nov 12, 2007 IP
  11. maxtan

    maxtan Guest

    Messages:
    98
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    I am using xml-sitemaps.com
    I got the problem the same as you.
     
    maxtan, Nov 12, 2007 IP
  12. ~*77*~

    ~*77*~ Peon

    Messages:
    72
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #12
    The online one? Or the standalone?

    The online one convinced me that it was finally the script for me and I downloaded it. While it definitely works, it could be better.

    Maybe I am just ignorant, but making a sitemap doesn't seem like it should take so long and be so processor intensive.
     
    ~*77*~, Nov 12, 2007 IP
  13. raj7

    raj7 Active Member

    Messages:
    6
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    86
    #13
    I use xml-sitemaps. I agree it's very resource hungry.
     
    raj7, Nov 13, 2007 IP
  14. ~*77*~

    ~*77*~ Peon

    Messages:
    72
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Glad to see it isn't just me... although it would be nice if it worked well for all.

    Anyone else also have issues of it being interrupted, despite checking the box to keep it from happening? I would say I wind up restarting each sitemap generation at least twice since it seems to stop some time after I go to bed... :p
     
    ~*77*~, Nov 17, 2007 IP
  15. Ibrahim Al Mohanna

    Ibrahim Al Mohanna Peon

    Messages:
    101
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #15
    there is some tools that you can prevent some page from being craled such as log in pages I do not remember the name right now
     
    Ibrahim Al Mohanna, Nov 20, 2007 IP
  16. MihaiOnly

    MihaiOnly Peon

    Messages:
    108
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #16
    If you want a big sitemap try xml-sitemaps.com but you'll have to buy a premium membership for this
     
    MihaiOnly, Nov 28, 2007 IP
  17. camp185

    camp185 Well-Known Member

    Messages:
    1,653
    Likes Received:
    51
    Best Answers:
    0
    Trophy Points:
    180
    #17
    Might be easier to just write your own script. My site has 25,000 pages+ and growing. I spent a couple of hours trying to find a free xml generator with no luck. Build my own in an hour. Done.
     
    camp185, Nov 28, 2007 IP
  18. d tea

    d tea Peon

    Messages:
    436
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #18
    Yea I agree...
     
    d tea, Nov 28, 2007 IP
  19. whitelion

    whitelion Peon

    Messages:
    28
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #19
    xml-sitemaps.com is good, i tried but it can be really slow with large sites.
     
    whitelion, Nov 30, 2007 IP
  20. ~*77*~

    ~*77*~ Peon

    Messages:
    72
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #20
    Maybe you've got a business there if it works well.

    I am just not inclined to do my own.
     
    ~*77*~, Nov 30, 2007 IP