Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Software to create sitemap of 16000 urls

Discussion in 'Google Sitemaps' started by anniseed, Aug 28, 2009.

0
  1. #1
    Hi

    I have a site with 16000 + urls. I'm curently using software called GsiteCrawler but it takes hours to index the whole site. Does anybody know of a better tool to index and create sitemaps for this many urls?

    Thanks
    SEMrush
     
    anniseed, Aug 28, 2009 IP
    SEMrush
  2. theapparatus

    theapparatus Peon

    Messages:
    2,925
    Likes Received:
    119
    Best Answers:
    0
    Trophy Points:
    0
    #2
    theapparatus, Aug 28, 2009 IP
  3. anniseed

    anniseed Peon

    Messages:
    143
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Dreamweaver for design, content on semi static/dynamic pages. Wordpress for blog section of site and phpbb for the forum.

    Thats my problem at the moment, time. I've been running my sitemap software all day and it's still not finished. so far it's up to about 18,000 urls.
     
    anniseed, Aug 28, 2009 IP
  4. new-media

    new-media Peon

    Messages:
    73
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    i had the same problem wilst looking there are some paid versions out there... i ended up using the python script from google - do u have ssh access to your server? its easy to use once up there.
     
    new-media, Aug 29, 2009 IP
  5. anniseed

    anniseed Peon

    Messages:
    143
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I'm afriad I don't. I can't afford to use a payed service ethier.
     
    anniseed, Aug 29, 2009 IP
  6. theapparatus

    theapparatus Peon

    Messages:
    2,925
    Likes Received:
    119
    Best Answers:
    0
    Trophy Points:
    0
    #6
    What specific software have you tried?

    The one I use will do about 5k pages an hour on a static (Movable Type) site with a decent connection. Check for broken links as well since it;s doing it outside of the software.

    90% of the pages usually scan quickly. It's the buried ones that take the time.
     
    theapparatus, Aug 29, 2009 IP
  7. Deus Ex

    Deus Ex Peon

    Messages:
    531
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #7
    What software is the best? I'll see if I can help in the direction of putting him to the paid software
     
    Deus Ex, Aug 29, 2009 IP
  8. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #8
    You can try this sitemap generator, but if it's your DB/webserver that is the cause for slowness then... Not much any crawler can do no matter how fast :) You can also try filter off URLs you don't need/want in the sitemap (e.g. forum profile pages or whatever)
     
    websitetools, Aug 30, 2009 IP
  9. anniseed

    anniseed Peon

    Messages:
    143
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I've tried Gsitecrawler (the one i'm using now) and the free google sitemap genorator but that is only limited to 500 URL's. I did try a serverside one, can't remember what it was called but it was rubbish.

    I really can't afford to buy a sitemap generator does anybody know if theres a good free one?
     
    anniseed, Aug 31, 2009 IP
  10. theapparatus

    theapparatus Peon

    Messages:
    2,925
    Likes Received:
    119
    Best Answers:
    0
    Trophy Points:
    0
    #10
    theapparatus, Aug 31, 2009 IP
  11. themoneyearth

    themoneyearth Peon

    Messages:
    55
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    themoneyearth, Sep 4, 2009 IP
  12. paulv200

    paulv200 Peon

    Messages:
    32
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    You could try thesitemapper at http://www.thesitemapper.com

    The speed of creating the sitemap does depend on things like internet connection speed, links found per page etc.
     
    paulv200, Sep 5, 2009 IP
  13. theapparatus

    theapparatus Peon

    Messages:
    2,925
    Likes Received:
    119
    Best Answers:
    0
    Trophy Points:
    0
    #13
    How quickly you pay them the 30 bucks to buy it....
     
    theapparatus, Sep 5, 2009 IP
  14. profitable

    profitable Peon

    Messages:
    28
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    what software have you tried before?
     
    profitable, Sep 12, 2009 IP
  15. edpgila

    edpgila Peon

    Messages:
    28
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #15
    I was tried too. And its work easily for me
     
    edpgila, Sep 13, 2009 IP
  16. lovecool

    lovecool Peon

    Messages:
    6
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #16
    I've been searching the tool,can everyone teel me?
     
    lovecool, Sep 14, 2009 IP
  17. wildogre

    wildogre Active Member

    Messages:
    969
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    58
    #17
    Google is also offering this service in webmaster tools.
     
    wildogre, Sep 15, 2009 IP
  18. brownman

    brownman Peon

    Messages:
    38
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #18
    It's better to use some online script, it will be much faster.
    I would recommend this one for PHP - enarion.net/google/phpsitemapng/
     
    brownman, Sep 16, 2009 IP