Large website sitemap

Discussion in 'Google Sitemaps' started by iesell, Apr 19, 2009.

0
  1. #1
    :confused:I need help creating a sitemap for a large site, exceeds 700,000 pages, I have used A1Sitemap generator as well as Keylime tie with no luck. If anyone knows of a sitemap generator which can handle a large site, your help is appreciated.
     
    iesell, Apr 19, 2009 IP
  2. DMC_Dante

    DMC_Dante Peon

    Messages:
    39
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    if you are on wordpress just use the google sitemap plugin and it should work no mather how big the site is
     
    DMC_Dante, Apr 20, 2009 IP
  3. ws01

    ws01 Member

    Messages:
    123
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    38
    #3
    try GSiteCrawler
     
    ws01, Apr 21, 2009 IP
  4. absoluteczech

    absoluteczech Guest

    Messages:
    37
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    x2 on that
     
    absoluteczech, Apr 21, 2009 IP
  5. kzenthil

    kzenthil Peon

    Messages:
    179
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I am searching for a sitemap generator for a larger site but dint find one.
    I tried http://www.xml-sitemaps.com/ , dint workout.

    Is there anything else ...
     
    kzenthil, Apr 24, 2009 IP
  6. events

    events Active Member

    Messages:
    391
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    73
    #6
    There could be a mod/ plugin for your site.. which cms is your site designed in
     
    events, Apr 24, 2009 IP
  7. themedseoservices

    themedseoservices Guest

    Messages:
    46
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Hi,

    check this link auditmypc.com here we have a option to generate sitemap online with unlimited url's generated.
     
    themedseoservices, Apr 24, 2009 IP
  8. renger1975

    renger1975 Peon

    Messages:
    87
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Testing a few right now. Will post back reviewing Gsite and audit.
     
    renger1975, Apr 24, 2009 IP
  9. newlogo

    newlogo Peon

    Messages:
    3,931
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #9
    i am also facing same difficulty for n number of products for one of site.
     
    newlogo, Apr 27, 2009 IP
  10. bass

    bass Active Member

    Messages:
    738
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    53
    #10
    I want to find large sitemap generator same ...
     
    bass, Apr 27, 2009 IP
  11. mdamin76

    mdamin76 Well-Known Member

    Messages:
    1,752
    Likes Received:
    146
    Best Answers:
    0
    Trophy Points:
    153
    #11
    try gsite crawler ( the upgrade version ) as the free version also have limit on file size.
     
    mdamin76, Apr 28, 2009 IP
  12. sastro

    sastro Well-Known Member

    Messages:
    214
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    105
    #12
    If the data came from database, create your own PHP script.
    To reduce bandwidth, you can use gz compressing with php.
    I've been using this for my website. Check it here www.pluw.com/sitemap.xml and www.pluw.com/sitemap.xml.gz
    The compressed decreased 10 times.
    Its automatic create every 24 hours. Also automatic ping to Google, yahoo, Moreover and Ask
    Check my indexed page on Google with "site:www.pluw.com"

    I can help you to create your own automatic sitemap for $30
     
    sastro, May 7, 2009 IP
  13. HomeComputerGames

    HomeComputerGames Peon

    Messages:
    871
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #13
    I had the same problem with only 150,000 pages so I wrote my own that has several sitemaps: index, categories, products and resources.

    The index is what it says, an index of my sitemaps.
    If your server would not let you genereate a large file you may have to set the php configurations.
    By default you only get a small amount of memory for a php script to use. If you surpase this you get a memory error.

    If that is not an option you can use Limit in your sql along with a startrow. So you process 50,000 entries, file saved. Next time it starts at row 50,001, create an additional file. Then repeat at 50K intervals. This gives several smaller files and the index is used to list these multiple sitemaps. This is Google correct and encouraged from what I have read.

    PS: Actuall I just looked and it goes:
    sitemapIndex.xml
    sitemapresources.xml
    sitemapcategories.xml
    sitemapproducts.xml
    sitemapproducts1.xml
    sitemapproducts2.xml

    The products-x are 50,000 pages each
     
    HomeComputerGames, May 7, 2009 IP
  14. advansys

    advansys Guest

    Messages:
    50
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    try auditmypc or gsitecrawler to get unlimited sitemap url generator but check once wheather all the generated URLs are valid
     
    advansys, May 10, 2009 IP