1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Too many url on my sitemap...... is that a problem

Discussion in 'Google Sitemaps' started by danieloffice, Jul 22, 2007.

  1. #1
    I an auto site that pull youtube videos.... I do a sitemap generation, and get over 10+K url on the sitemap......
    SEMrush
    Would there be any penalty or get banned if i submit this sitemap to G?
     
    danieloffice, Jul 22, 2007 IP
    SEMrush
  2. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #2
    Regarding the 10k number. You can have up to 50k pages in a single XML sitemap file. You may want to check my website sitemaps comparison article.
     
    websitetools, Jul 23, 2007 IP
  3. KiruSoft

    KiruSoft Peon

    Messages:
    97
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I think 10k is ok
     
    KiruSoft, Jul 24, 2007 IP
  4. Madmurphy

    Madmurphy Active Member

    Messages:
    341
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #4
    A sitemap with that many links in it would be a huge file, the bot may not be allowed to us that much bandwidth. you might want to break it down some just incase.
     
    Madmurphy, Jul 24, 2007 IP
  5. DearDirectory

    DearDirectory Peon

    Messages:
    90
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Break it down to 2500 a piece. It might be a safe strategy.
     
    DearDirectory, Jul 24, 2007 IP
  6. danieloffice

    danieloffice Peon

    Messages:
    472
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6

    Hi,

    You mean, after I generate the sitemap, I edit it and cut the links to 2500, then submit to Google?

    Is that any sitemap generator that we can set the number of max link to generate?

    thanks
     
    danieloffice, Jul 24, 2007 IP
  7. DaStarBuG

    DaStarBuG Peon

    Messages:
    621
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I split my sitemap after 49k links and gzip them.
    No problems here ;)

    Don't forget, if you splitt link to your sitemap index file, otherwise google wont find additional sitemaps.
     
    DaStarBuG, Jul 30, 2007 IP
  8. alignak

    alignak Peon

    Messages:
    81
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    I have several sitemaps with about 48k links, and no errors or whatever.
    Just make sure you just use url, and remove dates, or whatever from sitemap.
    If possible try gzip.
     
    alignak, Jul 30, 2007 IP
  9. danieloffice

    danieloffice Peon

    Messages:
    472
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #9


    some more info needed:
    1. why are mentioned the magic figure 48K or 48K, is that a max number?
    2. who to remove dates?
    3. for gzip files, is that sitemap.gz?

    thanks
     
    danieloffice, Jul 30, 2007 IP
  10. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #10
    I guess there are more, but A1 Sitemap Generator can.
    Anyways, the official page limit per xml sitemap file (see http://www.sitemaps.org) is 50.000 pages. (or 10 mb file size - whatever comes first)
     
    websitetools, Jul 31, 2007 IP
  11. catanich

    catanich Peon

    Messages:
    1,921
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #11
    The best one I found is http://gsitecrawler.com/ and it has no upper limit. But watch the file size (10M is the limit) but I don't know of anyone that has tested this.
     
    catanich, Aug 1, 2007 IP
  12. trichnosis

    trichnosis Prominent Member

    Messages:
    13,783
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #12
    having too many urls will not be a problem for google.

    there is only one criteria , you must limit your per sitemap with 10M.

    if you have more urls , you can create more than one sitemap
     
    trichnosis, Aug 6, 2007 IP
  13. estate agent

    estate agent Active Member

    Messages:
    300
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    73
    #13
    How many links should it be on a sitemap as maximum?
    Sorry for that but I do not know what (k) means? Is it thousend or hundred? :)
     
    estate agent, Aug 7, 2007 IP
  14. websitetools

    websitetools Well-Known Member

    Messages:
    1,508
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #14
    50k = 50 thousand, i.e. each xml sitemap file max 50.000. (Or file size 10 MB)
    But you can have multiple sitemaps listed/referenced in a sitemap index file.
     
    websitetools, Aug 7, 2007 IP
  15. estate agent

    estate agent Active Member

    Messages:
    300
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    73
    #15
    Thanks Thomas, for your explaination.
     
    estate agent, Aug 10, 2007 IP
  16. design2host

    design2host Active Member

    Messages:
    209
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #16
    its alway recommended to keep the sitemap file less than a 1000 links if you want it crawled fully. There is a Time frame for execution of each file and if your file is going to take too much of time to download, the chances are that your files are not going to be crawled. So limit your entries and increase your results.
     
    design2host, Aug 13, 2007 IP
  17. danieloffice

    danieloffice Peon

    Messages:
    472
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #17
    1000 links seems too small.................. that means only have 1000 pages indexed by Googel at the best, right?
     
    danieloffice, Aug 14, 2007 IP
  18. youngpatriot

    youngpatriot Peon

    Messages:
    33
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #18
    I have tested it on a site with around 750,000 pages. Gsitecrawler handles it just fine. It splits the sitemap for you based on size/number of links.
     
    youngpatriot, Aug 20, 2007 IP
  19. danieloffice

    danieloffice Peon

    Messages:
    472
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #19
    yes, it can handle unlimited pages, but it generate many "useless" pages...

    say, my game site only have 2000 games, so I am expecting it generate 2K url, but it goes up to 15K............... most of the "useless pages" are, for example, ......... 10 more url on the same "content" about "rating 1" "rating 2"... or so.........

    This becuase, each game could be rated by users....... so, every rating option becomes a url in sitemap.....

    Anyone knows how to get rid of those annoying pages......... ?
     
    danieloffice, Aug 20, 2007 IP