sitemap experts needed? what the hell is going on ...

Discussion in 'Google Sitemaps' started by darrens, Sep 14, 2008.

  1. #1
    Hi Guys,

    Ok im at the point of finishing i ... what the hell is going on with google site maps. I have looked in google webmaster tools and i have no errors and they tell me everything is good.
    This is my problem ....

    1) launched a new site approx 18 months ago that contained 6,000 unique pages ... no duplicate content.
    2) created and published a sitemap and submitted it to google - within 4 weeks they were into my site and indexing.
    3) I saw the indexing going up between 50-400 pages every few days and was very pleased.
    4) suddenly everything stopped and google had approx 5k out of my 6k pages indexed.
    5) waited, waited and waited for them to come back to get the final 1k pages but nothing.
    6) noticed some reported errors in my webmaster tools for URLs that were unavailable so i fixed these but still nothing , no new indexing.
    7) my site map is now at 11,000 pages but still google will not index anything? i know its not sand boxed as they send me traffic and i can find my site on page one for a few of my products.

    why is my sitemap stuck around the 5k mark? all seems to have stoped when google found errors in a few of my urls? but these are fixed and have been for months so why can i not get them to continue to index?
    I have even deleted the sitemap and resubmitted it trying to kick start google.

    suggestions please....
     
    darrens, Sep 14, 2008 IP
  2. catanich

    catanich Peon

    Messages:
    1,921
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I have been there and done that...

    You have a dynamiclly created database driven site. Correct? The template that you use is a file. And a file has a "last modified date" attribute. you have edited the database data, the not the file(s). Therefore the sitemap is telling google that there is no change to the "last modified date".

    In addition, verify that the "common code" components (header, footer and left nav) don't exceed 50% of the code. Otherwise it can be considered "duplicate content"

    Also, it would help if you supply the domain.
     
    catanich, Sep 15, 2008 IP
  3. darrens

    darrens Peon

    Messages:
    808
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #3
    hi Jim,
    Thanks for the reply ...
    Can you explain more on
    The template that you use is a file. And a file has a "last modified date" attribute. you have edited the database data, the not the file(s). Therefore the sitemap is telling google that there is no change to the "last modified date".
    my site map is created at 6am each morning? I can see that google downloads it almost every other day?
     
    darrens, Sep 16, 2008 IP
  4. kmap

    kmap Well-Known Member

    Messages:
    2,215
    Likes Received:
    29
    Best Answers:
    2
    Trophy Points:
    135
    #4
    goog can include or exclude any pages of your site

    Normally if site pages are similar google try to ignore them like generated by a script
    Some times Url create problem like if you are using very long URL
    sometimes you use similar looking url that can cause this

    Content is king , Key to success


    Regards

    Alex
     
    kmap, Sep 16, 2008 IP
  5. darrens

    darrens Peon

    Messages:
    808
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #5
    just noticed that my site map in webmaster tools is now at 4,000 mark???
     
    darrens, Sep 16, 2008 IP
  6. darrens

    darrens Peon

    Messages:
    808
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #6
    thanks kmap.

    although my pages are in the same structure they each contain unique content?
     
    darrens, Sep 16, 2008 IP
  7. JamesMichael Solutions

    JamesMichael Solutions Peon

    Messages:
    52
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I have alwasy read that you shouldn't exceed more than 1000 links per site map? Maybe try breaking them down into multiple maps, then maybe it will be indexed faster. Goodluck.
     
    JamesMichael Solutions, Sep 23, 2008 IP
  8. websitetools

    websitetools Well-Known Member

    Messages:
    1,513
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #8
    I think you are referring to HTML sitemaps / 100 links per page... With XML sitemaps for search engines it's 50.000 URLs per file following their own defined protocol. You may want to check this sitemaps page that has explanation and examples of all sitemap file kinds.
     
    websitetools, Sep 23, 2008 IP
  9. JamesMichael Solutions

    JamesMichael Solutions Peon

    Messages:
    52
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Thanks. Good read.
     
    JamesMichael Solutions, Sep 23, 2008 IP
  10. darrens

    darrens Peon

    Messages:
    808
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #10
    ok ... problem is back and it seems to be happening in cycles ...

    1) submit a XML site map and google webmaster tools tell me it fine.
    2) indexed to 6,000 pages but the site map contains approx 12,000
    3) stays at 6,000 for maybe 4 weeks
    4) at the 5 week the indexed 6,000 start to drop.
    5) week 6 the indexed pages is down to 3500 only ... down almost 50%
    6) week 10 the indexed pages starts to climb and within 2 weeks we are back to 6,000

    This then goes back to step 3 and starts all over again.
    My 2 problems ...

    1) at the best of times i have 6,000 indexed pages but i have 12,000 in the site map?
    2) why do i keep losing 50% of my indexed pages and then regaining them?

    suggestions?
     
    darrens, Sep 25, 2008 IP
  11. JamesMichael Solutions

    JamesMichael Solutions Peon

    Messages:
    52
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    What is your URL? What could you possibly have on 12,000 pages.
     
    JamesMichael Solutions, Sep 25, 2008 IP
  12. darrens

    darrens Peon

    Messages:
    808
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #12
    would love to give the url but im on a NDA with my client and im not allowed to discuss (pain in the a**) i have asked them but they have a marketing fool who for some reason thinks it would be a problem if i discussed with other industry people???

    12,000 pages are made up of 12,000 products the site sells ... each has a different description and most have images.
     
    darrens, Sep 26, 2008 IP
  13. darrens

    darrens Peon

    Messages:
    808
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #13
    any suggestions?

    in the last week my traffic has dropped to 1 unique a day? where normally i average between 50-60per day?
     
    darrens, Sep 28, 2008 IP