Google crawling but not indexing new content on high traffic site

Discussion in 'Google' started by nibblebot, Sep 21, 2009.

  1. #1
    I have a domain which gets 1M+ uniques/mo with a sitemap generated by my CMS systems. Google webmaster tools says it is downloading the sitemap daily but about 95% of the content from the past 10 days is not indexing. The site is putting out about 10 new pages a day, each of which is automatically added to sitemap. There are virtually no crawl errors or robots.txt problems.

    I was thinking that invalid HTML was the cause, but one of the articles it picked up in the past few days has plenty of invalid things happening.

    Any avenues that I may have overlooked?
    Thanks in advance!
     
    nibblebot, Sep 21, 2009 IP
  2. gr8webseller

    gr8webseller Peon

    Messages:
    1,097
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #2
    same here, i updated my blog yesterday, but still the articles are not get crawled,,,,, but before all my articles will get crawled within 1/2 hour
     
    gr8webseller, Sep 21, 2009 IP
  3. bestserviceforyou

    bestserviceforyou Peon

    Messages:
    957
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Google bots are just slipping lately. Give it a lil more time...
     
    bestserviceforyou, Sep 21, 2009 IP
  4. zebno

    zebno Well-Known Member

    Messages:
    936
    Likes Received:
    25
    Best Answers:
    0
    Trophy Points:
    138
    #4
    same is the problem with 2 of my new blogs. i buy an established domain with PR last month. uploaded a wordpress & start blogging on it.. added a robots file to root directory, submitted few of the posts to digg & facebook to get it indexed faster, submitted the site-map to webmaster tool & today it show 14/20 of pages indexed.. but yet there is no results when i search for the blog www.safekidsmaine.org

    any sugession ?
     
    zebno, Sep 21, 2009 IP
  5. technolarity

    technolarity Well-Known Member

    Messages:
    1,582
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    115
    #5
    Having exactly the same problem.
     
    technolarity, Sep 21, 2009 IP
  6. angilina

    angilina Notable Member

    Messages:
    7,824
    Likes Received:
    186
    Best Answers:
    0
    Trophy Points:
    260
    #6
    Is all the added content in the site new/fresh/unique?

    If all content is original then may be you should wait. Sometimes, Google stops to index/crawl my sites too. I just keep adding the content and try to get some backlinks and after few days, things turn normal again.
     
    angilina, Sep 21, 2009 IP
  7. aamigallery

    aamigallery Guest

    Messages:
    1,362
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #7
    The reason for this might be that your pages don't have enough backlinks.
     
    aamigallery, Sep 27, 2009 IP
  8. www.livesportz.com

    www.livesportz.com Well-Known Member

    Messages:
    280
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    103
    #8
    the same here too !

    i think soon they fix this or our sites have more troubles

    i lost some traffic too .
     
    www.livesportz.com, Sep 27, 2009 IP
  9. jagged

    jagged Peon

    Messages:
    253
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I'm having similar issues as well but i'm sure this will be resolved soon enough
     
    jagged, Sep 27, 2009 IP