25000 visits by Googlebot and no index.

Discussion in 'Websites' started by NewBoy, Aug 21, 2006.

  1. #1
    A company I occasionally deal with has a new site. It has only been online for less than a month and naturally has a PR0.

    It is a large, database driven site and has lots of pages about different widgets.

    Google (compatible;+Googlebot/2.1;++http://www.google.com/bot.html) visits it constantly for the past week. It visits 20000 URLs per day that it did not visit before. Over the past week it took over 100000 unique URLs.

    At the moment, only the top level of pages (around 10 pages linked from the home page) are indexed and are in the SERPs. These pages actually were in the SERPs within a week of site's launch and ranked for keywords they were intended for from day one they were included in the index.

    The question I was asked is whether this company did something wrong and Google simply ignores those thousands of pages about widgets it visits, or is it simply too early to expect the pages in the index?

    Honestly I never dealt with large sites and I am not sure how to interpret a site being online for 3 weeks or so and having been crawled at such a rate.

    Thoughs?
     
    NewBoy, Aug 21, 2006 IP
  2. NetMidWest

    NetMidWest Peon

    Messages:
    1,677
    Likes Received:
    151
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I've been tracking a spam site that was up and had some 63,000 pages indexed in less than 3 weeks. There are several others, and several past incidents where Google allowed the sites in the index.

    But since your site has not been added, I must surmise that your site is a high quality useful resource that Google will simply ignore. :D

    Seriously though, it would not suprise me to find that Google might throttle the adding of pages to the index over these incidents, spam or not, to avoid further problems.

    Time will tell, keep us updated.
     
    NetMidWest, Aug 21, 2006 IP
  3. IamNed

    IamNed Peon

    Messages:
    2,707
    Likes Received:
    276
    Best Answers:
    0
    Trophy Points:
    0
    #3
    You have to give it time. Keep gettting good inbound links and all the pages will index eventually. Not much more you can do.
     
    IamNed, Aug 21, 2006 IP
  4. linkbaitguy

    linkbaitguy Peon

    Messages:
    74
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Yes.

    Don't do a single thing.

    Just wait and wait.
     
    linkbaitguy, Aug 21, 2006 IP
  5. gudly

    gudly Guest

    Messages:
    66
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #5
    This is too Early to Predict.

    Also Please Provide Back Links For your other important pages.....

    Wait .... Wait and you will see changes
     
    gudly, Aug 21, 2006 IP
  6. soniqhost.com

    soniqhost.com Notable Member

    Messages:
    5,887
    Likes Received:
    96
    Best Answers:
    0
    Trophy Points:
    240
    #6
    I read that PageRank isn't updated that often, every couple months or so. Since your site is about a month old, it will take a bit before getting a PR.
     
    soniqhost.com, Aug 22, 2006 IP
  7. glengall1

    glengall1 Peon

    Messages:
    209
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #7

    It isn't updated very often on the toolbar but my suspicion is that it is updated constantly at Google.

    Is there not the thought that the higher the PR the more pages it will index and the deeper it will index?

    Also there is the matter of the quick sand of Google sandbox to contend with.

    So keep plugging away at highly targeted anchor text in your IBL's in relevant sites.
     
    glengall1, Aug 22, 2006 IP