How often does Googlebot crawl

Discussion in 'Google' started by Dediwebspace, Mar 22, 2007.

  1. #1
    How often does the googlebot crawl sites

    It uses one hell of alot of bandwidth and it doesnt yield any better results when you search on google

    Why does it use so much b/w
     
    Dediwebspace, Mar 22, 2007 IP
  2. rayner-x

    rayner-x Active Member

    Messages:
    183
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    53
    #2
    If your site's already indexed by Google then it will be crawled daily. Depends on how you optimize your site.
     
    rayner-x, Mar 22, 2007 IP
  3. daykevi

    daykevi Peon

    Messages:
    106
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #3
    It depends on a lot of factors
    *How many quality backlinks u have
    *How frequently u update ur content
    *The website stracture
    *Uses of session ID

    & a lot..................
     
    daykevi, Mar 22, 2007 IP
  4. thetafferboy83

    thetafferboy83 Active Member

    Messages:
    312
    Likes Received:
    72
    Best Answers:
    0
    Trophy Points:
    70
    #4
    and make sure your site has as few errors as possible.
     
    thetafferboy83, Mar 22, 2007 IP
  5. CountryBoy

    CountryBoy Prominent Member

    Messages:
    8,970
    Likes Received:
    754
    Best Answers:
    0
    Trophy Points:
    360
    #5
    Pretty regularly but if you offend it by failing to keep your content fresh it will visit less often. I suppose it visits several times a week if you update regularly, dropping down to once or twice a week if you don't.
     
    CountryBoy, Mar 22, 2007 IP
  6. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #6
    that chancges according to your backlink power . more backlink means more crawl
     
    trichnosis, Mar 22, 2007 IP
  7. Anita

    Anita Peon

    Messages:
    1,142
    Likes Received:
    51
    Best Answers:
    0
    Trophy Points:
    0
    #7
    This is one thing that Google uses PR for. A high PR site that is frequently updated could get crawled many times a day. I only have a PR5 (although I'm pretty sure it will be PR6 based on how Google is behaving), and get crawled 2-3X per day.
     
    Anita, Mar 22, 2007 IP
  8. windtalker

    windtalker Well-Known Member

    Messages:
    926
    Likes Received:
    34
    Best Answers:
    0
    Trophy Points:
    145
    #8
    This is what my personal experience is as well. My high pr sites that have not been updated for quite a while is being crawled several times a day by google.
     
    windtalker, Mar 22, 2007 IP
  9. zinruss

    zinruss Notable Member

    Messages:
    3,288
    Likes Received:
    237
    Best Answers:
    0
    Trophy Points:
    270
    #9
    Google spiders love forum or cms ( content management system ). I have a forum where spiders never leave the board.
     
    zinruss, Mar 22, 2007 IP
  10. login

    login Notable Member

    Messages:
    8,849
    Likes Received:
    349
    Best Answers:
    0
    Trophy Points:
    280
    #10
    The more PR you have the more they crawl.

    All your pages are not crawled 2-3 times a day right ?
     
    login, Mar 22, 2007 IP
  11. gogiants

    gogiants Peon

    Messages:
    357
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #11
    crawl very often
    but I don't know why so often?
    Why?
     
    gogiants, Mar 22, 2007 IP
  12. godmode

    godmode Well-Known Member

    Messages:
    4,453
    Likes Received:
    156
    Best Answers:
    0
    Trophy Points:
    190
    #12
    my site gets visited daily..and SERP changes accordingly..i notice ups and downs everyday
     
    godmode, Mar 22, 2007 IP
  13. jbladeus

    jbladeus Peon

    Messages:
    485
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    0
    #13
    all depends on the frequency of content you add.
    That is why forums generally see spiders working on them all the time.
     
    jbladeus, Mar 23, 2007 IP
  14. Dediwebspace

    Dediwebspace Active Member

    Messages:
    469
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    55
    #14
    Its just annoying
     
    Dediwebspace, Mar 23, 2007 IP
  15. NewBoy

    NewBoy Peon

    Messages:
    129
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #15
    19500 pages on average daily.

    Since we keep our site very lean in code and employ site-wide compression it only comes up to about 11K per page in traffic or around 215Mb in bot traffic.

    Sinse over 80% of our traffic comes from Google (Yahoo and MSN are very slow to accept our site and it is hosted in Europe) i have absolutely no reason to be concerned with a googlebot and actually do anything possible to make the bot's life easier. :)
     
    NewBoy, Mar 23, 2007 IP
  16. MattKNC

    MattKNC Peon

    Messages:
    2,578
    Likes Received:
    107
    Best Answers:
    0
    Trophy Points:
    0
    #16
    Be kind to Googlebot when it pays your site a visit. Make certain that your broken links are fixed, don't have a bunch of crappy code in place, and don't trick your visitors either.

    Googlebot knows when you have been naughty or nice.
     
    MattKNC, Mar 23, 2007 IP
  17. boyponga

    boyponga Banned

    Messages:
    1,013
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #17
    Depends on the backlinks you have, or how many times you update the site
     
    boyponga, Mar 23, 2007 IP
  18. megdilts

    megdilts Active Member

    Messages:
    448
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    78
    #18
    It looks like it's been crawling my every day lately. But only on some pages.
     
    megdilts, Mar 25, 2007 IP
  19. Janna122003

    Janna122003 Banned

    Messages:
    1,375
    Likes Received:
    39
    Best Answers:
    0
    Trophy Points:
    0
    #19
    It depends on your site. If you want to see how google crawl your site create an account to webmaster tools
     
    Janna122003, Mar 26, 2007 IP
  20. mickeyy

    mickeyy Active Member

    Messages:
    65
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    93
    #20
    The higher your PR and the more often you update your page(s) the more often Googlebot will crawl your site. However, Google Sitemaps does not seem to give any information about when the googlebot last crawled your site. They do tell you when they last downloaded the sitemap.xml and robots.txt file, but that seems to be about it. I would suggest either using a hosted utility or some other purchased or even freebie software to check those stats.
     
    mickeyy, Nov 23, 2009 IP