Googlebot is crawling my site >10.000 minutes

Discussion in 'Google' started by cuca, Mar 13, 2007.

0
  1. #1
    Hello,
    I have a small website with a forum.
    The forum url was blocked by robot.txt before but now it isn't.

    My problem, if it's a problem, is google bot is crawling my pages it's more than 10.000 minutes now :eek:
    Is it normal?
    Do I have to worry about it?
    Is googlebot lost somewhere in my forum?

    Here's the active users link:http://www.bytanza.com/forum/active_users.asp

    Guest 1 |06 Marzo 2007 alle 20:16|13 Marzo 2007 alle 21:19|10143 minuti |Search Robot|Googlebot

    Thank you in advance for your suggestions.

    Cuca
     
    cuca, Mar 13, 2007 IP
  2. TorchedSEO

    TorchedSEO Well-Known Member

    Messages:
    369
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    108
    #2
    Yea thats normal for a forum, soon I will have to start charging google rent for living on my server.
     
    TorchedSEO, Mar 13, 2007 IP
  3. exponent

    exponent Peon

    Messages:
    1,243
    Likes Received:
    60
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Google lives on one of my sites. It makes me feel kinda special.. only.. not really since it eats bandwidth.
     
    exponent, Mar 13, 2007 IP
  4. zinruss

    zinruss Notable Member

    Messages:
    3,288
    Likes Received:
    237
    Best Answers:
    0
    Trophy Points:
    270
    #4
    For forum, it is normal. I have 2-3 forums where google spider never leave the board.
     
    zinruss, Mar 14, 2007 IP
  5. Sjorritsma

    Sjorritsma Peon

    Messages:
    238
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #5
    In google Webmaster Tools you have the option to select the crawl rate.

    Set it to Slower. A slower crawl will reduce Googlebot's traffic on your server, but google may not be able to crawl your site as often.
     
    Sjorritsma, Mar 14, 2007 IP
  6. cuca

    cuca Peon

    Messages:
    6
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Hello,
    Thanks all for the suggestions.
     
    cuca, Mar 14, 2007 IP
  7. Pierce

    Pierce Active Member

    Messages:
    634
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    95
    #7
    googlebot is changing pages on my site every 30 seconds so it never ever leaves.

    Pierce
     
    Pierce, Mar 14, 2007 IP
  8. Qryztufre

    Qryztufre Prominent Member

    Messages:
    6,071
    Likes Received:
    491
    Best Answers:
    0
    Trophy Points:
    300
    #8
    Bots on your page (in most instances) are generally a good thing...and while they may get lost in your threads (or archives) they are doing what they do best...caching your pages for inclusion in their indexes.

    A google bot or two is nothing to worry about, and I generally ignore the 100+ slurps...as even they are harmless.

    You may wish to diddle your spiders list (if you are on vB then it's here: forum/includes/xml/spiders_vbulletin.xml) so that you can see who's who on a broader scale then default.
    Q
     
    Qryztufre, Mar 14, 2007 IP