When Google visits

Discussion in 'Site & Server Administration' started by nadlay, Jun 28, 2004.

0
  1. #1
    How do I know when Google has visited my site?

    Is there something in my weblogs I can look for to determine that a Google spider has been at my website?
     
    nadlay, Jun 28, 2004 IP
  2. gooner

    gooner Peon

    Messages:
    56
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Yes there'll be a reference to Googlebot in your weblog.

    eg.

    ***.68.82.142 - - [20/Jun/2004:14:48:14 -0400] "GET /robots.txt HTTP/1.0" 200 51 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
     
    gooner, Jun 28, 2004 IP
  3. SEbasic

    SEbasic Peon

    Messages:
    6,317
    Likes Received:
    318
    Best Answers:
    0
    Trophy Points:
    0
    #3
    You can specify the frequency of visits either in your robots.txt or in the meta tags.

    The more links and content you have, the more frequently the googlebot will visit.
     
    SEbasic, Jun 28, 2004 IP
  4. expat

    expat Stranger from a far land

    Messages:
    873
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Sorry but I don't think so. The revisit tag is a bit outdated...

    Good site good content good links = positioning = G Y M etc come daily
    M
     
    expat, Jun 28, 2004 IP
  5. vlead

    vlead Peon

    Messages:
    215
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I also feel that the revisit tag has no or very little (if at all) influence on the GoogleBot.
     
    vlead, Jun 28, 2004 IP
  6. john_loch

    john_loch Rodent Slayer

    Messages:
    1,294
    Likes Received:
    66
    Best Answers:
    0
    Trophy Points:
    138
    #6
    Frequency in bots file ? I think thats a fooboo. You can allow/disallow, but that's about it. Of course the bots meta is another thing entirely.. :)

    If you have access to php/mysql, and your site isn't too large, you might consider robotstats (http://www.robotstats.com/en/). It covers a tidy array of bots.

    If you prefer perl, consider Relax (http://ktmatu.com/software/relax/) to post process your server logs. It generates reports etc on which search engines have visited (it recognizes over 500) popular keywords, etc. It even eats WebStar logs (for the dated Macmasters among us) - it's highly configurable :)
     
    john_loch, Jun 28, 2004 IP