Shopping news - MySpace Layouts - Watch Anime Online - Wordpress Theme - Rome hotels

PDA

View Full Version : When Google visits


nadlay
Jun 28th 2004, 4:12 am
How do I know when Google has visited my site?

Is there something in my weblogs I can look for to determine that a Google spider has been at my website?

gooner
Jun 28th 2004, 4:17 am
Yes there'll be a reference to Googlebot in your weblog.

eg.

***.68.82.142 - - [20/Jun/2004:14:48:14 -0400] "GET /robots.txt HTTP/1.0" 200 51 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

SEbasic
Jun 28th 2004, 4:22 am
You can specify the frequency of visits either in your robots.txt or in the meta tags.

The more links and content you have, the more frequently the googlebot will visit.

expat
Jun 28th 2004, 6:55 am
You can specify the frequency of visits either in your robots.txt or in the meta tags.
Sorry but I don't think so. The revisit tag is a bit outdated...

Good site good content good links = positioning = G Y M etc come daily
M

vlead
Jun 28th 2004, 6:58 am
I also feel that the revisit tag has no or very little (if at all) influence on the GoogleBot.

john_loch
Jun 28th 2004, 8:38 am
Frequency in bots file ? I think thats a fooboo. You can allow/disallow, but that's about it. Of course the bots meta is another thing entirely.. :)

If you have access to php/mysql, and your site isn't too large, you might consider robotstats (http://www.robotstats.com/en/). It covers a tidy array of bots.

If you prefer perl, consider Relax (http://ktmatu.com/software/relax/) to post process your server logs. It generates reports etc on which search engines have visited (it recognizes over 500) popular keywords, etc. It even eats WebStar logs (for the dated Macmasters among us) - it's highly configurable :)