Was skimming my web logs, and noticed that Googlebot (a real one, checked the owner of the IP address) was grabbing the main page of my personal blog every 5 minutes (like on a set schedule) for the last 5 days. Has anyone else seen anything like that before? For example, these are the last 5 requests: Been going on for DAYS now...
Haven't seen that one yet. But if you want to point some links my way to get the bot off your blog, feel free
I just checked and I see the same though not for days. And not the same files either but at approx 5 min intervals. Just noticed that Mediabot follows redirects and requests the new page as well even though no AdSense javascript ever gets executed/parsed.
I got the average user pulling 400+ pages each (obviously way above average) for the last couple of days in awstats.. could that be the same thing?
Maybe the bot is just running into your site that often...You do have a very high value page, it could be looking for updates. I would be happy if goog hit me that much!
not on my blog, but on my blog forum it's been happening as well. Also: everytime a new thread gets created, the Google robot goes into within a minute.
You are probably seeing the AdSense Mediaparter bot. This is different... it's not AdSense, and had been grabbing the same page about 400 times/day for 1 1/2 weeks now.
If the user agent says Mediapartners it's the Adsense bot coming around. vBulletin has the ability to show user agents, so you should be able to tell if it says that or not.
Got it, thanks! The one I was thinking about was actually: 68.142.251.80 Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) 06:59 PM Guest Viewing Index But the other one: Blog Party Forum 66.249.65.40 Mediapartners-Google/2.1
No prob. Mediapartners usually hits new threads the second they are posted. Shawn's issue is very, very unusual.