1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Googlebot Grabbing Same Page Every 5 Minutes

Discussion in 'Site & Server Administration' started by digitalpoint, Jul 20, 2005.

0
  1. #1
    Was skimming my web logs, and noticed that Googlebot (a real one, checked the owner of the IP address) was grabbing the main page of my personal blog every 5 minutes (like on a set schedule) for the last 5 days. Has anyone else seen anything like that before?

    For example, these are the last 5 requests:

    Been going on for DAYS now...
     
    digitalpoint, Jul 20, 2005 IP
  2. Infiniterb

    Infiniterb Well-Known Member

    Messages:
    1,314
    Likes Received:
    50
    Best Answers:
    0
    Trophy Points:
    168
    #2
    Haven't seen that one yet. But if you want to point some links my way to get the bot off your blog, feel free :)
     
    Infiniterb, Jul 20, 2005 IP
  3. T0PS3O

    T0PS3O Feel Good PLC

    Messages:
    13,219
    Likes Received:
    777
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I just checked and I see the same though not for days. And not the same files either but at approx 5 min intervals.

    Just noticed that Mediabot follows redirects and requests the new page as well even though no AdSense javascript ever gets executed/parsed.
     
    T0PS3O, Jul 20, 2005 IP
  4. Design Agent

    Design Agent Peon

    Messages:
    3,061
    Likes Received:
    154
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I got the average user pulling 400+ pages each (obviously way above average) for the last couple of days in awstats.. could that be the same thing?
     
    Design Agent, Jul 20, 2005 IP
  5. prowess

    prowess Guest

    Messages:
    159
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Maybe the bot is just running into your site that often...You do have a very high value page, it could be looking for updates. I would be happy if goog hit me that much!
     
    prowess, Jul 20, 2005 IP
  6. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #6
    Been 1 1/2 weeks now, and STILL happening... what the hell dude? :)
     
    digitalpoint, Jul 24, 2005 IP
  7. Blogmaster

    Blogmaster Blood Type Dating Affiliate Manager

    Messages:
    25,924
    Likes Received:
    1,354
    Best Answers:
    0
    Trophy Points:
    380
    #7
    not on my blog, but on my blog forum it's been happening as well. Also: everytime a new thread gets created, the Google robot goes into within a minute.
     
    Blogmaster, Jul 24, 2005 IP
  8. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #8
    You are probably seeing the AdSense Mediaparter bot. This is different... it's not AdSense, and had been grabbing the same page about 400 times/day for 1 1/2 weeks now.
     
    digitalpoint, Jul 24, 2005 IP
  9. Blogmaster

    Blogmaster Blood Type Dating Affiliate Manager

    Messages:
    25,924
    Likes Received:
    1,354
    Best Answers:
    0
    Trophy Points:
    380
    #9
    It's this one: 68.142.250.140 or others starting with 68.142.25
     
    Blogmaster, Jul 24, 2005 IP
  10. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #10
    Check the user agent and you can tell.
     
    digitalpoint, Jul 24, 2005 IP
  11. Blogmaster

    Blogmaster Blood Type Dating Affiliate Manager

    Messages:
    25,924
    Likes Received:
    1,354
    Best Answers:
    0
    Trophy Points:
    380
    #11
    I check it in the vBulletin admin panel?
     
    Blogmaster, Jul 24, 2005 IP
  12. Infiniterb

    Infiniterb Well-Known Member

    Messages:
    1,314
    Likes Received:
    50
    Best Answers:
    0
    Trophy Points:
    168
    #12
    If the user agent says Mediapartners it's the Adsense bot coming around. vBulletin has the ability to show user agents, so you should be able to tell if it says that or not.
     
    Infiniterb, Jul 24, 2005 IP
  13. Blogmaster

    Blogmaster Blood Type Dating Affiliate Manager

    Messages:
    25,924
    Likes Received:
    1,354
    Best Answers:
    0
    Trophy Points:
    380
    #13
    Where do I go from here to see that ?
     
    Blogmaster, Jul 24, 2005 IP
  14. Infiniterb

    Infiniterb Well-Known Member

    Messages:
    1,314
    Likes Received:
    50
    Best Answers:
    0
    Trophy Points:
    168
    #14
    In the first drop down select search bots. In the second one select yes for user agent.
     
    Infiniterb, Jul 24, 2005 IP
    Blogmaster likes this.
  15. Blogmaster

    Blogmaster Blood Type Dating Affiliate Manager

    Messages:
    25,924
    Likes Received:
    1,354
    Best Answers:
    0
    Trophy Points:
    380
    #15
    Got it, thanks! The one I was thinking about was actually:

    68.142.251.80
    Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
    06:59 PM Guest Viewing Index

    But the other one:
    Blog Party Forum 66.249.65.40
    Mediapartners-Google/2.1
     
    Blogmaster, Jul 24, 2005 IP
  16. Infiniterb

    Infiniterb Well-Known Member

    Messages:
    1,314
    Likes Received:
    50
    Best Answers:
    0
    Trophy Points:
    168
    #16
    No prob. Mediapartners usually hits new threads the second they are posted. Shawn's issue is very, very unusual.
     
    Infiniterb, Jul 24, 2005 IP
  17. Blogmaster

    Blogmaster Blood Type Dating Affiliate Manager

    Messages:
    25,924
    Likes Received:
    1,354
    Best Answers:
    0
    Trophy Points:
    380
    #17
    No clue on that one, but if anyone finds out, I would love to find out as well :)
     
    Blogmaster, Jul 24, 2005 IP
  18. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #18
    After nearly 2 weeks, it finally stopped.
     
    digitalpoint, Jul 27, 2005 IP
  19. crazyhorse

    crazyhorse Peon

    Messages:
    1,137
    Likes Received:
    19
    Best Answers:
    0
    Trophy Points:
    0
    #19
    Maybe a engineer read your post. :D
     
    crazyhorse, Jul 27, 2005 IP