Hello. I've been watching my stats on about 4 seperate sites, and I've seen almost no hits by Googlebot or even MediaPartners googlebot in the past 3-4 days. This is down from dozens of hits every day on each site, at a minimum. My sites are still in the Google index, many in top positions-- so I don't think I've been penalized in any way(I've done nothing to warrant that, anyways). Has anyone else seen googlebot/adsense bot lately? I want them to find a few changes I've made in the past couple days... Thanks. -- Derek
Hmm, very intersting. I am still seeing regular activity. The results have definitely been dancing across the board though.
dkalweit, keep a look out for your page counts... I'd bet that you'll start to see a reduction on the number of pages indexed (Using the API)..
I've seen page counts jumping around for weeks now. For the past week or so, they're actually more steady than they have been(and at a decent level)... Any idea what this is? It sounds like you've seen it before? -- Derek
I've seen it on a number of sites I run... I can't give you a difinitive reason as to why it's happening (Otherwise I'd not have seen it), but if I were you, I'd keep a look on the API page count, and also check your Coop weight... I suspect it may fall (If you're running in the network).
What did you do when it happened to your sites? Did it come back? If so, how long did was it gone? I watch my weight and site: counts daily, so I do see the fluctuations. Nothing too horrible yet in the past couple days. -- Derek
I haven't seen them increase to the number I was getting previously... They're slowly becoming interested again, but It's been about 3 months now, so I guess we'll see.
Care to hypothesize as to why? Maybe the site wasn't updated often enough for Googlebot's tastes? Most of my sites don't change daily, unless it's a dynamic change based on user's activity/input. The thing that really confuses me, however, is that the Google MediaPartners bot has stopped visiting-- the MediaPartners bot is the easiest, it seems, to get to visit your site-- just put the adsense code up, and it crawls within minutes of getting pageviews... I'm getting over 30k pageviews/day on these sites, and nothing... Very weird... I wonder if Google has a large segment of machines down or something-- maybe my geographical area... -- Derek
Googlebot spidering on my main sites as well as the forums decreased substantially after the Christmas update and Santy worm hits -- initially, I was concerned that the modifications to the forum software designed to thwart Santy might ne having a detrimental effect but of course that wouldn't/shouldn't have any effect on the main site. Since then, I am seeing a slow return of Googlebot and new pages being added to the index. I have no idea how to account for the decrease in spidering frequency or what else is changing with Google. Interstingly, I did not see a decrease in visits from the MediaPartners-Googlebot during this time -- in fact, the opposite was true: visits from MediaPartners increased, leading me to ask in another thread here if it was possible that the historical separation between the regular Googleb ots and the MediaPartners bot had been discontinued -- i.e., is it possible that the MediaPartners bot is also returning information for the regular Google database? If so, given that the MP bot evaluates page theme and context (in order to select appropriate ads), does this have any implications for future changes in the way Google will categorize and rank sites? Who knows?
Googlebot 3373+18 47.16 MB 03 Mrt 2005 - 12:07 MSNBot 2864+18 83.54 MB 03 Mrt 2005 - 17:59 Inktomi Slurp 1506+183 12.52 MB 03 Mrt 2005 - 17:57 Google AdSense 754+2 6.03 MB 03 Mrt 2005 - 15:35 Never seen google adsense before
crazyhorse, what stats package is that? and what is the user-agent string that resolves to "AdSense"?
Well im using awstats i ll download the log files from the server first and then let you u know what the user string is.
66.249.65.147 - - [03/Mar/2005:12:43:46 -0600] "GET /robots.txt HTTP/1.1" 200 5700 "-" "Mediapartners-Google/2.1" I think thats the one , never seen it named like this before seperately im my awstats.
Okay. It's not a new bot then -- it's just how AWStats is naming it. Did you upgrade to a new version of AWStats recently? Previously, they didn't separate Googlebot from MediaPrtners/Adsense...
Well i guess the server admin did. Advanced Web Statistics 6.4 (build 1.812), that is the current version on the server. Still it is weird that my robot text editor software noticed me that it is a new spider, nevertheless its still Google. To get back to the main topic. Im running a phpbb forum and yes i have noticed a certain decrease in spider visits since christmas although on a average of a month my site gets spidered by google around the 50k of times.
Has anyone noticed a drop in non-phpbb forums? I'm wondering if Google is doing something about phpbb forums in reaction to Santy somehow? -- Derek
I noticed seriously increased googlebot activity around 2 days ago.. For me, Usually that happens around the time of the BL Update - Which fits as there was one very such BL update a couple of days ago
Gbot hasn't left my site for more than a few hours since before the begining of the month. It's only the 4th and it's already coming up on 3000 hits from gbot. I did notice yesterday that it slowed down though. It was hitting every 60 to 90 seconds before for days on end, then yesterday it slowed way down. Now it's hitting about every half hour or so.
googlebot comes to my forum board on a regular basis. On my friends forum board googlebot is almost always there 24/7. Its very cool
This has been SOLVED! My [former] ISP, ValueWeb(http://www.valueweb.com/) finally admits to the problem. They had a router blocking Googlebot, as I suspected and tried to get them to investigate. I am in the process of switching my dedicated servers to a new ISP(Server Matrix - http://www.servermatrix.com/), and I still have a few sites on ValueWeb's network, where I can see Googlebot coming through now. Sadly, my sites have lost considerable ranking, and were in the process of de-listing entirely. ValueWeb has lost a customer, and so far I'm much more happy with Server Matrix overall-- their customer service(both human and automated) is far superior, and their network seems to be far faster and more robust, etc. All for less money per month, at that! I won't miss ValueWeb. To all those that find this in the archives or whatnot, pressure your ISP-- there's a good chance it's them if Googlebot and MediaPartners bots suddenly STOP completely... -- Derek