Hi folks, Ive been watching google on my vbulletin forum, but just now i noticed google is using a vast array of IP addresses, ive counted 30/40 spiders for google adsense/google search. It used to only use 3/4 ips at MOST. Has anybody else noticed this? Pierce
uh yeah, but for the last 10 months its been 2 ips, not 30/40 ips! So, i dont agree with you on that. Pierce
let me put this in perspective, this is the last 20 minutes on the forum: Google AdSense Spider 66.249.73.66 1 Minute Ago Google Spider 66.249.73.66 1 Minute Ago Google Spider 66.249.73.21 15 Minutes Ago Google Spider 66.249.65.75 15 Minutes Ago Google AdSense Spider 66.249.70.187 15 Minutes Ago Google AdSense Spider 66.249.72.170 16 Minutes Ago Google Spider 66.249.66.200 17 Minutes Ago Google AdSense Spider 66.249.65.104 17 Minutes Ago Google AdSense Spider 66.249.67.75 17 Minutes Ago Google Spider 66.249.73.226 17 Minutes Ago Google Spider 66.249.72.110 18 Minutes Ago Google AdSense Spider 66.249.73.248 18 Minutes Ago Google Spider 66.249.70.56 18 Minutes Ago Google Spider 66.249.73.167 18 Minutes Ago Google Spider 66.249.73.139 19 Minutes Ago Google Spider 66.249.73.121 19 Minutes Ago Google Spider 66.249.72.235 19 Minutes Ago Google Spider 66.249.72.172 19 Minutes Ago Google Spider 66.249.73.102 19 Minutes Ago Google Spider 66.249.66.237 19 Minutes Ago Google AdSense Spider 66.249.66.164 19 Minutes Ago Google Spider 66.249.70.60 19 Minutes Ago Google AdSense Spider 66.249.66.66 19 Minutes Ago Google Spider 66.249.73.132 19 Minutes Ago Google AdSense Spider 66.249.66.71 19 Minutes Ago Typically, theres usualy 2 adsense spiders, and 1 google spider. And it never uses more than 1/2 ips per day. Im not complaining because I can handle the traffic, I am just wondering has anybody else seen this? Pierce
so what is the problem ? if you dont want it, you can block them with your .htacess file or robots.txt file
did you read my post? Im currious to know if anybody else noticed this during a particular time of age of there site, or if somebody noticed that this happened before to them. Pierce
I don't watch IP's as closely as you but yes I have also noticed A LOT of Googlebots and Google Adsense bots on my board today, many more than usual.
Google seems to behaving like Yahoo today. This is a change from how they have behaved in the past on all of my sites.
hello Cant say much myself but have noticed the indexed pages jump up on my sleek and king so something extra was going on then they slowy dropped and leveled out but are considerably higher then what they were.. thx malcolm1
I'm seeing the same thing on my vBulletin forums: Yahoo! Slurp (119), [B]Google (36)[/B], MSNBot (2), Voyager (3), BecomeBot (1), Teoma/Ask Jeeves (1), Google AdSense (1), Gigabot (1) Code (markup):
It may be unusual for you, but if you hit a certain link popularity, or during a deep crawl, then it's quite normal. In the past 4 days I've 28,198 Gbot hits on one site, not counting the subdomain hits. -Michael
This is just the main www of my site for the last 4 days: 3 189 + 2 392 + 1 390 + 2 393 = 9 364 I think one particular day it did somewhere in the region of about 15,000 pages in a single day. So its not unusual to be constantly crawled, its just Ive never seen 30 different ip's from google at the same time. I was just wondering is it a tipping point, or is a point where you become more favoured in the serps. Pierce
i believe they have access to the stanford ip range(64.*), which is class A, which is 16million addresses and i have noticed them on a 74.* range, which is another 16 million, so no they are not short of ip addresses. But in typical habbit Google only uses 2/3 ips per site when its crawling. Pierce
Spider don't scare me. My big sites are legitimate and if you are legit then let spider run free. I am sure it will only help me out.