1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

google unleashes spiders?

Discussion in 'Google' started by Pierce, Jun 4, 2007.

  1. #1
    Hi folks,

    Ive been watching google on my vbulletin forum, but just now i noticed google is using a vast array of IP addresses, ive counted 30/40 spiders for google adsense/google search. It used to only use 3/4 ips at MOST.

    Has anybody else noticed this?

    Pierce
     
    Pierce, Jun 4, 2007 IP
  2. danielro

    danielro Active Member

    Messages:
    187
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    58
    #2
    This is just a normal thing.
    Diffrent data centers have diffrent robots with diffrent ips :)
     
    danielro, Jun 4, 2007 IP
  3. Pierce

    Pierce Active Member

    Messages:
    634
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    95
    #3
    uh yeah, but for the last 10 months its been 2 ips, not 30/40 ips!

    So, i dont agree with you on that.

    Pierce
     
    Pierce, Jun 4, 2007 IP
  4. Pierce

    Pierce Active Member

    Messages:
    634
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    95
    #4
    let me put this in perspective, this is the last 20 minutes on the forum:

    Google AdSense Spider 66.249.73.66 1 Minute Ago
    Google Spider 66.249.73.66 1 Minute Ago
    Google Spider 66.249.73.21 15 Minutes Ago
    Google Spider 66.249.65.75 15 Minutes Ago
    Google AdSense Spider 66.249.70.187 15 Minutes Ago
    Google AdSense Spider 66.249.72.170 16 Minutes Ago
    Google Spider 66.249.66.200 17 Minutes Ago
    Google AdSense Spider 66.249.65.104 17 Minutes Ago
    Google AdSense Spider 66.249.67.75 17 Minutes Ago
    Google Spider 66.249.73.226 17 Minutes Ago
    Google Spider 66.249.72.110 18 Minutes Ago
    Google AdSense Spider 66.249.73.248 18 Minutes Ago
    Google Spider 66.249.70.56 18 Minutes Ago
    Google Spider 66.249.73.167 18 Minutes Ago
    Google Spider 66.249.73.139 19 Minutes Ago
    Google Spider 66.249.73.121 19 Minutes Ago
    Google Spider 66.249.72.235 19 Minutes Ago
    Google Spider 66.249.72.172 19 Minutes Ago
    Google Spider 66.249.73.102 19 Minutes Ago
    Google Spider 66.249.66.237 19 Minutes Ago
    Google AdSense Spider 66.249.66.164 19 Minutes Ago
    Google Spider 66.249.70.60 19 Minutes Ago
    Google AdSense Spider 66.249.66.66 19 Minutes Ago
    Google Spider 66.249.73.132 19 Minutes Ago
    Google AdSense Spider 66.249.66.71 19 Minutes Ago

    Typically, theres usualy 2 adsense spiders, and 1 google spider.

    And it never uses more than 1/2 ips per day.

    Im not complaining because I can handle the traffic, I am just wondering has anybody else seen this?

    Pierce
     
    Pierce, Jun 4, 2007 IP
  5. SFOD_D223

    SFOD_D223 Peon

    Messages:
    4,512
    Likes Received:
    174
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I'm glad I don't have to worry about the damn "G" spiders.
     
    SFOD_D223, Jun 4, 2007 IP
  6. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #6
    so what is the problem ? if you dont want it, you can block them with your .htacess file or robots.txt file
     
    trichnosis, Jun 4, 2007 IP
  7. Pierce

    Pierce Active Member

    Messages:
    634
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    95
    #7
    did you read my post?

    Im currious to know if anybody else noticed this during a particular time of age of there site, or if somebody noticed that this happened before to them.

    Pierce
     
    Pierce, Jun 4, 2007 IP
  8. EGS

    EGS Notable Member

    Messages:
    6,078
    Likes Received:
    438
    Best Answers:
    0
    Trophy Points:
    290
    #8
    Ditto..
    [size=.0001]Test[/size]
     
    EGS, Jun 4, 2007 IP
  9. RobertD

    RobertD Peon

    Messages:
    542
    Likes Received:
    25
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I don't watch IP's as closely as you but yes I have also noticed A LOT of Googlebots and Google Adsense bots on my board today, many more than usual.
     
    RobertD, Jun 4, 2007 IP
    Pierce likes this.
  10. Code Monkey

    Code Monkey Active Member

    Messages:
    248
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    60
    #10
    Google seems to behaving like Yahoo today. This is a change from how they have behaved in the past on all of my sites.
     
    Code Monkey, Jun 4, 2007 IP
    Pierce likes this.
  11. malcolm1

    malcolm1 Prominent Member

    Messages:
    7,148
    Likes Received:
    758
    Best Answers:
    0
    Trophy Points:
    310
    #11
    hello

    Cant say much myself but have noticed the indexed pages jump up on my sleek and king so something extra was going on then they slowy dropped and leveled out but are considerably higher then what they were..



    thx
    malcolm1
     
    malcolm1, Jun 4, 2007 IP
    Pierce likes this.
  12. oseymour

    oseymour Well-Known Member

    Messages:
    3,960
    Likes Received:
    92
    Best Answers:
    0
    Trophy Points:
    135
    #12
    I noticed the smae thing recently but I didn't give it much thought
     
    oseymour, Jun 4, 2007 IP
    Pierce likes this.
  13. SticKer

    SticKer Well-Known Member

    Messages:
    2,393
    Likes Received:
    78
    Best Answers:
    0
    Trophy Points:
    115
    #13
    yes google spiders can been seen more than they used to be on the site recently
     
    SticKer, Jun 4, 2007 IP
    Pierce likes this.
  14. BamaStangGuy

    BamaStangGuy Notable Member

    Messages:
    955
    Likes Received:
    51
    Best Answers:
    1
    Trophy Points:
    245
    #14
    I'm seeing the same thing on my vBulletin forums:

    Yahoo! Slurp (119), [B]Google (36)[/B], MSNBot (2), Voyager (3), BecomeBot (1), Teoma/Ask Jeeves (1), Google AdSense (1), Gigabot (1)
    Code (markup):
     
    BamaStangGuy, Jun 4, 2007 IP
    Pierce likes this.
  15. MrX

    MrX Well-Known Member

    Messages:
    1,563
    Likes Received:
    77
    Best Answers:
    0
    Trophy Points:
    140
    #15
    Same on mine today. Shot up outta no where like crazy. Then settled and was back to normal.
     
    MrX, Jun 4, 2007 IP
    Pierce likes this.
  16. mvandemar

    mvandemar Notable Member

    Messages:
    2,409
    Likes Received:
    307
    Best Answers:
    0
    Trophy Points:
    230
    #16
    It may be unusual for you, but if you hit a certain link popularity, or during a deep crawl, then it's quite normal.

    In the past 4 days I've 28,198 Gbot hits on one site, not counting the subdomain hits. :)

    -Michael
     
    mvandemar, Jun 4, 2007 IP
  17. Pierce

    Pierce Active Member

    Messages:
    634
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    95
    #17
    This is just the main www of my site for the last 4 days:

    3 189 + 2 392 + 1 390 + 2 393 = 9 364

    I think one particular day it did somewhere in the region of about 15,000 pages in a single day. So its not unusual to be constantly crawled, its just Ive never seen 30 different ip's from google at the same time.

    I was just wondering is it a tipping point, or is a point where you become more favoured in the serps.

    Pierce
     
    Pierce, Jun 5, 2007 IP
  18. GeorgR.

    GeorgR. Peon

    Messages:
    2,831
    Likes Received:
    78
    Best Answers:
    0
    Trophy Points:
    0
    #18
    google has hundreds of IPs.....
     
    GeorgR., Jun 5, 2007 IP
  19. Pierce

    Pierce Active Member

    Messages:
    634
    Likes Received:
    26
    Best Answers:
    0
    Trophy Points:
    95
    #19
    i believe they have access to the stanford ip range(64.*), which is class A, which is 16million addresses

    and i have noticed them on a 74.* range, which is another 16 million, so no they are not short of ip addresses.

    But in typical habbit Google only uses 2/3 ips per site when its crawling.

    Pierce
     
    Pierce, Jun 5, 2007 IP
  20. depotgang

    depotgang Peon

    Messages:
    530
    Likes Received:
    20
    Best Answers:
    0
    Trophy Points:
    0
    #20
    Spider don't scare me. My big sites are legitimate and if you are legit then let spider run free. I am sure it will only help me out.
     
    depotgang, Jun 5, 2007 IP