Googlebot visiting your site?

Discussion in 'Site & Server Administration' started by cmonline, Jul 26, 2006.

0
  1. #1
    As I've said before, I lack on the programming front ito knowledge and ability, but am working my way through it. I was having a big issue reading my log files, but I found a site (which most of you probably know - or don't need due to your higher levels of expertise:eek: ) where you can see the latest knowledge of the IP addresses for ALL the searchbots from most of the big engines...

    I'm not allowed to submit live urls' so this will have to do...
    ....iplists.com

    I think its great.. BUT if there is something better and easier, any heads up happily received?:D
     
    cmonline, Jul 26, 2006 IP
  2. redz

    redz Well-Known Member

    Messages:
    3,095
    Likes Received:
    114
    Best Answers:
    0
    Trophy Points:
    105
    #2
    they visit alot and use alot of my bandwidth on my bigger sites, i am not liking that lol.
     
    redz, Jul 26, 2006 IP
  3. ravianz

    ravianz Notable Member

    Messages:
    1,536
    Likes Received:
    55
    Best Answers:
    0
    Trophy Points:
    250
    #3
    GoogleBot visits the most than any other crawler but bandwidth consumed by it is less than that by MSN
     
    ravianz, Jul 27, 2006 IP
  4. shamess

    shamess Well-Known Member

    Messages:
    1,127
    Likes Received:
    25
    Best Answers:
    0
    Trophy Points:
    185
    #4
    Bots shouldn't take up much bandwidth. They only see no-script text. So images wouldn't be loaded from them.
     
    shamess, Jul 27, 2006 IP
  5. tonyinabox

    tonyinabox Peon

    Messages:
    1,988
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Yahoo visit my site 150 times another bot.
     
    tonyinabox, Jul 27, 2006 IP
  6. MN Sandy

    MN Sandy Guest

    Messages:
    392
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Use sitemaps to minimize spidering frequency;)
     
    MN Sandy, Jul 28, 2006 IP
  7. shamess

    shamess Well-Known Member

    Messages:
    1,127
    Likes Received:
    25
    Best Answers:
    0
    Trophy Points:
    185
    #7
    Google Sitemaps? Surely that would only stop Google's robots, ne?
     
    shamess, Jul 28, 2006 IP
  8. Muffin_man

    Muffin_man Peon

    Messages:
    137
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Can Yahoo and the other bots read google site maps?
     
    Muffin_man, Jul 28, 2006 IP
  9. shamess

    shamess Well-Known Member

    Messages:
    1,127
    Likes Received:
    25
    Best Answers:
    0
    Trophy Points:
    185
    #9
    Nope, not as far as I know.
     
    shamess, Jul 28, 2006 IP