As I've said before, I lack on the programming front ito knowledge and ability, but am working my way through it. I was having a big issue reading my log files, but I found a site (which most of you probably know - or don't need due to your higher levels of expertise ) where you can see the latest knowledge of the IP addresses for ALL the searchbots from most of the big engines... I'm not allowed to submit live urls' so this will have to do... ....iplists.com I think its great.. BUT if there is something better and easier, any heads up happily received?
GoogleBot visits the most than any other crawler but bandwidth consumed by it is less than that by MSN
Bots shouldn't take up much bandwidth. They only see no-script text. So images wouldn't be loaded from them.