1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Google Ad Bot Killing My Site. Need Help Verifying.

Discussion in 'Google AdWords' started by everett sizemor, Nov 30, 2006.

  1. #1
    We get hit pretty hard from a bot that calls itself AdsBot-Google+(+http://www.google.com/adsbot.html) with the IP: 72.14.199.65 .

    Here is the reverse DNS on that IP:
    http://www.zoneedit.com/lookup.html?ipaddress=72.14.199.65&server=&reverse=Look+it+up

    It all says Google up to this point. But then I try doing a forward look up and the tool doesn't find anything at all.

    We have a * crawl delay in the robots.txt file, but this bot doesn't seem to be paying any attention at all.

    User-agent: *
    Crawl-delay: 120

    Can anyone verify that this is or is not a google crawler? And if so, how did you go about it?
    SEMrush
    Thanks!
     
    everett sizemor, Nov 30, 2006 IP
    SEMrush
  2. T0PS3O

    T0PS3O Feel Good PLC

    Messages:
    13,220
    Likes Received:
    778
    Best Answers:
    0
    Trophy Points:
    0
    #2
    AdBot is AdWords, not AdSense. It checks landing pages for quality control. Do you run AdWords ads at all pointing to those pages? It looks like a Google IP - used by FeedFetcher before.
     
    T0PS3O, Nov 30, 2006 IP
  3. everett sizemor

    everett sizemor Peon

    Messages:
    40
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Sorry, yes I clicked on Adsense out of habbit. We do run Adwords. That's why I'm worried about blocking it because I don't want our relevancy score to go down the toilet.

    When I'm in webmaster tools checking the robots.txt file google says they just ignore the crawl delay.
     
    everett sizemor, Nov 30, 2006 IP
  4. mhdoc

    mhdoc Tauren

    Messages:
    840
    Likes Received:
    33
    Best Answers:
    0
    Trophy Points:
    0
    #4
    A few months ago a Google spider hit some of my sites and used 40-50 times the entire size of the site in bandwidth. It went on for two or three nights, stpped for a day or two, happened again, and then stopped.

    It's hard to know what to do since I hated to block Google :)
     
    mhdoc, Nov 30, 2006 IP
  5. everett sizemor

    everett sizemor Peon

    Messages:
    40
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    mhdoc - are you sure it wasn't someone spoofing that user agent? You're supposed to be able to do a reverse DNS to check for the host / domain, then a forward DNS to check the host with the original IP. It's kind of like double checking your math answers by doing the problem in reverse. The only thing is, I can't get any info from the forward lookup.

    Perhaps I'm doing something wrong. Anyone?
     
    everett sizemor, Nov 30, 2006 IP
  6. fastimprovement

    fastimprovement Peon

    Messages:
    168
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #6
    It *is* the Adwords bot checking out your relevancy.

    Thing is, it does it for *every* keyword in *every* adgroup, and every time you make a change it comes back to make sure it hasn't missed anything!!!

    Aaaaaaarrrrrggggghhhhh!

    (no more 100,000 keyword campaigns for me, thank you!)

    H
     
    fastimprovement, Nov 30, 2006 IP
  7. rustybrick

    rustybrick User ID 3

    Messages:
    384
    Likes Received:
    41
    Best Answers:
    0
    Trophy Points:
    158
  8. everett sizemor

    everett sizemor Peon

    Messages:
    40
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Thank you Barry. I see you at the conferences and read your blog, and I know you are on top of things. Thus, I do not take offense to the fact that you assume we have not read the instructions on that link over a dozen times.

    You will notice that in my first post I talk about doing a reverse DNS followed by a forward DNS. Perhaps I was doing it wrong, but the methods described at Goog Webmaster blog and the one recently by MSN do not seem to be working for me. That's why I came on here to ask for help.

    I do appologize if I sound like a guy who doesnt appreciate when someone is trying to help him, as that is not the case. Any and all help is appreciated.

    So for now I'm going to assume that it is indeed Google checking out my landing page relevancy. In that case, is there a way to get them to follow the crawl delay command in my robots.txt file?

    Kind Regards,

    Everett
     
    everett sizemor, Dec 1, 2006 IP
  9. T0PS3O

    T0PS3O Feel Good PLC

    Messages:
    13,220
    Likes Received:
    778
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Your pages are PHP I see - I started a thread here on DP in the PHP forum yesterday to assemble code that does the DNS lookups. Maybe that helps you verify them. It may well be rogue bots. Otherwise, you might have to move to a bigger bandwidth server - maybe you are outgrowing your site's home.
     
    T0PS3O, Dec 1, 2006 IP
  10. everett sizemor

    everett sizemor Peon

    Messages:
    40
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Are you looking at the fitness marketing or fitness directory sites? I'm sorry, I should have explained. Those are more my hobby sites than anything. This is for a client that runs a health and fitness ecommerce store. We get about 300,000 unique visitors per month.

    I'll go check out that post you mentioned. Thanks TOPS3O!
     
    everett sizemor, Dec 1, 2006 IP