Allow Adsbot but deny everything else (robots.txt)?

Discussion in 'Google AdWords' started by andheresjohnny, Jul 2, 2007.

0
  1. #1
    Folks,

    Would anyone be able to show me how to code my robots.txt file to allow Adsbot but deny everyone else to my entire domain?

    I just don't know what I'm doing with that file.

    Thank you
     
    andheresjohnny, Jul 2, 2007 IP
  2. Micromag

    Micromag Well-Known Member

    Messages:
    1,757
    Likes Received:
    67
    Best Answers:
    0
    Trophy Points:
    165
    #2
    just add on your robots.txt:

    User-agent: *
    Disallow: /

    The Google adsbot will ignore disallows if you do not specify it directly.
     
    Micromag, Jul 2, 2007 IP
  3. andheresjohnny

    andheresjohnny Well-Known Member

    Messages:
    964
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    120
    #3
    I didn't know Google adsbot will ignore disallows if you do not specify it directly.

    Thank you.
     
    andheresjohnny, Jul 2, 2007 IP