Expekt bonuses - Free Games - Rome hotels - Debt Consolidation - Cheat Codes

PDA

View Full Version : can this be done with google adsense bot


gdtechind
Nov 23rd 2005, 1:26 pm
i want to do something which a few ppl might find odd, but i want to stop all the robots to visit my page, but the googlebot for Adsense only.
i think it is called mediapartners googlebot, but am not sure about what should be robots.txt file for that :confused:

any help is appreciated.

thanks

ozami
Nov 23rd 2005, 2:10 pm
mediapartners-google <- thats the official name. Not sure what the command would be - need to read up on robots.txt commands

EWpro
Nov 23rd 2005, 2:34 pm
User-agent: Robot_Name
Disallow: /dir/

replace Robot_Name with the exact bot name or use wildcard '*' for all.

AFAIK there is no simple way to exclude all but one. You may need to create long list, for every robot except mediapartners-google.

Find out what robots are visiting your sites from your server log and for each bot you want to block add like this:

User-agent: Robot_Name
Disallow: /

EWpro
Nov 23rd 2005, 2:42 pm
User-agent: *
Disallow: /
User-agent: mediapartners-google
Disallow:

The above should block all except mediapartners-google. But actually it fails and block all bots that obey robot.txt - including google! So it may not help your case.

The one in my previous post is rather fool-proof.

gdtechind
Nov 23rd 2005, 3:31 pm
User-agent: *
Disallow: /
User-agent: mediapartners-google
Disallow:

The above should block all except mediapartners-google. But actually it fails and block all bots that obey robot.txt - including google! So it may not help your case.

The one in my previous post is rather fool-proof.

hmm, you said it right, it blocks many other robots as well

will use the earlier code

thanks a lot

elf98
Nov 23rd 2005, 3:38 pm
your robots.txt should contain someting like this:

User-agent: Mediapartners-Google
Disallow:

User-Agent: *
Disallow: /


First we match the mediapartners-google bot and disallows nothing.
After we have matched the adsense bot we match any bot and disallows everything!

jlawrence
Nov 23rd 2005, 3:49 pm
elf98 is correct. Allow first then disallow everything else.

gdtechind
Nov 23rd 2005, 11:59 pm
thanks a lot for that information

it works well
:D

maro
Nov 24th 2005, 12:45 am
Do you mind me asking why are you doing this?

saqib_islam
Jul 7th 2009, 8:28 pm
I think that helps us so google can not find which website are we using adsense easily and if there is any violation of terms you might get another chance to get out of it