Cheap Car Insurance - Free Advertising - Remortgages - Cell Phone - Flights

PDA

View Full Version : robots.txt + adsense


nevetS
Jan 4th 2005, 4:00 am
I'm developing in a sandbox right now. I need the adsense ads on my page to take a look at the overall design.

I want to allow the google adsense bot to view the page, and present me with a real ad, but I don't want the page indexed in google.

Is there a way to do this with robots.txt ?

My other options are screenshots, or manually coding it.

Slapyo
Mar 16th 2006, 9:44 am
You might try using this in the head of the page. It tells the bots not to index the page, and not to follow links.

<meta name="robots" content="noindex,nofollow">

mcfox
Mar 16th 2006, 11:12 am
You could block all bots except the ad bot.

User-agent: googleadsensebotwhosenameescapesmerightnow
Disallow:
User-agent: *
Disallow: /

Slapyo
Mar 16th 2006, 12:13 pm
Like this, the * at the end of the user-agent is a wildcard. I think the current version is Mediapartners-google/2.1. So this takes care of it if Google changes the version of the bot.

user-agent: Mediapartners-Google*
Disallow:
User-agent: *
Disallow: /