Hi, I've been struggling getting relevant ads on my search results page. My site is about recipes and I keep getting ads about credit reports. I think part of the reason is that I've disallowed robots from crawling my search results page. I was afraid of them going crazy and making millions of queries. Will Google Mediapartners bot do this, or are my fears unfounded? The other reason why I think I'm getting irrelevant ads is because of my domain name. It includes the word "view". I tried playing around with DPs AdSense preview tool and when I drop the view from my url, all the ads are relevant. Is there anything I can do to improve things? I don't want to change domain names. And I don't think I get enough traffic to get into the premium AdSense program and target specific keywords. Filtering URLs in AdSense just seems to get me fewer ads, not relevant ones. I've also tried adding bogus extra words to my URL to see if it would help, and it doesn't seem to work consistently. Any ideas?
robots.txt User-agent: Mediapartners-Google* Disallow: User-agent: * Disallow: / Let the media bot crawl at least is one suggestion.
I've been getting these ads since I launched several months ago. At one point, I realized that I had the keyword "rating" in my url, so I got rid of it. Ads were relevant for about a week and CTR improved 100-200%! But now they are back to the credit report ones. I've tweaked and waited a while. Generally after I tweak, I don't change it back since the results are roughly the same.
Thanks for the tip, noppid. I'm a little nervous about letting the bot crawl the search results page since it literally has millions of choices of where to go next. I wonder if the Mediapartners bot is different than the regular Google bot, and it's not greedy about seeking out new pages. Maybe I should just let the bot go for a few days and see how things go.