How do you determine the number of Ads to test at a time? If a paticular ad group only receives a few hundred impressions a month, and maybe a few clicks, is it worth testing different text? would testing 10 differnet variations be too much?
You should test as many as you can grip your brain around. My limit is 4 variations before loosing my mind. The reason is that you may suddenly find that a new ad variation starts getting 1000's of impressions on the same keywords. This can be due to the ad title, ad text, display url, and how it matches up against your landing page. So I'd say start with no more than 4, but don't be afraid to try 10 if you can handle it.
I'd focus on why you are only getting a few hundred impressions and clicks per month on your campaign(s) and less about the ad text. What is it your advertising ?
Well, thats a complete estimate, and thats PER ad group, and there are thousands. This is for very long-tail terms that are exact matches and very targeted.
According to Perry Marshall, you need 30 clicks to have statistical significance. How many ads you can work with is less important then keeping tabs on what it is that you're testing. I like to work with 2-4 ads per group. I can swap the second and third line in one pair and use a different copy or title with a similar minor variation between the two (punctuation, capitalization, word order...). Tracking too many ads in one group is overwhelming for me. Depending of how fast you're getting your clicks, the only question is how long you'll need wait before you get your 30 clicks and you know which ad to delete and which one becomes your control ad.
Once your campaign's been running for a while, and you've tried a few different adverts, the improvement that you'll see will be quite small - 30 clicks may be enough to tell a 3% clickthrough rate from a 5% clickthrough rate, but not a 5% clickthrough rate from a 5.2% clickthrough rate... Personally, if you've not got huge amounts of traffic, I'd stick to at most four alternatives - that way, you'll get some improvements immediately, instead of waiting months.
Try ~4 different Ad's for your main keywords, ones that are receiving a lot of clicks per day/week. Then use the obvious winners and run them against each other for your long tail keywords. You can test small variatians with your high traffic keywords and filter those through to your long tail keywords when you've found a marked difference. CustardMite makes a good point "30 clicks may be enough to tell a 3% clickthrough rate from a 5% clickthrough rate, but not a 5% clickthrough rate from a 5.2% clickthrough rate..." Heed CustardMite's advice
One more thought - you may want to look into Taguchi testing. This (in theory, at least) allows you to test multiple changes simultaneously, without having to test all combinations...
It's quite a complicated statistical thing. Suppose you want to test four different options on each of the four lines on your advert. To see which is the best combination of the four lines, you'd need to try 4x4x4x4 = 256 adverts. Taguchi testing says that you don't need to test all of these adverts, and that you can find out the best advert by testing certain, very specific, combinations of the alternatives. In this case, I believe you'd only need to try 16 combinations, so a test would take 1/16 of the time. It's more commonly used on landing pages, I believe, to maximise conversion rate, but it can, in theory at least, be used on advert testing.
Thanks for the heads up CustardMite. It sounds like an interesting concept to me, think I need to find out more about it. Off to go do some homework!
A friend of mine is working on a cool new site testing solution. He's looking for input from people like you who are interested in testing, but don't have the ideal solution. Can you <a href="http://www.simplysolvedit.com/testing/your_opinion_please.php">give us your input</a>? Just a few simple questions to answer...nothing spammy or scammy here...he's really just trying to get a handle on what end users want. Thanks so much! Nancy