I'm a PPC Account Manager, and I'm interested in other people's thoughts about A:B testing. The problem is that, whilst there are a number of articles out there discussing how to write a good advert, and it should be very straightforward to continuously test new adverts against your current ones, keeping the better ones, it's not really quite as simple as that. I've written a number of blogs on the subject (see articles on 14th, 22nd and 28th Feb on http://blog.epiphanysolutions.co.uk/), I can find nobody else on any blogs discussing it. I've read Perry Marshall's opinions on the subject, and don't always agree with his conclusions, but I feel there must be more people out there asking the questions that I do, and coming up with their own answers. For example, should you blindly optimise based on clickthrough rates, when your conversion rates are low - since there's no way to test the impact on your conversion rate, couldn't you sabotage your campaign without realising it? Or should you run adverts for 10-100 times as long, until you're confident that the conversion rate is unaffected? My blog discusses this in more detail, but I've had no replies, and can find nobody else discussing it. Are such things commonly discussed here? Or is there somewhere else that I can look to find 'the answer'? And if not, could somebody please reply to my blogs!? I feel sure that the questions that I discuss are important to anyone who utilises a:b advert testing, so I'm sure that people must have opinions on these things. Thanks, The Mite.
Hey Mite, welcome to the forums. I think you may want to look into or discuss multi variant testing. I agree, there is not much public information or discussion going on about it. A smart PPC manager is not just a PPC manager. S/he is a "Search Engine Marketer". Split testing and MV testing using the taguchi method has numerous SEO implications too. (Organic) I'm heading out now but would like to discuss this with you later as well as maybe trade data or experiences.
Much appreciated - I'm relatively new to Search Engine Marketing, and would appreciate the chance to talk to others about it. Regarding the Taguchi method, I've been reading up about it, and it sounds like the perfect solution to the problems I found in my second blog - about the decision to run single a:b tests, or test all combinations at once and wait a long time for results. I've found a free spreadsheet that does the hard work automatically, but it has two drawbacks. It needs a minimum of seven different variables, which would be difficult to manage on a regular basis, and it says that it shouldn't be used on PPC adverts, recommending instead the Mahalanobis-Taguchi system. I can find nothing on the internet about this system, nor anything useful about how to apply the Taguchi system on PPC - just a number of companies who'll do it for you in return for a big pile of cash. We can't really afford to have somebody else doing the work for us, as with 20+ accounts, each with a number of adgroups, and hence adverts, the cost would be prohibitive. Any thoughts would be greatly appreciated, The Mite
Curious how many have used those automated Taguchi software out there and found anything remarkable in ROI increase?
I'm away from the office right now but will PM you when I return. Google has a multi variant testing tool integrated into google analytics called "Website Optimizer" and it is an AMAZING tool. Best of all it's completely free. It's only in beta testing now. You might get an invite here: http://services.google.com/websiteoptimizer/ I was using Vertster and the Google tool is MUCH better. I'll give you a shout when I get back.
Sadly, we have no control over the content of the websites that we advertise (other than to make occasional suggestions), as we only manage the advertising campaigns for our clients. So my interest at the moment is limited to PPC advert text. After reading your comment about the Taguchi method (which I hadn't heard of), I've done some research into it, and it'll certainly help a lot. I've built a spreadsheet to generate tests for a specified number of variables and values, and then to analyse the results (didn't see any reason to buy one - it's not really all that complicated). As before, any thoughts/opinions/ideas would be very much appreciated (as Johnny 5 would say - NEED INPUT!). Thanks, The Mite