On August 3rd 2010, Google Announced: Adwords Campaign Experiments (ACE): As Google explains ACE is a free tool that makes it easier to test and precisely measure the impact of changes to your keywords, bids, Ad Groups and placements. Previously, the only way to measure the impact of these changes to a Campaign was to analyze performance before and after a change was made. The problem with this approach are all the outside factors, such as offline branding, seasonal demand or changes in what the competitors are doing that can impact results beyond the Campaign changes made. Joseph Kerchbaum wrote a great post in Search Engine Watch that describes the many uses for ACE.
Recently I used ACE to test two Landing Pages for a new Campaign my client launched. While there are other ways to test Landing Pages such as the Google Website Optimizer or rotating ads within an Ad Group, I found ACE to be a very easy way to carry out a successful simple A/B Landing Page test. This method allowed me to still easily test my ad copy and report automatically on results, which gets cumbersome with ad copy rotation. This method also did not require any special programming and only affected traffic being sent via ppc, which is not true of the website optimizer (unless duplicate pages are developed just for the test). As the post in Website Magazine points out to test more than one Landing Page or for multivariate testing the Website Optimizer is still your best bet
Below is a Step By Step Guide on how I used ACE to test Landing Pages for a new Campaign:
1. After uploading your new Campaign, duplicate all of your Ad Groups and rename them using the Adwords Editor. Keep everything on pause to start. These will be your Experiment Ad Groups. Keep everything organized by adding a modifier for each Ad Group. I added ACE in front of all of the Ad Groups. So I had two identical Ad Groups: Ad Group 1 and ACE - Ad Group 1.
2. In all of the ACE Ad Groups replace all destination URLs with the Landing Page you would like to test and upload your Campaign from the editor.
3. Now log into your Adwords account and navigate to the Campaign Settings. Within Campaign Settings scroll down to the bottom and click on the plus box. Add in the details of your Experiment. I set my Experiment to run 50/50, since it was a new Campaign and I was not worried about negatively impacting my current Campaign performance. Also add in the date you want to launch your test and the date you would like your Experiment to automatically stop running and hit save.
4. Now when you navigate back to the Ad Groups, you will see the beaker icon appear in front of all Ad Groups. Click on the arrow and a drop down menu will appear:
5. Choose Control only for all Ad Groups that do not have ACE in front of them; Choose Experiment only for all of your ACE Ad Groups. This changes the status of your Ad Groups from pause to un-pause.
6. Let your Campaigns run as you normally would but keep a few things in mind to get the best test possible:
- Make sure to make the same bid changes to the same keyword in the Experiment and Control Ad Group
- Any negatives you add, do so at the Campaign level, or to both Control and Experiment Ad Groups
- Any keywords paused or added, make sure to make the same change in the Experiment and Control Ad Group
7. Since the experiment you set up was at the Ad Group level you have to review the results at the Campaign level. Remember your test will run until the date specified in step 4, but you can review your test at any time using the Segment tab. Click on Experiment on the segment drop down
8. Find your test Experiment and you will see the performance of your Control vs. your Experiment Ad Groups as seen in the example below:
Blue Arrows indicate Statistical Significance. Google help page explains:
- One (up or down) arrow represents that there is a 5% probability >that the metric increased or decreased by chance rather than due to your experimental changes.
- Two arrows represents a 1% probability
- Three arrows represents a 0.1% probability
9. Once you have determined statistical significance you can launch the winning Ad Groups. If you find that your Experiment Ad Groups are outperforming your test Ad Groups, in account setting click on Launch Changes Fully.
One thing to note, once you apply changes then you will no longer be able to easily see the break out of performance between your Control and Experiment, only total Campaign metrics.
In my first ACE test I reported the following results:
Conversions
- +433%!
CPA
- -80.4%!
Conversion Rate
- +400%!
Three arrows were indicated for my conversions, conversion rate and CPA metric and thus there was only a .1% chance that the metrics increased or decreased by chance.
I was easily able to hit apply changes and have a major positive impact on my clients bottom line!
Happy ACE Testing!
