The AdWords Statistical Significance Calculator is a tool for measuring the statistical significance of goals and CTR differences between two data sets in an experiment, whether you’re testing landing pages, keywords, or ad text. The future goal is to cater this tool to as many AdWords metrics as possible, so suggestions are greatly appreciated!

Note: The Statistical Significance Calculator is currently being worked on and will return soon!

Goal Testing

When testing two different landing pages, you’ll want to use the goal calculator. Make two ads with the exact same copy to ensure that the only variable in your experiment is the landing page. With each ad having a different landing page, you can simply identify one as “Group A” and the other as “Group B” for the calculator. It is important to make sure that conversions are being pulled into your AdWords account in order to get the goal value attributed to the specific ads you are testing.

Ad Copy Testing

When testing two different ad copies, you’ll want to use both the CTR and goal calculators. The CTR calculator will let you know which copy resonates most with your audience, while the goal calculator will signify which ad is more likely to lead to your desired conversion. Make sure that the landing page for each ad is the same to ensure that the only variable in your experiment is the ad copy.

How to Use the Calculator

In your experiment, you should be testing “Group A” against “Group B.” This might mean that you’re testing a new, optimized landing page against an existing landing page or that you’re testing two variations of ad copy. Currently, this calculator accepts only two groups of data.

  1. Set the date range on your data set to capture only the time you’ve been running the experiment.

  2. Identify the metrics needed for the calculator that are associated with each respective group.

    1. If you are using the goal calculator, and your conversions aren’t being tracked in AdWords, make sure your AdWords account is linked to Analytics and go to Tools > Conversions to “Import from Google Analytics.”

    2. Input the metrics associated with each group you’re testing, and hit “Calculate” to find out if your results are significant or not.

      1. If not, you may need to run your experiment longer to reach statistical significance. If the calculator determines that your results are statistically significant, then congratulations! You’ve just completed a statistically significant experiment.