"How long should I run my experiment?" asks a ridiculously high number of online marketing managers. Typically, the answer to this question is more complicated than most want to hear. It may be a buzz phrase by now, but the goal with any experiment is to reach statistical significance--not simply to run an experiment for a pre-determined period of time. When measuring some AdWords metrics for a "winner," it can be difficult to determine when statistical significance has been reached, especially with the limitations of the AdWords Experiments feature. For any AdWords managers who are very hands-on with their accounts and want to verify statistically significant results, Delegator has provided the AdWords Statistical Significance Calculator. Whether you're testing ad copy, different landing pages, or any assortment of variables that can be separated into "Group A" and "Group B," this calculator allows you to work on actionable data alone.
Example: I am testing two separate landing pages with the exact same ad copy to see which page leads to the most conversions. I've been running my experiment for one week, and I have gathered this data so far:
Landing Page 1
- Clicks: 958
- Goal Completions: 33
Landing Page 2
- Clicks: 1,014
- Goal Completions: 45
With any test, Delegator encourages running for at least one entire week since shopping and browsing patterns can vary by day of the week. After one full week, the calculator will let you know whether your data is conclusive enough for you to act on it.
Yes, this calculator is designed specifically for AdWords account managers, but the functionality can work on any data you collect about your website through any channel. For additional testing ideas or help managing your account, give us a shout.