The Search Marketers Guide to Creative Testing and Optimization, Part 4

From selecting creative elements for testing to reaching statistical significance, this four part blog series reviews basic and advanced tips for conducting a successful creative test. Last week, we discussed how to prioritize tests based on return and the importance of limiting test elements. Today, in the final part of this four part series, we’ll walk through campaign rotation settings and the importance of implementing test tracking and reaching statistical significance.

Check Campaign Rotation Settings

Ensuring even creative rotation is one of the most commonly overlooked steps when implementing a creative test. Google offers three creative rotation settings: optimize for clicks, optimize for conversions and rotate evenly. Optimizing for clicks is the default setting, but does not promote a fair test since creative that provide more clicks, are displayed more often. Opting to rotate evenly results in a fair and more statistically significant test, as each creative receives an even number of impressions. This setting also favors the utilization of key performance indicators (KPI) outside of clicks and conversions, such as conversions per impression or return on ad spend (ROAS), to determine top performing creative.

Keep in mind that Google campaigns set to rotate creative evenly will only do so for 90 days after the last creative was enabled or edited (unless opted-out of this functionality). After this 90-day period, creative will automatically optimize for clicks. This shift in rotation setting occurs regardless of whether or not statistical significance has been reached within the test. Focusing on higher volume groups will help to reach significance within the rotation period. Pausing poor performing creative and generating new creative for testing will reset this 90-day clock.

Implement Tracking Before Each Test

Day-to-day campaign management and optimization has a tendency to overshadow ongoing creative tests. Once a new test has been launched, it is critical to take note and track the where and when of each new creative. Where in the account is the creative being tested? And when was the creative activated?

With larger scale accounts, locating old creative tests can feel like searching for a needle in a haystack. Taking note of when the account, campaign and group a new creative was added can mean the difference between a successful test and one that never happened.

Reaching statistical significance in a creative test can take days, weeks and sometimes months. Even for high volume groups that reach significance faster, knowing when a new creative was added defines the date range used for reporting and analysis. In addition, low-volume groups that utilize longer date ranges are more susceptible to Google’s 90-day rotation functionality and thus require more attention. Knowing when the 90-day rotation period is nearing its end means tracking when a creative test started.

Reach Statistical Significance and Stop

Online marketers often end creative tests too early or let them run for too long. Determining a winning creative through statistical significance requires patience, and is one of the most difficult steps in a successful creative test. Achieving statistical significance leaves little doubt that a new creative outperforms the other creative in the group.

Prior to calculating significance, a KPI such as conversions per impression, CTR, conversion rate or ROAS should be selected for evaluating the performance of each creative. By definition, creative performance is statistically significant if it is unlikely to have occurred by chance. To calculate this, many creative testing tools utilize a Student’s T-Test with a user-defined confidence level. This test determines the likelihood that the difference between a single creative and the average of all creative in the group, has not occurred by chance. A confidence level between 80% and 99% is standard, but keep in mind that low-volume groups do not support high-confidence levels and are not likely to achieve statistical significance as fast as high-volume groups.

In Conclusion

Search marketers are constantly searching for ways to find and engage their target audience. No optimization strategy is more central to accomplishing this goal than creative testing. Continuously testing to find more relevant and more compelling creative serves to not only increase CTR and Quality Score, but decrease costs and drive more revenue.

When implementing creative tests, search marketers must remain disciplined at every step. New creative must remain focused and relevant. Tests must be prioritized, tracked and statistically significant. Adhering to best practices and avoiding common pitfalls will help ensure that new iterations of creative will incrementally improve account performance. Though search marketers cannot guarantee that all creative tests will be successful, they can guarantee that all creative tests have been set up for success.



Download The Search Marketers Guide to Creative Testing and Optimization for additional best practices and two case studies from BoostCTR on how they successfully test and optimize creative.