From sporadic to systematic. Launching a testing program that drives results.

April 21, 2021 Stan

“Our team is really good at A/B testing,” said no marketer ever. Okay, maybe I’m exaggerating, but you get the point. I rarely find marketers that have implemented a strong testing program, and instead, I more often hear from marketers who understand the importance of testing, but admit that it hasn’t been a priority. It’s a “nice-to-have” and, it is on the roadmap for when there is extra — extra time, money, resources, energy — only executed when teams can afford to do so. However, with increased pressure for marketing teams to perform and deliver an ROI, I argue that teams can’t afford not to test campaigns.

With a little effort and discipline, teams can incorporate successful testing into existing processes without significant investments of time, money, resources, or energy. Here’s how.

Learn to write a good hypothesis

A good test always starts with a solid hypothesis — that is, a statement of expectation or prediction that will be tested and validated. It should always be written as a statement, and never a question. Here’s an example: “Changing the CTA button color from blue to red, will increase conversion rate because red is more eye catching.” A hypothesis should be made up of three parts — the variable, the expectation, and the rationale behind it. Let’s break this down a bit further.

The Variable. The first part of a hypothesis defines the variable that is changing. In our example, it’s the color of the button. This is important to include because it defines the difference between version A and version B which later will provide insight into why one version “won” the test.

The Expectation. The second part of a hypothesis should state the expected outcome. In our example, the expectation is that our change will result in an increased conversion rate. This is important to include because it defines the metric that will be used to determine a “winner” — the conversion rate. Other metrics like bounce rate, time on page, or secondary clicks will be interesting and insightful, but in our example, the primary metric to declare a winner will be the conversion rate.

The rationale. The last part of your hypothesis starts with the word “because” and it provides a reasonable explanation as to why the outcome is to be expected. This is often overlooked or forgotten, but it’s a critical part of developing a strong hypothesis. It forces marketers to base testing on rationale and to build on what’s already been learned — ensuring tests aren’t based on random ideas or indecisiveness. In less mature organizations, I often see teams test two subject lines because they couldn’t decide which one they liked more — the problem with this is that one would win, but nobody knows why it won, and there’s not much learning.

A solid hypothesis is the foundation of a test, and the strength of the hypothesis will determine the potential for marketing improvement.

Build it into the processes

Creating a culture of testing doesn’t just happen. It doesn’t take tons of time and energy, but you have to be intentional. In order to really make testing a practice within an organization, it must be built into the process — only then does it become a habit, a requirement, and a non-negotiable. Here are a few quick ways to build it into your already existing processes:

Add it to your creative brief. If you’re already using a creative brief to inform the design phase of a campaign, include a section about testing. Have marketers identify what part they want to test like the subject line or call-to-action, and provide information about the desired test.
Share results, and do it on a cadence. Whether it’s weekly or monthly, commit to sharing test results with the greater team. If possible, carve 5-10 minutes out of an existing meeting or add a section to an existing internal newsletter to share the results and valuable lessons learned. This creates an opportunity for knowledge sharing, builds momentum and excitement around testing, and keeps the team accountable to testing.
Templatize your tests. Whether it’s an email or webpage test, create new templates as new tests are developed so that similar tests can be re-run with minimal effort. This helps remove roadblocks, making it easier for the team responsible for executing the test and less likely that a lack of time, money, resources, or energy be a reason to not perform any given test.

Inciting change within an organization is hard, but it’s twice as hard to get people to adopt a new process or add a new program, than it is to work new elements into the existing process or program. Find ways to incorporate triggers for testing into your existing process and you’ll see momentum and inertia take over.

Track Results to Inform Future Campaigns

With so much technology at our fingertips, teams are capable of pivoting at the drop of a hat. With a flip of a switch, web traffic can immediately be directed to the higher performing version the moment a winner of a test is declared. The benefit is immediate and we like that. But did we learn anything? Was the pivot so fast that we missed the opportunity to learn from the test? Are we risking the opportunity to base future tests off of previous learnings? Speed is required in order to reach optimal results, but that doesn’t mean we have to skip the “human learning” part.

In order to get the most from testing programs, teams must record results from previous tests, evaluate them for learnings, and build on them for future planning. We don’t need a 10-page report on every test executed — a simple spreadsheet that includes information on what was tested and the result is enough. The act of recording this information forces us to pause and take in what we just learned, and as your spreadsheet grows with more results, you’ll start to recognize themes and commonalities. As more variables are tested and data on the results are collected, testing programs will improve and reap the benefits.

As technology continues to advance, the value marketing brings to an organization is more quantifiable than ever before, allowing teams to measure their efforts. With more visibility and transparency, comes a greater pressure to optimize every penny spent. Baseline metrics have been set, and teams are working hard to increase the ROI of every effort. A testing plan is no longer a “nice to have,” but rather it’s a “must have” in order for teams to reach their full potential. Whether you need help getting started, or are looking for ways to improve an existing program, drop us a line with all your questions and we can help get you to where you want to be!

Thank you for subscribing!