Email A/B Tests You Ought To Be Doing Right Now
Email marketing continues to be among the most reliable methods of engaging your customers. And like any investment, you want to see your programs continually improve and yield greater impact over time.
How can this be accomplished? A/B testing your emails.
No matter how much experience you may have with email, no one can assume that the solution we’ve used before will always perform successfully. The moods and vibe of our culture are continually evolving. A tone that worked pre-COVID often failed during the pandemic, and what eventually worked in 2020 may not work in the months and seasons to come.
The possibilities of what you can A/B test are nearly endless, so we’ve assembled our thoughts on where to start.
How do you plan an email A/B test?
Let’s first start with what you need to consider when planning an A/B split test. When you plan an A/B test, here are five steps to follow.
- Review past evidence and assumptions. Ask yourself, “what do we know and/or think we know already?” Look at previous campaign results. Compare them with previous goals. Where are you falling short? What has worked well?
- Form a hypothesis. A good structure is as follows: If we take this action, then this result will happen, because of this evidence or assumption.
- Document your success criteria. Choose one number to measure, like open rate, click-through rate, open-to-click rate, or click-to-conversion rate. Make it reasonable and achievable. Aim for incremental growth – skip the home runs.
- Prioritize your test variables. Choose one specific thing in the communication that will change between A and B. It’s good to be precise and use restraint so you can learn as you go.
- Consider all risks. Ask yourself what could go wrong by doing this test? For instance, if you’re trying new things outside your normal brand voice – consider limiting the test to a smaller number of subscribers than normal.
As you prepare to build your own A/B test plan, here are a few of our favorite elements of an email that you can test.
Subject Lines
We always suggest starting off by trying to improve your open rates. If you don’t get the open, then all the effort you put into creating the perfect content won’t matter.
Subject lines are just that – short introductions to what your email is all about, and the only part of your email that is guaranteed to be seen.
They’re also one of the most impactful (and simplest) elements in your email to A/B test.
For instance, consider testing emotionally-packed words against more direct and technical phrasing.
You might use different ways of phrasing the same “Call to Action” within your subject line, comparing emotionally-charged words, such as “love” or “celebrate” with more functional words such as “spend” or “grab” to see if your audience appreciates directness in their emails. It’s not shocking that people respond to emotion, huh?
Tone matters too. As you move along the customer journey, different tones will resonate based on the mood of your subscriber. We really like Nielsen Norman Group’s article on the “Four Dimensions of Tone of Voice.” Subject lines are great spots to test their 4 primary tone-of-voice dimensions, which are:
- Funny vs Serious
- Formal vs Casual
- Respectful vs Irreverent
- Enthusiastic vs Matter of Fact
You may think you know the right tone for your product during each step of the email marketing lifecycle, but your instinct will be nothing more than an untested hypothesis until you find out for sure.
Another test idea – see if reversing the order of your subject line has any impact on your open rates. Consider the example of “Get 20% off your next order when you create an account” compared to “Create an account and get 20% off your next order.”
Is the discount amount in the leading position more enticing to your customers, or do they follow the action of creating an account more reliably when that is the first thing they read?
And finally, with differing screen sizes, text sizes, and inboxes that people are using, there may not be enough space for your entire subject line to be displayed consistently. Testing how the length of your subject line may impact performance compared to what devices your audience tends to use can guide your efforts moving forward. Be sure to keep the message similar, to avoid any difference being attributed to completely different subject lines.
Preview/Preheader Text
Preview text, sometimes called the preheader, is shown alongside your subject in many email inboxes. Preview text is often used to reinforce what your subject line is conveying, but there’s more than one way to accomplish this. Your preview text might add an additional piece of information that your audience cares about to further interest them in opening your email (though, we’d recommend keeping anything truly crucial in your subject line). Or you might have a small CTA to drive home what action you want them to ultimately take.
Your preview text should be short and sweet, and work in tandem with the subject line of your email to pique the interest of your audience and entice them to open your email and engage with your content. Through A/B testing, you’ll be able to find how your preview text can do this most effectively.
Call-to-Action
Every email you send should have a purpose, and your Call-to-Action needs to be consistent with that purpose and give the reader a clear direction to follow once they finish reading.
The language that you use in your Call to Action can make or break how much of your audience decides to click through and take your desired action.
In general, people will respond more consistently to active and action-oriented language that instructs them on exactly what they’ll be doing, compared to more passive or generic language (i.e., “See the views” vs “See more”).
For instance, “Learn More” is a standard go-to CTA for many email marketers, but it’s rarely more effective than stronger action-led CTAs like “Inquire Today” or “Reveal the Deal” or “Buy Tickets.”
Testing what language resonates with your audience will give you valuable insight into how you can construct your emails and fashion your CTAs to increase engagement and revenue from your subscribers.
We’d also like to suggest never using “Click here” for your CTA, as it’s a wasted opportunity to describe the benefit of clicking on the link. You’ll also fail to clearly set expectations on what will happen when the CTA is clicked.
Which leads us to the next tip – you can also test the design of your call to action.
A button that stands out as a clear CTA tells the reader what they should be doing next. Buttons work great when there’s a single, actionable step that follows your email (make a purchase, sign up for an account, activate a service, etc.).
However, you may also consider using standalone text links instead, depending on your content and what your goal is. Text links work great as a “low pressure” method to provide the reader a path to continue learning more about your content. Often, text links can be more visually appealing as well, such as cases where you may be providing multiple references to places where the reader can find out more information.
Send Time (or Send Day)
Competition for your subscribers’ attention is tough. Their attention and time is at a premium, especially with the massive volume of content placed in front of them each day. To help your content stand out, you’ll want to test when your audience may be most likely to engage with any email you send.
By A/B testing different times for your emails to be delivered, you can find when your audience is most likely to have the bandwidth to not only open your email, but engage with the content you’re sharing. Most email platforms offer send-time optimization services that can be utilized to schedule your emails to be sent at effective times, as well.
If your email service provider offers a send-time optimization service, we suggest trying it out. Different subscribers are on different schedules, and some may be more likely to scan their inbox early in the morning versus mid-afternoon.
Email Design
Here’s an existential question to consider: Should your email lead with an image, or capture interest with a headline first?
In some cases, images provide added context and understandability to the message you’re sending, or reinforce the benefit to be received. While in other cases, images get in the way of what needs to be an urgent transactional headline (“Your Account Is Overdue”).
Trying out whether to lead with an image or a headline is one way to test what captures the attention of your audience.
Another test to try out is the “type” of image you may use to see what garners more engagement. Visual representations of what your copy is expressing (e.g., a screenshot of your mobile app) may help solidify the interest of your audience more than an “emotional” representation (such as using a cheery yet tangentially-related graphic).
Perhaps your audience doesn’t need a bright, colorful, energetic design and would respond to a more muted palette? Muted palettes work well in transactional types of emails, such as a receipt or delivery related message, in which the recipient only needs a few sober pieces of crucial information that they’re expecting.
Take a look at the color palette of your own organization. How would you translate it into moments where a more energetic interpretation of the palette may get higher engagement? Perhaps moments of celebration (like an anniversary email)? And then consider when a quieter and more muted palette would be useful (transactionals, receipts, onboarding).
Copy Length is Also Worth Testing
Your emails should always be easily skimmable to allow for a quick understanding of the message you’re sending. However, you may need to try out different lengths of your copy to see when it becomes too much, and your audience starts to lose interest before reaching the CTA.
Especially in promotional emails, it’s worth thinking of email like a school bus. It’s not the email’s job to teach you anything, it’s only job is to pick you up and drop you off at a place (website or app) where you can engage with the brand.
The only way to really know? Test it.
Mobile Friendliness
Another important aspect of your email design, and one that can be A/B tested, is whether your design should cater more to desktop or mobile devices.
Depending on your audience analytics, you may find that the majority of the recipients you’re sending emails to are opening on a specific type of device. That said, it’s still worth testing your assumptions.
By finding which style of design (desktop vs mobile) produces greater engagement via A/B testing, the chances that your future emails will see an improvement in engagement are strong.
Personalization & Your Audience
Many email service providers allow for personalization in your content, allowing you to create and deliver relevant and higher-performing content.
Details such as name, company name, or location can be used to improve the open and click-through rates of your emails. Personalized product recommendations almost always perform better than generic recommendations.
Alternatively, we suggest testing personalization that utilizes subscriber behavior, such as items or services they may have purchased already, when making new recommendations.
Personalization doesn’t always have to use the subscriber’s behavior, either. According to Campaign Monitor, 67% of Americans say they base their decision to open an email based on the “from” name.
Each email you send out could have a personalized “from” name (known as a “Friendly From”) that’s driven by the goal of your email. For instance, you might want your shipping updates to come from “[email protected],” while your newsletter comes from “[email protected].”
Selecting an appropriate sender name to attach to your emails can instill confidence in your audience that what you’re sending to their inbox is relevant and worth opening to read more.
Conclusion
Testing allows you to improve nearly every aspect of your email program – from audience segmentation and content relevance to design and copy tone.
As your testing continues, remember that you’re unlikely to find the perfect recipe on your first try. Incremental improvement is what you’re after – slow, steady improvement in the engagement metrics that mean the most to you.
Worth noting, one of the risks of A/B testing can be making assumptions about why a thing happened. It’s great to pair A/B testing (which does a great job of identifying what happens) with qualitative research (which helps understand why people respond in a certain way).
When looking at results, it’s good to identify that any “why” conclusions are assumptions and document them as such.
What’s great is you’ll then have the start of a new and testable hypothesis. And you can keep on improving!
The author of this article, Tim Yeadon, is Principal and Creative Director at Clyde Golden, an email marketing and digital direct agency in Seattle.
