This article is a part of our encyclopedia, and is editable by you. Edit ➜
What Is Email A/B Testing?
Email Campaign A/B testing involves sending two or more variations in your email marketing campaign to determine which version attains a higher open rate or conversion rate compared to the others.
In email A/B testing, you may want to compare how different email subject lines, CTA button color or images perform in terms of attracting conversions.
Ideally, A/B testing in the context of email campaigns involves a 50/50 split meaning half of your audience will receive variation A while the remaining half gets variation B.
Email A/B Testing Key Takeaways
- Better email open rates and click-through rates, result in more website visitors and increased sales.
- An A/B test eliminates decisions based on opinions, leading to a clear direction for your business.
- A/B tests improve conversion rate and help you understand what messages resonate most with your audience.
- With proper A/B testing, any business owner can attain a consistent path for growth.
One way to attain this is through running A/B tests on your email campaigns.
Understanding How Email A/B Testing Works
Here is an illustration of how the email campaign A/B testing would look like:
The ultimate goal for any email A/B test is to find out which variation of the email campaign garners the best results.
In our illustration above, Subject line A attained a 19.4% open rate, while Subject line B attained a 27.8% open rate.
Once the A/B testing is concluded and the winning version found, you automatically switch to the winning version. in our case, subject line B is the winner.
Therefore, our subsequent emails for this particular campaign will have subject line B.
Your email campaign A/B test might include the following simple steps.
Step1. Pick what you will test. For instance, in our case, we are testing the subject line.
Step2. Determine which metrics will guide you in choosing the winning version. [could be open rate or CTR]
Step3. Decide who you send the campaigns to / set the sample size [send to 25% of email subscribers]
Step4. Measure results and pick the best version.
Simple A/B Testing Strategy Vs Advanced A/B Testing Strategy
Email A/B testing can vary in complexity.
Simple A/B testing focuses on sending multiple subject lines, changing CTA color, or different images to test which one generates more opens.
Advanced A/B testing includes using completely different email templates to see which one generates more CTR.
If you are using emailing tools with advanced features, A/B testing your campaigns is very easy as you can use the email builder to create 2 different variations of your email campaigns.
What You Should Test In Your Campaigns
If you are using the best email marketing tools, you can test any aspect of your email marketing campaigns and attain improved results.
Here are few ideas of what to test in your email campaigns.
- Call to action [Example: “Shop Now”! vs. “Shop our Spring Collection”]
- Subject line [Example: "Enjoy 25% discount on our Spring Collection" vs. "Spring Collection Offers"]
- Testimonials [include (specific testimonials) or Not to include at all ]
- Email content layout [single column vs. two columns / varying placement of your email elements]
- Personalization [Example: "Dear Mr. Davis" vs. "Our esteemed customer"]
Each of these variations is likely to have an effect on different aspects of the conversion process.
Email Campaign AB Testing Best Practices
- Always test simultaneously to ensure time-based factors do not affect your results.
- For more accurate results, test a larger sample as you can.
- For quicker and easier Email campaign A/B testing, make use of tools available to you.
- Testing early and often helps to attain the best results
Real-World Examples Of Successful Email A/B Testing
Here are two real-world examples of successful AB tests:
1. How Blenders Eyewear increased revenue by 48%
Blenders Eyewear, launched an email A/B click test campaign.
The campaign featured two versions of emails with different visual styles but similar images and calls to action. Version A got a 33% increase in clicks and a +48% lift in revenue.
2. How Designhill increased clickthrough rate by 5.84%
Designhill, one of the fastest crowdsourcing platforms did an email blast aiming at promoting its content and increasing the CTR.
Just mentioning the title of the blog in the subject line of the email got a majority of click-through compared to the version requesting recipients to review the post with the blog's title.
The company scored a 5.84% higher CTR and a 2.57% higher open rate by including the title of the blog in the subject line.
How To Run An Email A/B Testing Campaign
- Know what you are testing
- Focus on the frequently sent emails
- Split the subscribers list randomly
- Test highly divergent variants. (e.g, plain text versus HTML emails)
- Keep in two variants
- Wait for up to a week to test
- Check if the results are statistically significant
- Act on the results
Email A/B testing does not have to a strenuous or expensive exercise. Different emailing tools come with an A/B testing feature. Therefore, you will want to go for these advanced emailing tools:
Klaviyo’s the ultimate ecommerce marketing platform trusted by 50,000+ brands, like Chubbies, Living Proof, and Hint. Deliver higher-value, personalized marketing experiences across email, mobile, and web.
- Up to 250 contacts: Free!
- Up to 1,000 subscribers: $30 per month
- Up to 2,500 subscribers: $60 per month
- Up to 5,000 subscribers: $100 per month
- Up to 10,000 subscribers: $150 per month
Up to 20,000 subscribers: $375 per month
Businesses using Klaviyo:
Campaign Monitor makes it easy for you to create, send, and optimize your email marketing campaigns.
- 1,000 contacts - $29/month
- 5,000 contacts - $59/month
- 10,000+ contacts - $149/month
- 1,000 contacts - $49/month
- 5,000 contacts - $99/month
- 10,000+ contacts - $149/month
- 1,000 contacts - $89/month
- 5,000 contacts - $149/month
10,000+ contacts - $249/month
Businesses using Campaign Monitor:
Contributors to this article:
- Pat Walls, Founder @ Starter Story
- David Gitonga, Writer @ Starter Story
Want to start your own business?
Hey! 👋I'm Pat Walls, the founder of Starter Story.
We interview successful business owners and share the stories behind their business. By sharing these stories, we want to help you get started.
Interested in starting your own business? Join Starter Story Premium to get the greatest companion to starting and growing your business:
- Connect + get advice from successful entrepreneurs
- Step by step guides on how to start and grow
- Exclusive and early access to the best case studies on the web
- And much more!
Did you know that brands using Klaviyo average a 95x ROI?
Email, SMS, and more — Klaviyo brings your marketing all together, fueling growth without burning through time and resources.
Deliver more relevant email and text messages — powered by your data. Klaviyo helps you turn one-time buyers into repeat customers with all the power of an enterprise solution and none of the complexity.