Use A/B split testing in your email marketing
Use A/B split testing in your email marketing
1. Check that your email list contains at least 1,000 subscribers to determine if a split test will be statistically valid.
The larger your list, the better your testing results and the more you can trust that those results will apply across your core audience. For example, if you have an email list of only 10 people, a split test where five people are sent version A of an email and five people receive version B won’t give you enough data, once you take into account people not opening your emails or not clicking on your links.
2. Choose your sample size using a sample size calculator or your email platform's built-in calculator.
Many major email marketing platforms, such as Mailchimp, HubSpot, and ActiveCampaign, have a built-in sample size calculator. These calculators use the platform’s own internal algorithms to calculate how many people should receive version A of an email and how many people should receive version B of an email in your split test. You can also use a free sample size calculator from providers such as: Survey System. SurveyMonkey. Optimizely. ClinCalc.com.
3. Pick whether you want to measure and improve your open rates or click through rates, and look at your historical data to set a baseline.
Look at your past three to five email campaigns and identify your average click-through rate or open rate. This will be the measurement that you want to beat going forward.
4. Isolate a single evergreen test variable, such as subject lines or the time of day that you send emails.
An A/B split test can only be successful if you remove all potential variables and are only testing one change at a time. For example, if you change both the sender name and the subject line, you won’t know exactly what drove the change in open rates or click-through rates. Example email variables to test include: Sender name: Your Brand Name or Your Newsletter Name. Subject line: Sale on Product X or 10% Discount on Product X. Audience personalization: Dear [First Name] or Dear [Full Name]. Images: Including a header image in an email or leaving the header image off completely. Content layout: Moving the location of a CTA button, or testing one-column text versus two-column text. Prioritize testing evergreen email strategies, such as your sender name. While you can test one-off marketing strategies, such as promoting a Christmas sale with a flat-rate discount versus a percentage-based discount, you will see the most value out of tests that influence your overarching, ongoing email marketing.
5. Run your split test at the same time using one control email that is unchanged from your standard practices.
Instead of introducing two new changes and seeing which performs best, find ways to iterate and improve on your existing content and email strategy. Avoid introducing new variables, such as sending version A in the morning and sending version B in the afternoon. After you’ve chosen which variable you want to test, run your test: Version A should be your standard default go-to content, whether that’s your current sender name or your regular subject line style. Version B should take that variable and introduce your hypothesis. For example, adding personality to your sender name, or updating your subject line with an eye-catching emoji icon. Send the test simultaneously to your email list. Most major email platforms will manage this automatically for you. Follow the on-screen prompts in your specific platform to set up your split test and change the one variable you’re focused on.
6. Review the results of your split test and pick the winner based on your goals.
If you have a large list of more than 1,000 recipients, a single test may indicate a clear winner where one variable led to a visible increase in open rates or click-through rates. If you have a smaller list of only a few hundred people, you may need to run several tests to determine if one day’s results was a fluke or not. Success will look different to every company. In general, any email variant that increases your results by a few percentage points indicates a strategy change for the better.
7. Continue to build on your learnings and make split tests an ongoing part of your email strategy.
Your audience’s preferences and needs don’t stay the same, and neither should your email marketing strategy. Consider incorporating split tests on an ongoing basis in every email, sending a segment of your audience a test before distributing the winning variant to the rest of your list. Most email platforms have this feature built in, and you can find it in your platform’s documentation.