Email Marketing Automation made easy! Learn more >

a/b testing

Be successful with A/B Testing in email marketing

A/B testing is a both simple and effective way to maximize the newsletter opening rate and click rate.

You just have to determine two possible subject lines and the size of your test groups. mailworx will then find out, which of them is the most successful one!

In the following paper, we would like to show you how to A/B test your email campaigns.


What is A/B Testing?

How does A/B testing work?

How to A/B test email campaigns with mailworx

Status Information

Measure the A/B Test


If your question hasn´t been answered in this whitepaper, please let us know. We´d gladly catch up on this.


Ask your question


What is a A/B Testing?

The subject and the sender are the most essential decision criteria for subscribers, when it comes to opening or not opening the newsletter. The A/B Testing enables the optimization by testing which subject line or which sender's name achieves the best opening or click rate.

Measurements of more than a million emails have shown that A/B Testing can bring you click rates, 2.5 times higher than without testing and up to 50% higher opening rates.


How does A/B Testing work?

If you want for example send a newsletter to 100,000 recipients, the perfect subject line is really important. When using A/B testing, two random test groups are selected - e.g. each 10% of the  total group.

5 % (5,000) of the recipients will get the first subject line and the other 5% will get the alternative subject line. After for example a day, the better performing (better opening rate or click rate) subject line is determined as winner and the remaining 90,000 recipients get optimized email campaign.

The 10% are chosen randomly and don´t know that they are a part of a test group. Because of that, they just behave the way they would always do.


How to A/B test email campaigns with mailworx

Using this tool in mailworx is really simple. Just choose "A/B Split Test" as send type for your campaign.

ab testing

Step 1: Choosing the test attribute

You can optimize the subject line or the sender's name. Even if you think the sender is somehow given, there are slight variations you can try out. Instead of using your company name, you might choose the name of a team member or the responsible key account manager. Use A/B Testing to see, which version works best for you.

ab testing

Step 2: Determine the size of your test groups

The size of your test groups depends on the size of the whole email list, you want to send your campaign to. In many cases a test group of 10 % to 30% is suitable. With the slider you can select the size and you can see right away the amount of emails of each group.

ab testing

Step3: Define the winner criteria

In A/B Test, the remaining subscribers get their newsletter just after a special period of time. We recommend to give the test phase at least one day, to make sure, that most recipients of the test campaigns had the opportunity to open the email. Afterwards mailworx will automatically send the better versions to the remaining subscribers.

winner criteria


Status information

During the testing process, you can see the status information of the A/B Split Test in the real-time statistic of mailworx.


Measure the A/B Testing

The success of the A/B testing depends on three factors:

  • Is the size of the test group suitable?

  • Is the test attribute (subject/sender) variable enough?>

  • Is the waiting period between the test and the real sending sufficient?


Did you get significant results from your A/B Testing?

You can view the statistic of your split test in the "Statistic" tool by selecting A/B Split Test. There you can see, whether the test was meaningful or not.

measure ab testing


But what to do if the test wasn´t successful? You have three different options:

  • Raise the testing period

  • Enlarge the size of the test group

  • Try to find more variable subjects / senders