A / B testing of mailing lists. Analysis of the conversion of letters.

A / B test measures interaction for different versions of the same email with a sample of your recipients. After creating your marketing email, you will create version B. This is required to pass analysis of how each option affects the opening or clicking of an email.

Depending on how each version of your email campaign is performing over a period of time, the tool may choose the winning version to send to the rest of the recipient list.

Note: you will not be able to fulfill A / B testing email saved to automate your workflows.

Create your A / B test emails.

Services for conducting A / B tests have a similar principle of working with e-mail. Let's look at the example of HubSpot.

• In your HubSpot account, go to Marketing > Email.
• Click on Create email... This will be the A version of your A / B test.
• In the left sidebar, click A / B iconand then press the button Create A / B Test.

• In the dialog box, enter name for Option B email, then press OK... After creating both versions of your A / B test email, you can switch between them at any time by clicking on A or B tab.

How to optimize the conversion of email campaigns?

• Suggestions: experiment with the sentence environment. You can check e-book versus white paper or video.
• Copy: experiment with the formatting and style of your content. You can check out simple paragraphs versus bullets. Also compare a longer block of text with a shorter block of text.
• Email Sender: try sending an email from an employee's email address rather than a general department address.
• Picture: try different images to see how your conversion rate affects.
• Subject line: change the length of the subject line or add personalization.
• All email: The fastest way to achieve radical results and create a landing page that generates a lot of conversions is to check all of your email. Iterate the entire email that affects the image placement, subject line, and copy. Once you get a statistically significant result indicating a better performing option, you can continue optimizing with small tweaks. 

Select the criteria for your A / B test.

When you are finished editing email options A and B, adjust your test options by clicking the tab A / B on the left side of the content editor.

This is where you can configure your testing parameters.

•  Distribution A / B: use the slider to determine what percentage of contacts will receive option A and option B. You can also configure the slider to send option A to 50% and option B to 50%. And then analyze the statistics of both versions after submitting.
  • Can be 50:50 in A / B test using any sample size.
  • All other sample subdivisions require each sample to include at least 1000 contacts. Excludes any contacts that have previously been hard-edged or unsubscribed. 

Note:  contacts that were previously abandoned or unsubscribed will be removed from your test distribution, resulting in a small sample size.

• Winning metric:  if you choose to send options A and B to a small group and then send the winning option to the rest of the recipients, click  Winning Metric - drop-down menu. This is how you can select the metric that determines how the winning option will be selected: by delivery, clicks by delivery or clicks by opening .
• Test duration: with  use the slider to set the time period for collecting data before choosing a winner and submitting a win option. Your A / B test can take up to 100 hours.  
• If the sample size is too small, please send:  if the final size of the list is too small, less than 1000 contacts, only the version selected in this drop-down menu will be sent to recipients.
• If the results are inconclusive, please submit: If the results for the winning metric are not final after the end of the testing period, the version selected here will be sent to the rest of the recipients. Make sure your email is sent with a large enough sample size and the test duration is long enough to collect results.

Note:  you can cancel the submission of winnings while the A / B test is being processed. In your HubSpot account, go to section Marketing > Email... Press name the email you want to cancel. In the upper right corner, click the drop-down menu “Actions" and select " Cancel sending winnings.    

Send a test email and collect the results.

• Click the tab  Send or Schedule.
• Choose a specific time to schedule your A / B test email. Or press "Send" in the upper right corner to send an email now.

After your email has been sent, wait enough time to collect the results. You can analyze the performance of your email options in email details:

• In your HubSpot account, go to section Marketing > Email . 
• Press name your email. The A / B test results will be summarized on the Performance"... The winning email version and metric will be highlighted in green.

• In the upper left corner, click the dropdown menu Email version, to analyze email results for all email in general, or for any of the options.

Based on materials from the site: https://knowledge.hubspot.com.

✔️ Will a winning email A / B test increase sales?

It will improve communication with customers and, possibly, give rise to sales. For a guaranteed increase in sales, you need to test all the pages of the site.

✔️ What to focus on when testing email?

On the behavior of customers, on how they react to your newsletter and how they perceive the design.

✔️ Need to test mail for different types of devices?

Yes, the behavior of users of different types of devices is different, especially if the design is not adapted for mobile devices. It's worth experimenting with all kinds of devices.

✔️ What is the main question to ask yourself when forming a hypothesis?

"Why?" Why do users react this way? Why isn't this element attractive? etc.

✔️ How many contacts must be on the list to run an A / B test?

To A / B test a sample of your list, you need to have at least 1000 contacts.

✔️ What is Mailchimp A / B Testing?

A / B testing, also known as split testing, is when you send 2 versions of an email to a segment of your audience and track which one gets the most clicks. If the process seems tedious, many email marketing services like Mailchimp have automated it to make it easier.

✔️ How to take the HubSpot abs test?

Create letters for A / B testing. In your HubSpot account, go to Marketing Email... Click Create email... In the upper left corner of the editor, click test Run test... In the dialog box, enter a name for E-mail Option B, then click OK.