How to A / B test your site. Details with examples.

Marketers create landing pages, email copies, or design call-to-action buttons. It can be tempting to use your intuition to predict what will make people click and convert.

Basing marketing decisions on “feel” can be quite detrimental to results. Better to do an A / B test, sometimes called a split test.

Different audiences web-resources behave differently. What works for one company may not necessarily work for another. In fact, conversion rate optimization (CRO) experts hate the term “best practices” because they may not actually be the best for you..

A / B tests can also be tricky. If you're not careful, you can make incorrect assumptions about what people like and what makes them click. Decisions that can easily mislead other parts of your strategy.

Keep reading to find outhow to conduct A / B testing before, during and after data collection so you can make the best decisions based on your results.

To run an A / B test, you need to create two different versions of the same piece of content with changes in the same variable. You will then show these two versions to two audiences of the same size. Next, analyze which one performed best over a period of time, long enough to draw accurate conclusions about your results.

A / B testing helps marketers observe how one version of marketing content works together with another. Here are two types of A / B tests you can run to improve your website's conversion rate. 

Example 1: User Experience Test.

You might want to know if clicking on a specific call to action (CTA) button at the top of your home page will improve your click-through rate instead of keeping it in the sidebar.

To test this theory, you must create another, alternative web page that reflects this change in CTA placement. The existing design or "control" is Version A.

Version B is a "challenger". Then you test these two versions by showing each of them a certain percentage of site visitors. Ideally, the percentage of visitors seeing a particular version is the same.

Example 2: testing the design.

You might be wondering if changing the color of your call to action (CTA) button can increase your click through rate.

To test this theory, you must design an alternate CTA button with a different button color that will result in the same landing page as the control. If you typically use the red call-to-action button in your marketing content and the green option gets more clicks after an A / B test, this might be appropriate. By now changing the default color of your call-to-action buttons to green, you will increase the number of clicks on it.

Benefits of A / B Testing.

This analysis has many benefits for the marketing team, depending on what you decide to test. Above all, however, these tests are valuable to businesses because they are cheap but highly valued.

Let's say you are hiring a content creator with a salary of $ 50,000 per year. He publishes five articles a week for the company blog, a total of 260 articles a year. If on average a company blog post generates 10 leads, it can be said that generating 10 leads for a business costs just over $ 192. Salary of $ 50,000 ÷ 260 articles = $ 192 per article. This is a solid chunk of change.

Now, if you ask this content creator to spend two days developing an A / B test for one article instead of writing two articles in that time period, you can save $ 192 because you are publishing one less article. 

But if this A / B test finds that you can increase the conversion rate of each article from 10 to 20 leads, you just spent $ 192 to potentially double the number of customers your business gets from your blog.

If the test fails, you lost $ 192, of course, but now you can make your next A / B test even more useful. If this second test doubles your blog's conversion rate, you ended up spending $ 284 to potentially double your company's revenue. No matter how many times your A / B test fails, the ultimate success will almost always outweigh the cost of doing it.

A complete A / B testing suite for marketers.

There are many types of split tests you can run to make the experiment pay off in the end. Here are some general goals that marketers set for their business in A / B testing:

• Increase website traffic. Testing different blog posts or web page titles can change the number of people who click on that hyperlink to get to your site. This can increase website traffic as a result.
• More high coefficient conversions. Testing different locations, colors, or even anchor text on your CTAs can change the number of people who click those CTAs to land on your landing page. This can increase the number of people who fill out forms on your website, send their contact information to you, and “convert” into leads.
• Lower bounce rate. If your site visitors leave quickly after visiting your site, testing different blog views, fonts, or feature images can lower this bounce rate and retain more visitors.
• Cart with abandoned goods... According to research by MightyCall, from 40% to 75% customers leave an item in their shopping cart without paying for it. Testing different product photos, redesigning checkout pages can reduce this bounce rate.

Now let's go through the checklist for setting up, running and measuring an A / B test.

How to conduct A / B testing.

Let's take a look at the steps to take before you start an A / B test.

1. Select one variable to test.

As you optimize your web pages and emails, you may find that there are a number of variables that you want to test. But to assess how effective a change is, you need to isolate one "explanatory variable" and measure its performance. Otherwise, you cannot be sure which one is responsible for the performance changes.

You can check for more than one variable for one web page or email; just make sure you test them the same way.

Look at the various elements in your marketing resources and their possible design, wording, and layout alternatives. Other things you can check include email subject lines, sender names, and various ways to personalize your emails.

Keep in mind that even simple changes, such as changing the image in your email or the words on your call-to-action button, can lead to significant improvements. 

Note. In some cases, it makes more sense to test multiple variables rather than just one. This process is called multidimensional testing. 

2. Define your goal.

Although you will be measuring the number of indicators for each test, choose the main one that you should focus on. Do this before running the test. This is your "dependent variable".

Think about where you want this variable to be at the end of the split test. You can formulate a formal hypothesis and test your results based on this prediction.

If you wait until you think about what metrics are important to you, what your goals are, and how the proposed changes might affect user behavior. You will not be able to customize the test in the most efficient way.

3. Create "control" and "challenger".

You now have an independent variable, a dependent variable, and the desired result. Use this information to set an unmodified version of what you are testing as your "control". If you are testing a web page it is unmodified as it already exists. If you are testing a landing page, this will be the design and copy of the landing page you usually use.

From there, create a variation or "challenger". This is the website, landing page, or email that you will test against your control. 

4. Divide the groups of samples into equal and random.

For tests where you have more control over the audience, as with emails, you need to test with two or more equal audiences. This will give you the final results.

5. Determine your sample size, if applicable.

How you determine the sample size also depends on your A / B testing tool as well as the type of your A / B test.

If you are doing email A / B testing, you probably want to send the test to the smaller part of the list in order to get statistically significant results. In the end, you will choose a winner and submit the winning option to the rest of the list. 

If you are a HubSpot Enterprise customer, you can use the slider to help you size the swatch group. This will allow you to perform a 50/50 A / B test on any sample size. Although all other subdivisions of the sample require a list of at least 1000 recipients.

If you are testing something that does not have a limited audience, such as a web page, then the duration of the test will directly affect the sample size.

You will need to let your test run long enough to get a significant number of views, otherwise it will be difficult to determine if there was a statistically significant difference between the two options.

6. Decide how significant your results should be.

Once you've chosen a target, think about how significant your results need to be to justify choosing one option over the other. Statistical significance is a very important part of the A / B testing process and is often misunderstood. 

The higher the confidence level percentage, the more you can be confident in your results. In most cases, you want a minimum confidence level of 95%, preferably even 98%. Especially if it was a time consuming experiment. However, it sometimes makes sense to use a lower confidence level if you don't need the test to be that stringent.

7. Make sure you only run one test at a time in any given campaign.

Testing more than one thing for one campaign - even if it doesn't refer to the same asset - can complicate your results. For example, if you are A / B testing an email campaign that is directed to a landing page at the same time as A / B testing that landing page. How do you know what change has caused the increase in the number of leads?

What to look for during an A / B test.

Let's take a look at the steps to take during your A / B test.

1. Use an A / B testing tool.

To do A / B testing on your website or email, you need to use an A / B testing tool. 

For non-HubSpot Enterprise customers, other options include Google Analytics experiments, which lets you test up to 10 full versions of a single web page and compare their performance using a random sample of users.

2. Check both options at the same time.

Time plays an important role in the results of your marketing campaign, be it the time of day, day of the week, or month of the year. 

If you ran version A within one month and version B a month later, how would you know if the performance change was caused by a different design or a different month?

When you run A / B tests, you need to run two options at the same time, otherwise you may be left unanswered.

The only exception here is if you test the timing yourself, for example, find the optimal time to send emails. This is a great thing to test. Depending on what your business offers and who your subscribers are, the optimal time to engage with a subscriber can vary significantly depending on the industry and target market.

3. Give the A / B test enough time to get useful data.

Again, you need to make sure you let your test run long enough to get a sizable sample size. Otherwise, it will be difficult to tell if there was a statistically significant difference between the two options.

How long is "long enough"? Depending on your company and how you perform A / B testing, statistically significant results can occur in hours… or days… or weeks. 

Much of how long it takes to get statistically significant results depends on how much traffic you get. If your business is not getting a lot of traffic to your site, it will take you much longer to run an A / B test.

4. Ask for reviews from real users.

A / B testing has a lot to do with quantitative data. But it doesn't necessarily help you understand why people do certain things and others don't. While doing an A / B test, why not collect quality feedback from real users?

One of the best ways to get people's opinion is through a survey. You can add a form on your site that asks visitors why they don't click on a specific CTA, or a poll on your thank you pages that asks visitors why they clicked a button or filled out a form.

For example, you may find that many people clicked on a call to action leading them to an e-book, but once they saw the price, they didn't contact. This information will help you understand why your users are behaving in a certain way.

After an A / B test.

Finally, let's take a look at the steps to take after your A / B test.

1. Concentrate on the goal metric.

Although you will be measuring several metrics, focus on that metric of the main goal when you analyze.

For example, if you've tested two email variations and selected leads as your primary metric, don't get carried away by open or CTR. You may see a high click-through rate and a low conversion rate, in which case you can choose an option with a lower click-through rate.

2. Measure the significance of your results using our A / B test calculator.

Now that you've determined which option works best, it's time to determine if your results are statistically significant. In other words, are they enough to justify the change?

To find out, you need to run a statistical significance test. You can do it manually or using services.

If you use a calculator, the process will be as follows. For each variation tested, you will be prompted to enter the total number of attempts, such as emails sent or impressions viewed. Then enter the number of goals you have met - you will usually be looking at clicks, but there can be other types of conversions as well.

The calculator will give you the confidence level of your data for the winning option. Then compare this number with the value you chose to determine statistical significance.

3. Act based on your results.

If one option is statistically better than the other, you have a winner. Complete your test by disabling the losing option in your A / B testing tool.

If neither option is statistically better, you just learned that the tested variable did not affect the results, and you will have to mark the test as inconclusive. In this case, stick to the original version or run another test. You can use erroneous data to help you define a new iteration in your new test.

While A / B tests help you influence your results on a case-by-case basis, you can also apply the lessons you learn from each test and apply them in your future endeavors.

For example, if you've done A / B testing in your email marketing and found repeatedly that using numbers in email subject lines results in higher click-through rates, you might want to use this tactic in most of your emails.

4. Schedule your next A / B test.

The A / B test you just completed may have helped you find a new way to make your marketing content more effective. Don't stop there. There is always room for more optimization.

You can even try an A / B test on a different feature of the same webpage or email that you just ran the test on. 

For example, if you've just tested your headline on your landing page, why not do a new test for your body copy? Or a color scheme? Or images? Always keep an eye out for opportunities to increase conversions and leads.

A / B testing allows you to understand what content and marketing your audience wants to see. Start A / B Testing Today!

Based on materials from the site: https://blog.hubspot.com

✔️ When is A / B testing a bad idea?

If there is no specific goal. Testing should only be done with an expert.

✔️ What is a null hypothesis in AB testing?

It is the hypothesis that any difference in results is the result of sampling error or standard deviation.

✔️ How many page visits do I need to get good results with A / B testing?

Before you can check the results of an A / B test, you must ensure that it reaches statistical significance. Many services have a special calculator.

✔️ Does A / B Testing Affect SEO?

Yes, it improves usability and traffic. This is part of the SEO promotion.

✔️ What is Web Application A / B Testing?

A / B testing (also known as split testing or group testing) is a method of comparing two versions of a web page or application in order to determine which one performs better.

✔️ How to test a site?

6 stages of website testing.
1. Analysis of functionality.
2: Usability Testing.
3: Testing the interface.
4: Check compatibility.
5: Testing performance.
6: Security check.

✔️ What can be tested with A / B testing?

A / B testing allows you to find out which words, phrases, images, videos, testimonials and other elements of the site work best. Even the simplest changes can affect your conversion rate.

✔️ How long should the A / B test take?

Typically, the website needs to be tested for a minimum of seven days. You need to make sure you reach statistical significance and then test for another seven days if not. When it comes to data, more is almost always better than not enough.

✔️ How can I check the site for errors?

Make sure your HTML or XHTML is free of errors by validating it with W3C Markup Validation, the official validation tool of the World Wide Web Consortium. There are other tools like HTML Tidy, Google Webmaster Tools. All of these tools find duplicate meta tags, broken links, missing headers, or other errors in your code.