Creating & Improving Your First A/B Test

You don’t want to run an A/B test all by yourself. Why? Because there are so many amazing SaaS tools out there that do all the heavy lifting for you while ensuring your test results are statistically significant. 

These are robust CRO platforms that provide everything needed to run an optimization program in a methodical, sustainable manner. Some of the most popular tools are VWO, Optimizely, AB Tasty, Google Optimize 360, and Convert. Most offer at least a 30-day free trial, which should be enough to cover your first A/B test’s duration. 

Explore Different Platforms

Assuming this is your first test, why not use the time to explore the different platforms? Check out their features and capabilities. Some questions you should be asking should include: 

• How easy is it to integrate this tool with my existing marketing/tech stack?
• How does the platform affect the performance of my website?
• Does the platform offer more value beyond just A/B testing? How?
• What is the level of customer support provided?
• What is the pricing like? Can I alter it according to the needs of my business?

TIP: If needed, get the answers to these and other questions through a product demo or product tour during the free trial phase.

Creating Your First A/B Test

Once you’ve zeroed in on the tool you’re going to use, you need to integrate it with your website and set your test up in the respective tool. (Think of it as a draft you make before you hit publish on your blog.) 

Depending on the complexity of the idea you’ve chosen to test, you could either use the tool’s visual editor to create your variation or ask your UI team to code one into the tool. In either case, you have to run it past a thorough quality check to ensure the experience is consistent on every device, browser, platform, etc. 

Next, you need to choose other variables for your test such as the conversion goal, audience, etc. (All of which are fairly easy to navigate through.) All you need to ensure is you’re doing QA at each step, and that’s about it. You’re ready to go live! 

You’ll also want to also make sure you are doing one A/B test at one time. Testing too many different elements at once can make it very confusing to see which edit actually achieved the higher result. If it helps, plan out a schedule of what elements you’d like to test and which ones are the highest priority then keep track of the changes you make to see which garners the best results.

Avoid Common A/B Testing Mistakes Made By Budding Optimizers 

1. Not doing QA at each stage of the test setup
2. Not selecting the right audience for the test
3. Running tests on pages with very low traffic
4. Making changes to the test mid-way
5. Stopping a test too soon
6. Not tracking test data properly
7. Running overlapping tests on the same traffic
8. Not considering small gains as a win
9. Giving up after the first test!

The Importance Of Conversion Research

Did you know that one in every eight A/B tests fails? Savvy optimizers spend a major chunk of their time doing something that actually brings success to their CRO programs: conversion

research. When it comes to ideating what to optimize and how your website can turn out to be a goldmine of insights. Here are three simple steps to help you tap into that. 

STEP 1

Figure out where the drop-offs are happening. Go to your website analytics. Regardless

of whether you’re using Google Analytics or a less popular one, start tracking key visitor journeys on your website. Zero in on the pages where you’re leaking visitors. This can be done by setting up funnels.

STEP 2

Conduct visitor behavior analysis. Your website analytics tells you where you’re leaking visitors or users, but not why or what they’re actually doing on your website. This is where qualitative

research using visitor behavior analytics tools (such as heatmaps, session recordings, etc.) comes to your rescue.

 

Chances are, the conversion optimization platform you’re trying out provides these capabilities built in. If not, you can always try a standalone tool such as Hotjar or Fullstory. 

Here’s what these different tools will tell you:

1. Heatmap: what parts or elements of your webpage are getting the most attention
2. Session recordings: playback of your visitors’ real-time activity on your website
3. Click map: where visitors are clicking the most on a webpage
4. Scroll maps: what percentage of a full-length webpage are people scrolling down to
5. Form analysis: which form-field is the point of friction and other insights
6. Website surveys: what your visitors actually feel about your website

STEP 3

Once you’ve figured out where and why your website visitors are dropping off, the obvious next step is to put the insights into action. What you need to do now is create something called a hypothesis and test it. Remember, you don’t have to conduct conversion research for your website or digital properties just once. Rather, it should be a continuous process you rely on in order to have the most updated insights on your website’s performance and come up with

hypotheses that really move the needle. Conveniently, we wrote another blog post specifically about prioritizing your hypotheses that you can read here.

Want to know more about improving your CRO? Download our latest white paper, “The Crawl, Walk, Run of Conversion Rate Optimization”. It’s a complete CRO roadmap that takes you beyond analytics and puts you in the driver’s seat.

Missed seeing the first blog on our CRO whitepaper? Catch up here.