Starting an A / B testing campaign or a CRO strategy as a whole can be a daunting prospect, especially if you are already working on traffic acquisition (SEO and PPC, etc.). You need to choose your software, possibly appoint a CRO agency, or choose to learn as you go. There’s a lot to think about, so here’s some help on where to start with creating an a / b test hypothesis.
Why make assumptions?
It’s really tempting to dive right into a / b testing and start testing all those little things that you wanted to change on your website but didn’t have the development resource to make it happen. Yes, you can get results with the fruits at your fingertips and the obvious changes that need to happen, but real results come from building informed hypotheses created from user data and information.
Using a plethora of information sources, we can build a clear picture of areas of your website that can be improved and, most importantly, guidance on what that might look like. This process eliminates guesswork, bias, HIPPO (opinion of the highest paid person) and ensures that a / b testing is user-based.
Before analyzing the data on which you will be making assumptions to modify your website, the first step is to make sure that your data is clean and of high quality. This way you can be confident in the decisions you are making. If you haven’t checked your Analytics data for a while, I recommend that you do an analytics audit first, before diving into the analytics. The last thing anyone wants is to end up making assumptions based on bad data because the a / b tests will fail and time will be wasted.
Which data and metrics are best to analyze?
To find areas of your site that can be improved, page and user metrics are the best place to start. Look at the landing pages and site entry points to see if you are losing traffic (which you probably paid to acquire) immediately. Analyze the following:
- Rebound rate
- All pages with a bounce rate of 60% or more with good session volume
- Average time on page
- Check how much time users spend before clicking or exiting, too low and it might suggest that the content is not appealing enough, too high can mean the page can be confusing for users
- Conversion rate
- Do users convert after accessing this page? Otherwise they could bounce back immediately and if they don’t bounce they could get lost in the following pages of their journey.
If your landing pages are performing well, user issues could be further along in the user journey. To find problematic pages where users leave the site before converting, analyze the pages with:
- High exit rate
- Any page with a high number of pageviews and an exit rate greater than 35%
- Average time on page
- Again, how much time do users spend before clicking or exiting, if you have a high exit rate and low average time, it can signal issues on the page.
Performing this type of scan will point out pages and areas of your site that may be experiencing issues. The next step is to analyze these pages with qualitative user data.
Qualitative user data
From the quantitative analysis, you have a list of areas and pages that need special attention. The next step is to analyze user behavior on those pages and try to find out how frustrated users are and leave your website.
Qualitative user data can show us the different types of user behavior on your website, this comes in forms like heat maps, scroll maps, and session recordings.
A basic heat map will show us where users are clicking and, more importantly, where they are not. We can find the content and business conduct page elements missed by users and make assumptions about how to reposition them in an a / b test to make gains.
The scroll cards will also show us the content and important elements missed by users, but from a page length perspective. You can find what content is scrolling (skipped) or positioned so far down the page that it missed.
Scroll cards can inform your hypothesis about moving content up or down the page so that as many users as possible see the most important items by as many users as possible.
Session recordings are a great way to save your thoughts. Finding examples of users stumbling over the problem areas you previously identified can be a very satisfying ending to your analysis and to confirming your hypothesis.
Session recordings can be easy to lose, most software can record thousands of sessions, which saves time trying to find problems on the site just by viewing the recorded sessions. Only use session recordings to find details when you are fairly sure about the previous analysis and can identify what you are looking for.
The information from this activity allows us to formulate test hypotheses based on qualitative and quantitative data. This eliminates guesswork and testing based on hunches. The ground test assumption in user data gives you a much higher chance of successfully performing a / b tests. Increase your test pass rate and increase your conversion rate faster.
To learn more about optimizing your website to get the most out of your existing website traffic, visit our CRO agency service area or contact our team today.