Usability testing is one of the most effective methods for finding user experience issues with an existing piece of software or ensuring that a prototype meets user expectations and is easy to use.
Remote or in-person?
In the pre-Covid world, usability tests were either run at a hired venue or your offices, provided you had two rooms available and a good internet connection. Since lockdown has forced us to conduct our research remotely, we have become rather good at it and still prefer it now that lockdown has lifted. In the beginning of lockdown, we wrote a blog post investigating at all the pros and cons of remote testing.
In the few instances that we need to run an in-person test, we will discuss a venue and logistics in the kick-off session.
For our remote session, we stream the test over a secure Youtube link so your team can watch them live or we can record them to watch later. We edit the recorded videos slightly to remove the parts where we are getting set up. There is a lot of, “Can you hear me?” in remote testing.
What can’t be tested
Usability tests are great at finding out problems in a design or customer journey. What it can’t do is prove that a customer will definitely use it or how much they will pay for it, outside of the test.
Usability testing will ensure that you don’t have obstacles in the way of a customer completing their transaction. It will help you understand why customers are behaving a certain way, or why they are dropping off at a certain point of your journey. It’s possible to deduce how customers, similar to the ones we tested, will behave in these scenarios, but because we don’t know how large the target market is, we can’t guarantee that behaviour will happen in your entire customer base.
Usability testing, like all qualitative research, pairs very nicely with quantitative data. Your analytics can tell you what customers are doing and usability tests will tell you why, as well as give suggestions to improve their behaviour. Your analytics will then validate if your iterations are successful.
Structure of the test
We don’t just test the interface in a usability test. As part of the ice-breaker, we spend time understanding your customers’ behaviours, motivations and feelings. We can delve into the nuances of why they chose a certain product or how they feel about a brand.
We will ask them to complete certain tasks we have decided on in the testing plan. While they complete these tasks, we’re keenly observing, taking note of any problem areas or places where users get stuck. For example, test users might struggle to find the CHECKOUT button. In these instances, our team members ask guiding questions such as “What’s going on now?” or “Where to next?” to elicit more information about the user’s experience. These questions are not designed to frustrate the test user but rather to investigate tricky spots which may have been overlooked.
We will ask for final thoughts about the overall experience. Usability tests can’t prove if customers will definitely use something outside of the test, but it’s sometimes rather interesting to ask them to imagine themselves using it tomorrow to see what they come up with.
How we measure findings
This is something that a customer will enjoy or complete easily.
Causes confusion and doubt. Some of these are inevitable, but enough of these can cause "death by a thousand paper cuts".
Slows down the task and causes some irritation. These will reduce trust and credibility and if this happens a few times, customers will probably give up.
These issues will obstruct a process completely and will stop a customer from completing the task. These will also reduce repeat usage.
How it works
Kick-off the project
We start with a kick-off session with you to determine what needs to be tested. This could be mobile or desktop; a prototype, concept, app or live site.
We will also need to define who we will be recruiting for the day. We use two types of dimensions to help create a recruit brief: demographic dimensions (age, income, location…) and behavioural dimensions (tech adverse, relies on social proof, currently looking for life cover…).
Finding the right people to test is vital so we use a team of the best recruiters in South Africa, Europe and the USA. We create a carefully designed screener based on your brief to ensure that respondents meet your target market criteria. We also recruit one extra in case there are any last-minute cancellations.
Create the script
Based on what needs to be tested or what learnings you are looking for, we will formulate the best way to gather the findings. Knowing what types of questions to ask, and when to ask them, comes from a familiarity with research theory and years of experience.
We will share this with you before the test to get your input.
Get the recruits set up
Make sure they have access to the internet and have a device that can load your site or prototype. With remote sessions, we rely heavily on video calls so, before the sessions, we will confirm what phone or computer the respondent will be using, test their internet, and buy them data if needed.
Run the usability tests
We usually test with 5 customers a day in 45 minutes to 1-hour sessions. Remote research has its technical issues, so we will record all the videos, slightly edit them and play them back to you the next day or in the workshop as if they were live. You are also able to watch them live on the day if you wish.
Present the findings
There are two ways we can process and present the findings after the test:
A formal debrief document laying out all the complications discovered during testing. Paired with videos featuring the most impactful segments, this document highlights the most important findings for which, we create precise, actionable recommendations for both the short and long term, including possible sketches of the solutions.
This option is recommended for involving stakeholders who weren’t able to watch the tests, or if you’re not in a position to make immediate changes and need a record of the findings.
A workshop run a day or two after usability testing has finished. Working with the project team, we focus on the most pressing issues, coming up with quick wins to better the user experience. This option has a much faster turnaround time and closer involvement with your team.