This article is an updated version of one from August 2020
For years we have been advocates of in-person research. Between 2012 and 2017, we (Jacky and Chris) have designed and helped build 4 dedicated UX research labs at Flow Interactive and Deloitte Digital. Over this time we experimented with different layouts, technology, one-way glass, and waiting room designs.
Usability testing labs we made over the years
When we started How Might We, one of our main objectives was to reduce the cost of UX research so opted for a mobile lab that was just two laptops, a testing phone, and a great little device called Mr Tappy. We then just needed two rooms (one for the testing and one for viewing) and we could run an in-person test almost anywhere.
The mobile lab at a client’s office
During the lockdown, we were forced to move all our research remotely. Initially, we had some teething issues and assumed that we would get back to in-person testing rather quickly. But as the lockdown continued we learnt a lot about how to run successful remote research and overcame all the initial issues. Now, almost three years later, it’s what we recommend to most clients.
The main advantage we have experienced is how relaxed the participants are during a session. We have to spend quite a bit of time at the beginning of an in-person interview getting them relaxed and used to the weird situation they are in. Often they are flustered from rushing to get to the venue, dealing with transport, and worrying about what is expected from them. With remote sessions, there is still a little uncertainty about what is expected of them, but not having to deal with travel time and unfamiliar surroundings makes a huge difference in how quickly we can get into deep conversations. This also leads to fewer sessions being delayed and fewer no-shows.
Another benefit that has converted us to remote research, is how well our recruits match our recruit brief. Just like all good nerds, we measure everything and have been measuring how well our recruits match the recruit brief. Over the last two years, we have been able to keep this above 96%. Without going into just how we measure that, and what that means exactly (that will need a dedicated post), we can say that it’s much better than before. With a considerably larger pool of potential respondents, all over South Africa and internationally, we can find a lot more people in our client’s target market to chat to.
There are definitely downsides to remote research though. The biggest has to be that we can no longer read body language. With in-person testing, we are able to gauge a lot about a person’s answers by their facial expressions and body language. With remote testing, we only have their voice and sometimes a video of their face.
One other thing to consider is that we potentially lose out on people who don’t have access to a phone or computer and internet access.
At the beginning of our remote research journey, we were very worried about the above two points above, but we realised over time that we still get a lot of unspoken responses through voice alone, just more subtly. It’s in the hesitations, the “ums”, and the emphasis on certain words. Because we are able to spend less time making respondents feel relaxed, we can spend more time investigating these cues. Lastly, almost all of our client’s customers need to have some form of internet access to be their clients.
It’s for these reaons that we now recommend remote research to most of our clients. It’s easier, cheaper, and pretty much as effective as in-person research.