To be customer-centric, you need to involve your customers in product research and developmentーIt is known Khaleesi. But with COVID, lockdowns and social distancing becoming the new normal, how is this possible? Can we rely on remote research as our only contact with customers? Will this change our view of remote research after lockdown?
It’s been something that has been debated for a while, some people seeing it as a Holy Grail of sorts, given that it’s more cost-effective, allows access to a much greater pool of potential participants, and avoids a host of hassles, from transport issues to scheduling the participants. But we believed, with many others, that it could never compete with face-to-face testing for the nuances it reveals in user behaviour. The reality now, however, is that COVID has forced us to find ways to adapt research that keeps the respondent and researcher safe.
The options may not just be to run a remote test or not though. A middle road that has been suggested by some in the field – including our friends Joburg-based UX design company Experio – is finding ways to conduct in-person research with as much caution as possible. Testing spaces are highly sanitised, masks are worn, and conversations with users are held through Perspex screens. The upside is that in-person is possible, but at what cost? We wonder if screens and strict hygiene measures, while a necessity, could put some participants on edge and interrupt the rapport that is vital for establishing a relationship between researcher and participant.
Without the benefit of being able to observe a user in as relaxed and ‘normal’ a state as possible, we view remote testing as preferable over the ‘face-to-face with caution’ alternative. This isn’t, however, the end of the discussion around this approach. As design expert Nick Bowmast outlines, the UX community needs to keep in conversation by sharing research, reflecting on experiences from the field, and documenting what’s been learned to arrive at best practice approaches. After all, if there’s any industry that understands how iterative processes work, it’s user-centred design.
So we’re back where we started: with no choice but to embrace remote usability testing, but making the best of the situation. One way to make the most is to take the hint from Nick Bowmast and use this time to study remote testing itself and share those findings. If you hadn’t guessed yet, that’s what the rest of this article will summarise: how we approach remote testing and what we’ve learnt.
Over the course of lockdown, we have completed over forty remote user research sessions, each of approximately one hour and all testing digital products from the healthcare, insurance and investment sectors. Testing falls under two umbrella categories: usability testing (can customers find the buttons on your app, for example) and ethnographic research (how your digital products interface with the motivations, beliefs and cultural tensions of your users, for instance).
Let’s begin with a case study that looks at ethnographic research for a funeral cover product aimed at individuals from lower-resource settings. Testing was conducted entirely over mobile to reach participants without desktop access. The first snag was smartphone penetration, which prevented the research from engaging with a host of intended users. Second, due to varying degrees of connectivity, it was necessary to send participants mobile data. This also necessitated inviting more participants than required, since a few dropped off due to connectivity failures.
For some users, these types of technical issues adversely affected their ability to feel safe. For many others, however, the experience of talking on the phone is second nature, so they readily relaxed in their home environments and became candid – often much more than at our office in town. The issues around not being able to read body language were offset by the ability to read tone of voice from participants who opened up as if chatting to a friend. One must consider that the nature of this research project meant that over-the-phone testing was a good fit, because participants were simply being asked how they felt about the topic and how it relates to their family.
The second research setting involved usability testing around a short-term insurance product aimed at individuals from higher-resource settings. This is perhaps the best fit for remote testing, since participants have access to the smartphones and connectivity that enables this research from a distance. Screen-recording allows us to analyse how participants engaged with the interface, providing rich data on usability issues.
The major benefit here was that these higher-income individuals availed themselves more readily for remote testing than they typically do for in-person studies, and they gave us more of their time during the sessions. This is likely since for these users, working from home provides them with greater flexibility to pursue tasks at their leisure. The drawback experienced by users in this scenario was around privacy and security, with some expressing concerns that the research team would have ongoing access to their phone and their financial information. To overcome this, we found that we needed to allocate more time than normal to explain clearly what was being recorded and what was not.
We will always prefer face-to-face research, but COVID has presented us with an opportunity to refine remote testing so that after lockdown we will be able to rely on it more when needed and possible. If you have any questions about running your own remote research please feel free to contact us. We are always up for a coffee and a chat (be it remotely for now).