While many research facilities have re-opened for business and taken measures to ensure the safety and peace of mind of respondents and researchers alike, that doesn’t mean researchers are jumping at the chance to get back into the field. To get a better projection for the return to in-person research as well as the longevity of Online Qual, we surveyed moderators and clients to learn about their experience with Online Qual during COVID-19 and what they anticipate for their future research. While the results are richly nuanced, if you read no further, know that Online Qual will be the preferred approach to qualitative research for the foreseeable future.
“I think the flexibility and speed are assets. We’ve spent too much time on the road historically when the advantage of in-person isn’t necessarily justified relative to the experience we’ve had online.” — Client
Across the board researchers’ opinion of Online Qual has improved from pre-Covid times or stayed the same for those who already used to it. Insights remain robust and Online Qual fits the needs well of the vast majority of projects. Clients report greater convenience and enjoy being able to message the moderator in situ with probes rather than waiting until the end of the group when the moderator checks in for questions that would have been more organic earlier. They appreciate being able to fit in more groups across wider geographical areas with niche recruits than they would otherwise. Less traveling means more productivity, saving on travel costs, and the ability for more stakeholders to log in and view interviews. This also means more people in the virtual backroom, though both client and moderators lament not having the space to debrief and springboard insights and iterate between and after sessions.
Online Qual is not without its drawbacks, of course. The most commonly reported are connectivity issues that can both delay the onset of interviews as well as disrupt the flow of conversation. While this can be remedied by tech checks in advance, unforeseen issues can still come up. The adequacy of existing platforms is another grievance, as frustrations with numerous steps and tech issues quickly erode the patience of stakeholders. Thankfully the increase in demand for Online Qual has already driven innovation, as companies like Recollective and Discuss.io have recently come to offer more seamless experiences. While some Moderators worry about missing rich insights from the inability to see non-facial paralanguage, they also note that respondents seem more candid with Online Qual. This may be due to the Online Disinhibition Effect, as we have also seen a drop in decorum when, for example, a respondent begins folding laundry or steps away from their screen without a word. Interruptions from barking dogs or a crying child are also risks with the territory. Some clients worry about screen recording and leaking confidential stim; and while this is a valid concern, there is no record of it ever happening in Online Qual. All participants sign an NDA and moderators explicitly state at the onset that they will be viewing confidential information and have agreed not to disclose it to anyone in any way.
“I think there is a workaround for everything. Yes, you don’t get the in-person ability to read faces, body language and tone as much. But, at the same time, I find that some consumers are actually more honest, transparent and real in online methods.” — Client
Even so, Online Qual affords a compelling alternative to in-person the present situation notwithstanding. Three in Four moderators said online IDIs are better or as good as in-person. On the client side the predilection is slightly lower at 60% with 40% saying it’s a good substitute. The percentages are similar when it comes to dyads and slightly lower for triads, but when it comes to focus groups of 4-6, 80% of clients said it was a good substitute but not on par with in-person. 60% of moderators agree. As such, the larger the group, the less suitable Online Qual is.
With this context, half of clients are looking forward to returning to in-person research (though not counting down the days) while half don’t feel strongly about it. Ensuring rich insights while maintaining the safety of all parties is top priority. Even with facilities taking measures to prioritize safety, there is still the risk of air travel for moderators and clients. As such, many are uncomfortable returning to in-person until a vaccine is available. Perhaps the solution is to only work with local moderators, although that would still require clients to log in remotely. Until then, one-third of clients expressed intention to conduct more Online Qual in the future while two-thirds said they would conduct the same as they currently are.
“The speed and flexibility of recruitment has been a surprise. It makes the experience that much more fluid, and the approach when combined with stay-at-home order means we’re able to source a wide variety and representative set of participants very quickly.” — Client
When the smoke finally does clear from Covid-19, if given the option to choose in-person over online, clients and moderators alike reported the same three drivers: ease of connecting and iterating in person, richer insights, and relationship building. But as Online Qual remains the norm, the challenge is on us as researchers to work around the pain points until they no longer exist. Maybe we look for different non-verbal cues online than we usually do in person. Maybe we plan video debriefs after every interview to iterate and discuss. The culture of Online Qual is not yet set, so the onus is on us to collaboratively shape it.
While skeptics may question if Online Qual can be as good as in-person, I am reminded of the film Fearless that climaxes in a tournament between martial artists of different schools. Before the fight, two wushu virtuosos are having tea and one asks the other which style of wushu is better. The other replies that no school is superior, but the skill of the practitioner is what determines victory. Similarly, while each has its advantages and disadvantages, the skill of the moderator is what determines the conversation and the richness of the insights elicited. Greater familiarity also leads to greater facility with the medium, so the more you do, the more adept you get. We’ve moderated over 150 online interviews since March and anticipate many more virtual conversations in the coming months as we work closely with our clients to provide creative ways of speaking with consumers during the COVID-19 pandemic.
“I’m totally biased on this, but I just find that you can get almost everything you need through online methodologies if it’s done right!” — Client
Talk Shoppe Internal Survey, June 2020. Moderator n=18 and Client n=10
Commentaires