Researchers claim that AI chat bots can conduct high-quality conversations

Research interviews conducted by artificial intelligence can help academics to conduct studies at an unprecedented scale and produce results that are more error-free, it has been claimed.

Two scholars from the London School of Economics have developed a chat bot powered by a large-scale language model that, they say, can complete conversations with thousands of participants in less than an hour.

Rather than having a standard set of multiple-choice and open-ended text questions, as is often the case with online surveys, a chat bot takes a conversational approach, collecting respondents’ responses and using them to generate new questions within a wider set of parameters. .

Its creators, Friedrich Geiecke, assistant professor of social sciences, and Xavier Jaravel, professor of economics, say the tool uses best practices from the academic literature—for example, it encourages participants to express their opinions freely and ask follow-up questions. to ensure clarity.

They also say the chat bot exhibits “psychological empathy,” using its follow-up questions to seek to understand the interviewee’s point of view as closely as they do.

The tool is made freely available for other researchers to download, adapt and use.

In tests, it has shown amazing results. The team of Sociology Ph.D. students at Harvard University and the London School of Economics who were asked to evaluate the quality of text-based interviews rated them as broadly comparable to interviews conducted by human experts.

And when nearly 1,000 research participants were asked to evaluate their interaction with a chat bot, the majority said they enjoyed and preferred this interview method to open text spaces. Only 15 percent of respondents said they would prefer the interview to be conducted by a human.

Respondents also tended to give more detailed answers than they did with traditional open text boxes, with a 142 percent increase in the number of words written.

These advantages were particularly evident in relation to political questions. Here, researchers found that participants preferred interacting with a chat bot because they viewed it as a “non-judgmental entity,” allowing them to “feel more comfortable and express their opinions freely.”

When combined with platforms that allow academics to recruit large survey samples, such as Prolific, the tool can enable research at a significant scale, at a cost of around 3 to 4 pounds ($3.79–$5.05) per participant.

Jaravel said they were “closer” to adding voice communication to the chat bot and were also looking at how AI could be used to analyze the collected responses, which could help because this approach could lead to researchers having “millions” of sentences to sort through. by using

Although this tool will not be immediately accepted by anthropologists and sociologists, who are used to conducting very long interviews with small samples, he said that this tool may close the gap between high-quality and quantitative research of scholars such as economics and politics. scientists.

“I think this will change the way economists and political scientists do research,” Jaravel said Times Higher Education. “As we add more features, we’ll get closer to a high-quality conversation, and that may be close, but that’s not the ultimate goal.

“The goal is that we can do something that’s really low-cost, and less expensive than what political scientists and economists are doing right now, that will have a big impact on getting information out and keeping respondents engaged.”