(gentle music)
Narrator: Artificial intelligence increasingly helps guide decisions, from approving loans to screening job applicants. But what happens when AI starts running the process those decisions depend on? In hiring, that means conducting the interview itself. Can an AI interviewer screen candidates in a way that leads to better hiring matches than a human recruiter?
Brian Jabarian: In this paper, we are basically trying to understand what happens when you replace human interviewers by AI agents in the job of conducting job interviews. So there has been this huge interest of AI agents, how they can provide information to other humans to make a decision, but less so about what happens when AI collects information from humans. So the idea here was, is AI capable of screening the right skills during a job interview that are going to translate downstream as a better match?
Narrator: Carnegie Mellon’s Brian Jabarian, a former Chicago Booth principal researcher, and his coauthor partnered with a recruiting firm that was hiring for customer service positions. Seventy thousand candidates were randomly assigned to interview either with an AI agent or with a human recruiter. Candidates knew when they were speaking with AI, and final hiring decisions were always made by human recruiters reviewing the interview transcripts and audio.
Brian Jabarian: What makes our setting very powerful is the following three aspects. The first one is that we partnered with a real company hiring for real jobs, and therefore candidates and employers faced real economic incentive and constraint. The second aspect is the scale of it. We had, like, 70,000 candidates. And the third aspect was the richness of our dataset.
Narrator: Because these were real jobs, the researchers could track what happened after the interview, who received offers, who started the job, and who stayed—and the results were striking.
Brian Jabarian: What we found in this experiment is that, actually, AI leads to better job outcomes. It means that candidates who get a job interview with an AI are more likely to receive a job offer, are more likely to start the job, but also are more likely to stay in the position after 30 days, but also up to four months. So the retention rate is much better for this group. This retention basically doesn’t come at the cost of productivity loss. And when you compare the groups of employees who got interviewed by an AI and the employees who got interviewed by a human, we don’t see a difference in these productivity metrics. So that’s a very good sign because it means that you can improve retention, as in, you can improve the match quality without deteriorating productivity. The second reason that we can tell that this retention is a good outcome is because there is no deterioration also, or no difference in the separation after 30 days. Forty percent of people left voluntarily in both cases.
Narrator: So why would an AI interview lead to better outcomes? The answer appears to lie in the structure of the interview itself.
Brian Jabarian: AI is not so magic. Basically, AI is an interesting tool here to more consistently perform different interviewing tasks, such as covering a certain set of topics, having a certain order, having a certain language. And this consistency, basically, in return helps the candidates to better reveal themselves their true type, during the interview.
Narrator: Both human and AI recruiters were required to cover 14 interview topics, but humans naturally vary in how consistently they conduct interviews. Fatigue, time pressure, and conversational drift can all change how questions are asked. AI removes much of that variation.
Brian Jabarian: AI is more consistent than human recruiters in performing this task. They more consistently cover more topics. They have less variations in the follow-ups and in the flow of the interview and also more consistently apply a higher language across the interviews. Right? When you are a human interviewer, sometimes you start the job interview in the morning and you perform really well at your task, but maybe for some, maybe for reasons of fatigue, you might be less consistent,basically. So AI here helps you having this consistency, which is what we mean by having this control. In response to this consistency, what we say is that the candidates are more likely, basically, to reveal their true type in a job interview conducted by an AI than in a job interview conducted by a human. They reveal more precisely this type of signal that the human recruiters are going to value for making, for issuing a job offer. The candidates who got a job interview with an AI are more likely to display interactivity as a signal during a job interview and less likely to display a (indistinct) cues. If we were in a world where AI was artificially inflating your value during a job interview, you would basically see deterioration in terms of the productivity downstream, but we don’t see this here. So, basically, AI here helps screening more precisely the type of candidate that the recruiters are looking for in this market.
Narrator: The finding isn’t that AI should replace humans. It’s about how AI and humans can work together.
Brian Jabarian: The overall takeaway of this research is not that you should replace all the time humans by AI. It’s more like a subtle, it is more subtle and more complicated. It’s that AI adoption shouldn’t be seen as a task-by-task decision, but more about organizational design as a whole and basically thinking about how to combine AI agents working together with human agents in the workplace.
Narrator: AI doesn’t eliminate human judgment, but when used thoughtfully, it can structure the way information is gathered and improve the decisions that follow. (upbeat music)
