AI on the Job: How Agents are Changing the Way We Hire

A Recruiting Revolution

Thirty years ago, job seekers paged through newspaper ads, made cold calls, or approached strangers at corporate networking events. Today, online applications and social networking platforms like LinkedIn, as well as applicant tracking systems, enable hiring managers to sift through hundreds of resumes, some real, some bot-generated, in search of talent. Even with keyword search and automated prescreening tools, recruitment remains a cumbersome part of daily business.

Now, the artificial intelligence workforce has arrived, and its agents are seeking roles in recruitment and hiring. Incorporation of AI into workflows has become pervasive across industries worldwide. As of late last year, 93% of Fortune 500 Chief Human Resource Officers surveyed by Gallup said their organization had incorporated AI tools and technologies to enhance business operations.

Equipped with a new workforce of agents, large language models, and predictive algorithms, business leaders are now rethinking the role of the recruiter. This bold move aims to enhance efficiency, streamline the hiring process, and potentially shift the role of human recruiters.

A new study led by Chicago Booth’s Brian Jabarian is testing the efficiency, effectiveness, and ethics of bringing AI to the interview room. The research is part of an ongoing five-year collaboration that Dr. Jabarian has developed with one of the global leaders in recruiting process outsourcing (RPO), PSG Global Solutions, a subsidiary of Teleperformance (TP) Specialized Services. Led by Jabarian, this multi-phase research aims to study how AI impacts the entire operational chain value in core RPO activities and related ones in business processing outsourcing.

Dr. Jabarian, Howard and Nancy Marks Fellow, Principal Researcher at the Roman Family Center for Decision Research and Research Affiliate of the Center for Applied Artificial Intelligence, examines how emerging technologies transform cognitive work, decision-making, and the design of organizations, markets, and institutions.

This summer, Jabarian and his collaborators completed the first study among many in the coming years. Alongside co-author Dr. Luca Henkel, Assistant Professor of Finance at Erasmus University Rotterdam, they are providing evidence on whether AI voice agents can not only match human recruiters in this complex task of conducting job interviews but also help improve recruitment outcomes in several dimensions without damaging core operations or candidate satisfaction.

AI in the Interview Room

Specializing in recruitment process outsourcing, firm PSG Global Solutions decided to shake up a crucial stage in the hiring process, job interviews, with AI voice agents. The decision to explore interview automation was visionary for a firm with a global reputation for delivering top candidates and a refined recruitment process.

Over 70,000 job applicants for roles in healthcare, IT, and industrial sectors were screened using a combination of human-led and AI-led interviews. Invited candidates were randomly selected to either follow a traditional interview process with human recruiters, to interview with AI agent “Anna,” or to choose between the two interview paths. In all cases, the final hiring decision was made by a human recruiter. At the beginning of each interview, the AI voice agent disclosed its artificial identity.

Since the project began in March, the research team has been carefully tracking key performance indicators such as the number of job offers, starters, and one-month retention rates. So far, the results are promising: Anna has outperformed traditional hiring methods across nearly every hiring metric. Interviews conducted with the AI agent led to 12% more job offers, 18% more job starters, and 16% higher retention rates after 30 days of employment.

The Ethics of Taking the Human out of HR

Trained to respond in real-time to a range of possible interview responses, Anna also has the potential to carry out live skills assessment tasks. “AI agents like Anna could radically transform the entire recruitment process,” said David Koch, Chief Transformation and Innovation Officer at PSG Global Solutions.

According to Koch, AI recruiters could simulate scenarios to glean problem-solving and conflict management skills. Freed from high-volume repetitive screenings, more adept recruiters may have more bandwidth to gauge minute differences between top candidates as well as qualities like organizational cultural fit.

For roles with more technical or routine tasks, incorporating real-time evaluation into the interview with Anna could truncate the long and cumbersome interview process. Expedited matchmaking could close the gap for 7.2 million currently unemployed Americans.

Yet, AI hiring managers may be less well-suited for assessing applicants for roles requiring more mercurial and unpredictable tasks like crisis management, creative problem-solving, or thought leadership. “Whether to use AI versus human hiring managers is conditional on the characteristics of the role, the skills involved, and the experience and qualities of the candidates, and an open question that we aim to tackle in our next study,” said Jabarian.

Some candidates disengaged during the interview with Anna—3.2% of candidates ended the AI interview because an AI aversion and 8% because of an AI system failure. Despite this, when given the option to interview with the AI hiring agent, a staggering 78% of applicants opted for Anna over a human recruiter. Barring initial hesitation around some of Anna’s less organic mannerisms and some technical difficulties, many candidates found the AI-driven interview less intimidating and more efficient.

Current recruiters, however, maintain a healthy level of skepticism. Engaging with other people through interviews is more than just a core job function—it’s a rewarding social function that provides value and fulfillment. And if AI tools like Anna are able to move through the majority of the traditional interview process, recruiters will have to develop new ‘meta-analysis’ skills related to process evaluation.

This fundamental shift in the daily functions performed by recruiters signals a larger organizational restructuring may be on the horizon. “There is a growing need for reskilling and upskilling across the entire recruitment ecosystem,” said Koch. Envisioning and designing this next generation of careers will be folded into the next phases of the research.

“ It’s not yet possible to delegate the more nuanced, in-depth evaluation of candidates entirely to AI. We will need human intervention to oversee and review the performance of AI recruiters,” Jabarian explained. And, evaluating qualitative skills—by reading social cues, facial expressions and body language—still seem to need the ‘gut check’ of a real person.

From Handshakes to Algorithms

As AI tools like Anna become more integrated into recruitment, organizational leaders will be challenged to approach AI implementation using a framework oriented towards ethical and responsible transformation. For Jabarian, using a series of structured and randomized studies is valuable for helping companies like PSG Global Solutions and Teleperformance see the direct impact of different AI interventions. He hopes the research will help companies clearly communicate the economic and social value of AI to their stakeholders.

“Every day we’re observing new ways that this technology is shaping the culture and structure of the modern workplace in the age of transformative AI,” said Jabarian. “Our goal is to support that transition by providing robust evidence.”

The collaboration marks the beginning of a broader discussions about the future of AI in the workplace, and is part of a collection of projects led by Booth faculty and researchers that are supported by CAAI.

Read the full publication here.