AI Can Discover Corporate Policy Changes in Earnings Calls

Text mining of quarterly earnings call transcripts has become a staple of investment research, as investors search the transcripts for clues about a company’s future profitability, among other things.

While some information is relatively straightforward to extract, other information is trickier. Take corporate investment policies, for example—those often aren’t stated simply, or at all. But Georgia State’s Manish Jha and Baozhong Yang, Georgia State PhD student Jialin Qian, and Chicago Booth’s Michael Weber designed a method using OpenAI’s ChatGPT that they suggest is capable of discovering these sometimes-hidden policies.

Earnings calls are the cocktail parties of corporate communications. In contrast to the formal prose in annual filings, the candid speak during earnings calls is how company managers often reveal their strategic plans and give estimates for their company’s future sales and profits. Particularly in the Q&A session between management and analysts, they sometimes dish out valuable private information.

But not every piece of information is stated plainly. For example, an executive may say, “We are investing in growth initiatives.” While the line doesn’t directly state as much, this might imply some large, upcoming capital expenditures that could affect near-term profitability. It can take an experienced analyst with the skills of a cold-war diplomat in understanding the unspoken intentions behind statements to discern such investment policy changes.

The researchers find that ChatGPT also has the ability to uncover such information. They gathered almost 75,000 transcripts of quarterly earnings calls from nearly 3,900 US public companies from 2006 to 2020. They also obtained financial and balance-sheet data and stock-return information.

They matched these transcripts to data from the Duke CFO Survey, focusing on the responses to a question about anticipated changes to a company’s investment policy. After the matching, they had a sample of about 1,300 quarterly transcripts with supporting data.

The researchers then fed the transcripts into ChatGPT 3.5, prompting it to report back on—using a 5-point scale ranging from “decrease substantially” to “increase substantially”—how much the company planned to change its capital spending over the following 12 months. They translated the responses into a score for each transcript, and then validated their system by comparing these scores to the corresponding CFO survey responses about corporate investment plans. They find the scores and responses to be highly correlated. (They also had ChatGPT give a one-sentence explanation for each scoring decision, which they compared with the score derived by ChatGPT, for a random sample, to ensure accuracy.)