The Equation How AI Can Make Smarter Predictions

Artificial intelligence can be overconfident, making predictions or statements that turn out to be wrong. A team of researchers that includes Chicago Booth’s Veronika Ročková came up with a solution: Give AI a way to evaluate and calibrate its own uncertainty, allowing a user to decide how much to trust a prediction.

For example, imagine using AI to help diagnose a patient’s illness. The researchers’ method builds a “tree” from past cases whose final diagnoses are known, grouping those cases into “branches” and “leaves” according to shared features such as a fever or cough. The AI assesses possible diagnoses for the new patient, calculating a score to reflect how well each prediction fits the symptoms. Meanwhile, the method assigns the new patient to a leaf and considers how well AI did at diagnosing other cases from that same group. If the AI was largely accurate, it can confidently give a few possible diagnoses. But if it wasn’t, it should issue a wider range of diagnoses. To learn more, read “Is ChatGPT Confident About Its Answer or Just Bluffing?”