The Hazards of Second-Guessing
- February 16, 2021
- CBR - Behavioral Science
The “wisdom of the crowd” maxim holds that a group is better than an individual at producing an accurate estimate, since an average across many people will remove the bias and random error in an individual guess.
The same is often true when an individual makes multiple guesses—a phenomenon that has been dubbed the “wisdom of the inner crowd”—whereby averaging a person's first and second estimates will generally produce a more accurate response than taking either one alone. People seem to know when they've misguessed and, if given another shot, often make a second guess that's in the right direction.
Gaertig and Simmons hypothesized that the process was largely unconscious—and further, that asking people explicitly to evaluate their first guesses might disrupt the process so much as to dissolve the effect.
They carried out a series of experiments in which they asked participants to make a guess about something unknown. In one experiment, for instance, participants guessed what percentage of people preferred Indian food over Mexican food; in another, what percentage had a Twitter account; and so on.
Then all the participants made a second, different guess about the same initial question, but half the participants were asked to explicitly decide whether their first guess was too high or too low before guessing again, while the other half were only asked to make that second guess. Participants were paid a small amount for accuracy, which provided an incentive to make accurate predictions.
Recommended Reading Even When Algorithms Outperform Humans, People Often Reject Them
Research unpacks why people often prefer the predictions of humans.Even When Algorithms Outperform Humans, People Often Reject Them
Overall, the participants asked to analyze whether their first guess had been too high or low performed worse on their second prediction—that is, their second guesses were more likely to be more extreme and thus less likely to be in the correct direction relative to the first. The results held across several experiments, both online and in person.
The researchers also wanted to test the phenomenon in a situation without the boundaries of percentages, so they asked another group of participants to look at stock prices for 10 well-known companies and predict what the prices would be two weeks in the future. As before, one group analyzed their first guess before making a second one—and once again, this group produced second guesses that were more extreme than the first, making the average less accurate.
Recommended Reading How Opinion Polls Are Presented Affects How We Understand Them
Perceptions of the future may depend on the format of forecasts as much as on the forecasts themselves.How Opinion Polls Are Presented Affects How We Understand Them
Asking people explicitly to evaluate their first guess may cause them to use that first guess as a reference point, which can lead to a second guess that's further from the actual target, the researchers explain. More work is needed to understand exactly why that is, but in the meantime, supervisors who are asking for estimates—for example, on the likelihood of a product's success—may want to make sure not to upend the inner-crowd phenomenon.
“In general, managers who attempt to elicit the wisdom of the inner crowd from their employees should carefully design the elicitation of second guesses, so as to discourage them from considering how their first guess was wrong,” the researchers conclude.
More from Chicago Booth Review
The debate has raged for 300 years and counting.What Is the Line Between Self-Interest and Selfishness?
We want to demonstrate our commitment to your privacy. Please review Chicago Booth's privacy notice, which provides information explaining how and why we collect particular information when you visit our website.