(gentle music)
Narrator: The role and value of higher education has been closely scrutinized in recent decades. So too have the processes that colleges use to determine who they admit. Traditionally, colleges have considered multiple factors, such as transcripts, grades, letters of recommendation, college essays, and extracurricular activities. For the past 100 years, tests such as the ACT or the SAT have also played a significant role, providing administrators with standardized metrics for evaluating prospective students.
Frankel: So if I get an A in a calculus class in my high school with this teacher, and you get an A in that calculus class in a completely different school with a different teacher, how do we compare the two? What do those mean? If I get a test score of 1400 on the SATs and you got a test score of 1400, we know what that means. We don’t have to figure out where you’re from, what your background is. The test is the same test everywhere. So that’s the benefit of these standardized tests.
Narrator: That’s Chicago Booth’s Alex Frankel. He and his coauthors wanted to explore the effectiveness of a trend among colleges to move to a test-optional policy, one that doesn’t require applicants to submit standardized test scores. Seen as a way to increase college access for students from underrepresented backgrounds, such policies have proliferated since the turn of the 21st century, and COVID-19 accelerated that adoption.
Frankel: In economics, there’s a very simple observation that applies very generally, which is you can’t make better decisions with less information. Giving me more information should only help me make decisions, no matter what my objectives are, no matter what my goals are. So if we think about the test scores as a useful piece of information that helps predict everything selective colleges say they are interested in predicting—so test scores are predictive of your first-year grades, your grades at graduation, your graduation rates, how well you perform after graduation—why not look at this information to make a better decision about what students you want to admit?
Narrator: The researchers ran an exercise that questioned whether the availability of more information truly aids decision-making when student circumstances, such as study habits and access to resources for test preparation, vary widely. They developed a mathematical model to explore if these real-world factors might challenge the notion that more information always improves decision outcomes.
Frankel: We show that under some pretty broad set of assumptions, the basic logic really does hold that letting students submit applications without standardized test scores really is effectively just hiding information from the colleges, leading them to make worse decisions than they otherwise would make. Because it’s the colleges who are deciding how to use this information. It’s the colleges themselves who get to see the test scores and decide, “Here is the algorithm I’m going to use. Here’s how I’m going to train my admission officers to factor in the test score.”
Narrator: When considering why colleges might continue to remain test optional, the researchers propose several possibilities.
Frankel: For instance, imagine a student who says, “I simply refuse to apply to a test-mandatory college. I think that’s a bad signal of the college’s values. I don’t want to be associated with any such institution.” In that case, it would make a lot of sense for colleges to say, “All right, well, I don’t wanna require test scores because then students aren’t going to apply to my college, even though the test scores are very valuable.”
Narrator: Another theory involves the decision-makers themselves.
Frankel: One possibility that doesn’t sound so crazy is that college admission officers can go to as many trainings as you want. You can tell them, “Here is the formula you should use. Here’s how much weight to put on different factors.” And then they sit down, they look at an application, and the test score at the top is just too salient for them. They can’t avoid thinking about it.
Narrator: Colleges also may not want to deal with the outside scrutiny that could come with being a test-mandatory school.
Frankel: And so think about people like the alumni of the college, the parents of the current students, politicians, the media, the general public. For some reason, all of these people seem pretty interested when selective colleges like Harvard, University of Chicago, Columbia, Yale, and so forth are changing around their admission policies. And so it might be that colleges want to get a good class of students, but they also want to avoid being yelled at by all of these other stakeholders who just care a lot about their admission policy. And you can imagine the following situation: There’s a student with a low test score that the college likes and a student with a high test score that the college doesn’t like. If the college sees the test scores and the broader public knows that the college saw these test scores for admission, maybe people outside get very angry at the college that they’ve admitted the student with the low test score and rejected the student with the high test score. This is unfair. Maybe we want to sue them. We want to pass a law that bans this, or we just write a lot of very angry articles in the newspaper. But if the same college admits and rejects the same pair of students without seeing the test scores, now maybe people are a lot less angry. Well, there’s no way the college could have known that it was admitting the low-test-score student over the high-test-score student. Given the information the college had, it made a pretty reasonable decision. And so maybe now we don’t have to be so angry at the college.
Narrator: Whatever the drawbacks of test requirements, the researchers don’t see standardized tests disappearing from the admissions process anytime soon.
Frankel: Even while the majority of them have remained test optional since COVID, a number of them have moved back, and it looks like some more are on the way of moving back toward using test scores. And it’s hard to imagine those colleges, at least, flipping back. Some of the colleges that have gone back to test mandatory have said, “Look, we’ve seen our data and we’ve realized that these test scores are actually really important for our admissions process. They do help us predict which students are going to thrive.” And so it’s hard to imagine those colleges going back. It’s easy to imagine other colleges requiring test scores again because they’re looking at the same data and making the same conclusions that, yeah, test scores actually are helpful.
