Illustration by Chris Gash
When Lauren Young’s daughter, Claudia, submitted the list of middle schools she would like to attend to Brooklyn, New York’s District 15 school lottery, she knew that neither her test scores, nor her grades, nor her behavior reports from elementary school would help her land a spot at her top choice. Nor would her family’s neighborhood: unlike most public-school districts in the United States, where students historically have gone to their local school as a matter of course, District 15 does not automatically assign students to the school closest to home.
Instead, as with an increasing number of children in urban school districts across the US and around the world, Claudia would earn a spot through an algorithm designed to match her with a school on the basis of her preferences, demographics, and the school’s open seats. As a result, deciding which schools to put on her wish list, and how to rank them, was a yearlong ordeal of background research, school visits, and endless discussion. “She faded in the process,” says Lauren. “She got tired of it before we were done.” Lauren lasted longer but still felt it was a “system designed to drive me crazy.”
Recent scandals and controversy around high-school and college admissions reflect the pressure parents can feel to get their children into coveted schools. The competitive landscape is fierce, as made clear by executives and celebrities spending hundreds of thousands of dollars on fraudulent schemes to secure their children university acceptance letters. And the stakes are high: part of the concern over New York City’s elite public high schools failing to represent the racial profile of the city stems from an understanding that good schools can confer lifelong advantages, offering a way up the socioeconomic ladder through both caliber of instruction and contact with high-achieving peers.
These controversies focus on admittees to selective schools, and yet similar dynamics are increasingly playing out for students such as Claudia, competing for places that are offered independent of aptitude tests and admissions officers—though often still with race and class in mind. Her district moved to nonselective placements for this year’s incoming sixth graders in an attempt to both simplify the admissions process and reduce the racial and socioeconomic segregation associated with selection on the basis of tests, grades, and recommendations. The placement system takes family income and language background into account, with schools aiming for socioeconomic-diversity targets that reflect the district: 52 percent of students should qualify for free or reduced-price lunch, live in temporary housing, or speak English as a second language.
Whether District 15’s plan succeeds is a matter of keen interest to other school districts, politicians, families, and educational researchers—but also to economists. Because while public education has long been a touchstone for debates about race, equality, and a society’s priorities, open-enrollment lotteries have more recently made schools a crucible for research into matching systems, behavioral economics, and game theory. Will more school choice mean greater overall welfare for students? Could it also reduce inequality in a system that currently favors wealthier, better-connected, and often racially homogenous families? Or do such systems require too much time, energy, and knowledge for parents and students to use them to their best advantage?
While considerable attention has been paid to the experiences of participants in these markets, often overlooked are the decisions that define the systems themselves. In each case, policy makers and other administrators hope to maximize particular outcomes, and use the details of market design to do so. It is here, in particular, where economic research is playing a role, revealing the benefits and drawbacks of different approaches—and producing insights that often lead to real-world change.
An invitation to game
Chicago Booth’s Seth Zimmerman got interested in school lotteries as an economics graduate student at Yale in the late 2000s. The public-school system in New Haven, Connecticut, where Yale is located, had embraced open enrollment since at least the late 1990s, following a landmark Connecticut Supreme Court ruling that the state was responsible for ensuring equal access to the best public schools. Under Connecticut’s Open Choice program, which the state established in 1997, children in urban school districts such as New Haven’s can attend school in nearby suburban districts, and suburban children can enroll in urban schools.
Other districts have been making similar moves. Since the late 1980s, 46 states have passed laws that either allow or mandate open-admissions policies, according to the nonprofit Education Commission of the States, with the idea of giving all students access to their area’s best schools, even if their families cannot afford to live in the neighborhoods that house those institutions. This impulse has been encouraged at times by government incentives, according to Diane Ravitch, a professor of education at NYU, such as the Obama administration’s Race to the Top, which distributed more than $4 billion across states that undertook significant educational reform. More broadly, the trend toward open admissions also aligns with two philosophies that have dominated US public education over the past 30 years: that schools should compete with one another for students (and therefore resources, since government funding tends to be linked to student numbers), and that schools should be held publicly accountable for student performance, at once facilitating and fueling this competition.
President George W. Bush’s 2001 No Child Left Behind Act enshrines both approaches in law, and has led to the expansion of a suite of programs and policies that give students and their families options apart from their nearest neighborhood school. Sometimes these involve the creation of new institutions entirely, such as charter schools, which are run by private organizations but are publicly funded, or magnet schools, which are public schools (or programs within public schools) with specialized offerings and selective-enrollment policies. Open enrollment, on the other hand, usually involves working with the schools a district already has, but expanding access to them—often with the intent of addressing school segregation when residential segregation has proved unyielding.
But researchers and policy makers alike have noted that the mechanism for transforming a captive-market system involving hard borders between school catchments into a fair and functioning free-market system is far from simple. In order to match open-enrollment students with available places, the New Haven Board of Education adopted a mechanism that encouraged students and their families to rank schools strategically, considering not only which schools they liked most, but also which schools they had the best chances of getting into after taking into account demand from other students.
Say Student 1 had found in School A his dream destination, but he also knew that School B—where his sister was enrolled—would be an adequate option. What he really wanted to avoid was School C. You might think he should rank the institutions in that order: A, B, C. But the way the New Haven algorithm worked, each student’s first-choice school was prioritized, meaning that if Student 1 missed out on School A in the first round of matching, he couldn’t count on School B either, despite having a sibling there (an advantage, according to the rules of that lottery). This was because anyone who had put School B at the top of her list would get priority over Student 1, who had ranked it lower; Student 1 would have squandered his sibling advantage. As a result, his chances of landing in School C could well have been higher if he had ranked his preferences honestly than if he had said School B was his first choice.
Research suggests that this system, versions of which school boards across the US have embraced, can increase overall welfare, on top of other advantages. For example, Duke’s Atila Abdulkadiroğlu, whose seminal work in the early 2000s with Tayfun Sönmez of Boston College prompted Boston Public Schools to rethink the mechanism it employed (see “School choice in practice,” below), argued in a 2011 paper with Yeon-Koo Che of Columbia and Yosuke Yasuda of Osaka University that it may be preferable to some other school-assignment systems in part because it breaks ties between students who want the same school by rewarding the student who ranked it higher. In other systems, such decisions are usually determined by chance, using a random lottery. And these situations are common: even if siblings or residential proximity give some students greater priority over others, many students will still have equal claim on the same school. (This dilemma also occurs at the college level, where universities sometimes use early-acceptance policies to help them choose between equally qualified candidates on the basis of the candidates’ passion for the school.) Abdulkadiroğlu and his coresearchers argue as well that a system that rewards strategizing might even benefit “naive” students ill-placed to game it, since strategic students would avoid putting popular schools at the top of their lists, leaving more spots for everyone else.
But the drawbacks are evident. First, there is significant room for regret if a student feels she has misjudged her chances at getting into one school or another. Second, the system rewards risk-taking, but for only select students: if you had a 1/100 chance of getting into your favorite school and a 1/10 chance at your second favorite, you should rank your second-favorite school at the top of your list—unless you had the option of opting out of the system entirely should you miss your 1/100 chance. Students for whom private school is an option can take a chance on the most popular public schools and go private if they don’t get the outcome they want.
More fundamentally, there is no easy rule of thumb for understanding your chances. Even if you could effectively parse historical data on levels of demand for one school or another, others might be doing the same, meaning that how your fellow students are ranking their choices in the same year you are ranking yours—the only year that matters—will remain opaque.
The system leaves even sophisticated individuals befuddled. “Occasionally we’d get notes from Yale faculty with school-aged kids asking us how it worked,” says Zimmerman of the New Haven lottery.
Even systems simpler than the so-called Boston Mechanism then being used by New Haven may be too complicated for many users. Research by Cornell’s Alex Rees-Jones and UPenn PhD candidate Samuel Skowronek finds that 23 percent of medical students mistakenly misrepresented their preferences in a strategy-proof system that mirrored the real-life residency match that they had only just taken part in. Chicago Booth’s Eric Budish and UPenn’s Judd Kessler have found similar results for course allocation at the Wharton School at the University of Pennsylvania (see “You can’t always say what you want,” below). These findings and experiences raise concerns that families lacking time or statistical know-how would be least prepared to navigate algorithmic matching processes—and perhaps more fundamentally, that such processes may not be as effective as anticipated if students and their parents don’t understand them.
Is honesty the best policy?
Last September, Kirsten Youngren attached a handwritten list of 27 high schools to her refrigerator. Over the following six months, the slip of paper became crowded with amendments: dates and times for school visits, and long, wiggly arrows noting schedule changes; checks, strike-throughs, and stars to code experiences at the schools; and highlighting to indicate long commutes.
Youngren’s daughter, Bridget, is a rising ninth grader in Lower Manhattan who was navigating a wide variety of options for high school. She had taken the controversial test that determines placement at eight of New York’s elite, or “specialized,” high schools, mostly under the influence of her best friend, who wanted to go to Stuyvesant, the fourth-best public high school in the US, according to national school-ranking website Niche, and the school at the center of the recent debate in New York over admissions policies and race. Bridget was also interested in “screened” schools—that is, public schools that prioritize interested students on the basis of academic performance, attendance records, or art portfolios—as well as nonselective public schools, which would admit her on the basis of New York’s strategy-proof lottery system, which Atila Abdulkadiroğlu and his collaborators, MIT’s Parag Pathak and Harvard’s Alvin Roth, helped design. (The previous system left tens of thousands of students unmatched until late summer and then often assigned them to schools they didn’t want; the new design helped Roth win the Nobel Prize in Economic Sciences in 2012.) And Bridget was weighing three Catholic high schools.
Wharton’s system was based on a “fake money” auction model in use at many schools at the time, in which students were provided an endowment of points or other currency and used this endowment to bid strategically for the courses they hoped to enroll in. After the initial round of bidding, students could buy or sell course seats in subsequent auctions. Such systems have been criticized by some economists for driving participants to bid strategically, and thereby hide their true preferences, potentially leading to inefficient and unfair course allocations. At Wharton, the system worked poorly enough that only about a quarter of students thought it was fair.
In 2011, shortly before Wharton decided to change its system, Budish published research describing a mechanism he calls “approximate competitive equilibrium from equal incomes.” Following this mechanism, students report their preferences directly (rather than via bids) and are randomly assigned roughly (but not exactly) equal endowments of points. Prices for all of the courses are set by a computer, which searches for a set of prices that will result in each student getting the best schedule she can afford (given her budget and reported preferences) and also clear the market, or balance supply and demand for each course. The work for the computer is onerous—for a context as complex as Wharton’s, there are more possible sets of prices than there are atoms in the universe, Budish points out—but for students, who are no longer required to be strategic and canny bidders, the task is greatly simplified. Whereas the old Wharton system left room for students to regret their bidding strategy, Budish’s mechanism requires only that they truthfully express what they want.
Ordinarily, Budish might have tested the new approach by assigning study subjects—not necessarily Wharton students—certain preferences for course schedules and analyzing whether the mechanism created a fairer and more efficient distribution of the most popular classes. This is how the vast majority of market-design experiments are performed.
But economists have recognized that a common assumption in market-design research—that people are able to articulate what they want—is often untrue, particularly for complex decision-making problems. This was crucial for Budish, because the superiority of his mechanism over others hinged on its ability to get an accurate reading of market participants’ desires, even if those desires were complicated. At Wharton, for example, when MBA candidates consider courses, they are balancing the draw of star professors, the desire for manageable workloads, and the need to fulfill degree requirements. The number of courses on offer is vast, and the possible schedules vaster still—in the hundreds of millions.
So instead of studying a sample of subjects randomly assigned to certain preferences, Budish and Judd Kessler, an economist at Wharton, asked a sample of real Wharton students to report their real schedule preferences, used Budish’s mechanism to come up with hypothetical schedules, and then offered the students multiple possible schedules to see which more or less met their needs (a bit like what you might experience in an optometrist’s office: “Can you see more clearly with Schedule A or Schedule B?”). This allowed them to assess how effectively the students reported their own preferences.
While the researchers were expecting the difficulty of reporting preferences to reduce the method’s relative effectiveness to some extent, the degree to which it did surprised them. Budish’s mechanism outperformed Wharton’s incumbent system on fairness and efficiency, but students’ reporting mistakes—some of which likely stemmed, Budish notes, from the time constraints placed on them and the unfamiliarity of the lab setting in which the experiment took place—reduced that gap significantly. Moreover, no one type of mistake was particularly prevalent, meaning no easy fix presented itself.
Still, the improvements, as well as factors such as students’ favorable perceptions, persuaded Wharton to adopt the method, with some adjustments and significant user training and support to help maximize accurate reporting. In the first year of the new system, 65 percent of MBA candidates said they thought course allocation was fair.
Wharton’s move contributes to a tradition of market-design research making its way from theory to practice. “Al Roth championed the idea that economists act like engineers,” says Budish, who studied under the Nobel Prize–winning Stanford economist. “That requires a general engagement with problems, and being interdisciplinary in looking for solutions. It’s also meant there’s a great feedback loop between theoretical research and real-world problems, which is powerful—and fruitful.”
Eric Budish, “The Combinatorial Assignment Problem: Approximate Competitive Equilibrium from Equal Incomes,” Journal of Political Economy, December 2011.
Eric Budish and Judd Kessler, “Bringing Real Market Participants’ Real Preferences into the Lab: An Experiment that Changed the Course Allocation Mechanism at Wharton,” Working paper, December 2018.
The fact that Bridget’s parents could pay Catholic-school tuition—though not the much-higher fees at other New York private schools—gave her more options than many of her peers had. But her mother felt the advantage was even more deeply rooted than that: Kirsten, an architect who works from home for a firm that accommodates flexible hours, was able to invest a huge amount of time in the process, researching the schools and the system itself. “There’s something to be said about the amount of work that needs to be done,” she says. “It was sometimes awkward being one of the few parents at Bridget’s middle school who had the flexibility to put in that time.”
And even though the middle school—a public school on the Lower East Side with a high number of low-income students—and others like it offered help to students and families navigating the lottery process, sometimes that advice was ill-conceived: Kirsten recalls someone at a presentation last fall saying that students should think twice about applying to the most popular schools, since their chances of getting in were small. Kirsten ignored the suggestion, but parents who took the advice to heart might have needlessly given up their child’s chance, since New York’s matching system rewards “truth telling,” or honest ranking of a student’s favorite schools, over strategy games.
In their research on New Haven’s schools, Zimmerman and his coresearchers wanted to quantify the effects of this sort of inequality, and particularly to understand how it played out in a system even more strategically complicated than New York’s. Past research assumed either that students and their families were strategizing correctly or that they were making one of a limited number of possible mistakes, such as not knowing their own priority group or playing naively by simply listing their choices in order of their preferences. With Princeton’s Adam Kapor and Christopher Neilson, Zimmerman surveyed 417 families who took part in the New Haven high-school lottery in 2015 and 2017, asking how they ordered their rankings and why, and then analyzed the matches that had resulted.
They find that people consistently under- or overestimated their odds of getting into a school and strategized poorly as a result. Moreover, the researchers’ models showed that unless mistakes could be reduced by a third for ninth-grade matches and by almost two-thirds for elementary-school placements, a system that rewards strategic play would perform worse than one in which families simply rank their preferences honestly.
“It’s important to think about mistakes,” Zimmerman says. “If you could get people to play the system perfectly, that would be the better matching mechanism. But we were far from that.” The researchers tried developing an app that would increase families’ understanding of their odds and help them strategize accurately. But they soon saw that simply adopting a truth-telling approach made more sense. Their work found an audience in New Haven Public Schools, and for the 2019–20 school year, the city began employing a matching algorithm similar to New York’s.
Districts that want to strategy-proof their matching systems—remove the incentive for students to report anything but their true school preferences—typically choose between two types of algorithms, top trading cycles and deferred acceptance. A top-trading-cycles mechanism prioritizes efficiency: it allocates spots at various schools in such a way that no two students would both be made better off by swapping their school assignments. Deferred acceptance creates stable matches; a match is unstable if a student at one school prefers another, and has higher admissions priority (based on the hierarchy of student traits the district uses to determine priority at different schools) at her target school than any of its admitted students does.
Because DA produces stable matches, DA systems are more restricted by districts’ priority structures than are TTC systems. This can prevent swaps of schools between students, even when both students would prefer the trade. The result may be lower student satisfaction, but the district might accept that cost if it sees a benefit in preventing swaps—for instance, to limit the average distance between students’ homes and schools, to keep bussing costs down.
Chicago Booth’s Jacob Leshno says that currently most districts don’t use TTC systems, and he suggests a potential reason many have opted for DA instead: TTC systems are harder to explain to students who don’t get the schools they want. In a DA system, when a student doesn’t get the match she wants, the explanation is straightforward: she didn’t have high enough priority at that school, based on whatever priority structure the district has created. But in TTC, it can be more difficult to explain in a nontechnical way why a student received a disappointing match, except that it was part of an efficient distribution of satisfactory matches overall.
However, Leshno and Stanford’s Irene Lo wanted to help administrators make full use of their options for school-matching systems by providing tools to help explain how TTC school-assignment algorithms work. Their research demonstrates it’s possible to explain matches under TTC systems to students and parents using the same palatable notion that applies to DA systems, removing a big impediment to their implementation.
In essence, a given student has an endowment based on her priority at each school, which the researchers represent as tokens. Her budget of tokens is different for each school, since some factors—such as whether she lives nearby—are school specific. Each school also has a “price,” or a minimum number of each token students need in order to afford admission. Because students can trade school assignments based on their preferences, a student with insufficient tokens for School A, her target school, could potentially still afford admission using tokens for School B, provided there was at least one student admitted to School A who preferred School B but didn’t have enough B tokens. After assignments are made, districts can even publish prices publicly, showing the minimum number of each token that was required for admission to each school, to help students and their families verify the student is matched to their favorite of all the schools they can afford. The researchers suggest that being able to frame outcomes in this way may make it easier for districts to adopt TTC mechanisms for school choice.
Horizontal versus vertical variation
One realization that struck Lauren Young periodically as she was considering Claudia’s middle-school options was that, for all the schools did to advertise their differences—art every day versus gym every day, a focus on math and science, a forward-thinking principal—they were ultimately more alike than not. “The system encouraged parents to overemphasize what’s special about a school, and you forget that they’re all teaching the same curriculum and they’re really only different at the margins,” she says. When she thought about it this way, the District 15 lottery was “a choice, but not a choice.”
In economics, vertical differentiation refers to variation within a set of goods such that all consumers have the same preferences—one product is of higher quality than the rest, and everyone prefers that product. Horizontal differentiation occurs when different consumers have preferences for different things. Generally speaking, school choice has the opportunity to make a bigger difference, Leshno says, when there’s greater horizontal differentiation—that is, when students don’t all want the same thing.
But Leshno and Lo’s larger finding is that the debate over which algorithm to use risks obscuring a more powerful lever for student happiness: a school district’s priorities. Should the sibling of a current student get priority at a particular school? Should children have first dibs on their neighborhood school? Should attendance at a school open house give students an advantage in a lottery—a factor that diversity advocates argue hurts poorer, nonwhite families? As districts decide which traits to prioritize and how much relative weight to give to each, they determine, to a large extent, the set of schools available to each student to choose from. That makes the choice of priority structure hugely important, no matter the algorithm: generally speaking, student welfare will be higher in a district with a sensible priority structure than in a district with a poorly designed one, regardless of the mechanism either is using.
“When you have priorities that make sense, that’s where things change, and so that’s where the discussion should be,” Leshno says. He emphasizes that districts need to be transparent about not only their choices of priorities, but also how those choices affect which sets of schools are available to which students.
Nor should the debate over matching mechanisms disguise the limits of market design. “All of this only helps to the extent that we create better matches,” Zimmerman says. “We’re still allocating students to the schools we have: creating better matches doesn’t fix the problem of bad schools.”
Carol Burris, a former high-school principal, executive director of the Network for Public Education, and author of the book On the Same Track: How Schools Can Join the Twenty-First-Century Struggle against Resegregation, is even more critical. Lotteries as the primary vehicle for admissions do not decrease segregation, she argues. And in the meantime, they elide problems related to curriculum, disparities in which can exacerbate segregation. Schools try to distinguish themselves using their varying offerings, when in fact every student deserves to attend a school with a rich and varied curriculum. Choice, says Burris, often becomes a substitute for the hard (and more expensive) work of improving individual schools.
School matching is not necessarily a zero-sum pursuit, Leshno says. School choice does not create additional educational resources, but with a well-designed system, a district can make the most of the resources it does have. When they’re functioning as they should, school-matching systems have the potential to better pair what schools have to offer with what individual students need, “even if you accept the fact that there aren’t enough good seats for everybody.”
Choice only goes so far
Lauren Young felt good about the list of schools Claudia finally submitted to District 15, but more ambivalent about the process itself. They had left off two schools that Young and her husband felt were entirely wrong for their daughter, and ranked all nine others. Private school was not a consideration, for financial and philosophical reasons, but other ways of opting out of the system did cross Young’s mind from time to time: “The whole experience made me realize why people move to the suburbs,” she says. Kirsten Youngren felt similarly, visiting friends in New Jersey last year and wondering whether her family shouldn’t perhaps uproot from Manhattan and start building a life in a place where kids go to the local public school, and can count on that local public school being sound.
In April, Claudia found out where she will be starting sixth grade this fall: Brooklyn Collaborative, which had been fourth on her list. The three schools she ranked higher were historically white-majority institutions with affluent student bodies and the best academic reputations in the district. Meanwhile, very few affluent children—a group into which Claudia falls—attended Brooklyn Collaborative in the past; its incoming class of sixth-graders will now be the district’s most affluent, a product of the district’s diversity priorities and the way parents and students ranked their choices.
Lauren is happy with the school’s pedagogy. Claudia, meanwhile, was disappointed not to have gotten into the arts-focused school she had ranked No. 1, and appealed the decision unsuccessfully. Given the strength of her desire to get into her No. 1 school, a strategic-play mechanism might have worked in her favor, but it is impossible to know for certain.
A month earlier and across the East River, Kirsten Youngren’s daughter, Bridget, learned that she had been matched with the 12th-ranked school on her list of 12 New York City public high schools. Bridget decided to enroll in a Catholic school this fall.
- Atila Abdulkadiroğlu, Parag Apthak, and Alvin Roth, “The New York City High School Match,” American Economic Review, May 2005.
- Atila Abdulkadiroğlu, Yeon-Koo Che, and Yosuke Yasuda, “Resolving Conflicting Preferences in School Choice: The ‘Boston Mechanism’ Reconsidered,” American Economic Review, February 2011.
- Atila Abdulkadiroğlu and Tayfun Sönmez, “School Choice: A Mechanism Design Approach,” American Economic Review, June 2003.
- Adam Kapor, Christopher Neilson, and Seth Zimmerman, “Heterogeneous Beliefs and School Choice Mechanisms,” Working paper, September 2018.
- Jacob Leshno and Irene Lo, “The Cutoff Structure of Top Trading Cycles in School Choice,” Working paper, November 2018.
- Alex Rees-Jones and Samuel Skowronek, “An Experimental Investigation of Preference Misrepresentation in the Residency Match,” PNAS, November 2018.
More from Chicago Booth Review
We want to demonstrate our commitment to your privacy. Please review Chicago Booth's privacy notice, which provides information explaining how and why we collect particular information when you visit our website.