The Science of Economics

Nearly 400 years ago Galileo performed the first recorded laboratory experiment, timing balls as they rolled down an inclined plane to test his theory of acceleration. Since that time, laboratory experiments have been the cornerstone of the scientific method. Over the past 50 years, economists have increasingly used the experimental model of the physical sciences and psychology as a way to understand human behavior.

Economists typically make use of "naturally occurring" data (i.e., data generated by natural phenomena), such as information on interest rates, and conduct empirical work using statistical models to explain various phenomena in the market. The laboratory environment has appealed to a growing number of economists because it allows them to gather data under very controlled circumstances, permitting a clean insight into the cause-effect relationship.

Typically, lab experiments in economics are based on the assumption that insights gained in the lab can be directly applied to the outside world, a principle referred to as "generalizability" by University of Chicago professors Steven D. Levitt and John A. List.

In the study "What Do Laboratory Experiments Tell Us About the Real World?" Levitt and List address the challenges of interpreting experimental work in economics by constructing a model to help shed light on experimental results that likely are generalizable. While the basic strategy underlying lab experiments in the physical sciences and economics is similar, the fact that economists study human behavior raises fundamental questions about the ability to extrapolate experimental findings beyond the lab.

"Our study is a reaction to the explosion of experimental economics in the past few decades and the willingness of economists to accept a number that comes out of a lab at face value," says Levitt. "There are many reasons, based in economic theory, why one should not necessarily take lab results lock, stock, and barrel and apply them to the real world."

The authors note that economists typically fall into two camps: those who steadfastly believe in the validity and direct applicability of all lab experimental results and those who are staunch skeptics. Levitt and List advocate a middle ground, considering particular aspects of individual experiments and developing guidelines for whether or not lab results will be readily generalizable to the real world.

Under Observation

According to standard neoclassical economic theory, individuals are intrinsically selfish, making decisions motivated solely by monetary gain.

In Levitt and List's model, decisions are influenced not just by financial implications, but also by at least four other considerations: 1) the extent to which one's actions will be scrutinized by others; 2) the amount of emphasis placed on the process by which outcomes are reached (as opposed to only the outcomes themselves); 3) the context in which a decision is embedded; and 4) the self-selection of the individuals making the decisions.

Being scrutinized and knowing one's behavior is being recorded might lead people to act differently in the lab than in many market settings. The authors argue that people's choices are driven not only by a desire to maximize wealth, but also by the desire to "do the right thing." When people know that they are being watched, they will likely put more weight on the morality component of making decisions.

In some cases, the basic structure of a lab experiment can lead to questionable results, as discussed in List's recent study "Dictator Game Giving is an Experimental Artifact." In a simple "dictator game," two people bargain over a fixed amount of money. The dictator decides what the split of money will be. Such games test issues of fairness, equity, and reciprocity.

In one of the first dictator game experiments, a researcher gave subjects a choice between dictating an even split of $20 with another student or an uneven split ($18/$2) favoring the dictator. The majority of the students opted for an equal split.

While in the early dictator game experiments recipients do not know who gave them the funds or why, List's new study includes a variation whereby dictators can choose to "take" money from the other player. This manipulation leads to drastic changes in behavior: far fewer subjects are willing to give money when the range of choices includes taking the other player's money. The new result suggests that the altruism observed in previous versions of the dictator game was partly due to people's fear of looking stingy. Once they can avoid looking stingy by simply not taking the other player's money, subjects are far more likely not to give away their own money.

List's variation on the dictator game experiment provides an important example of why it is problematic to measure social preferences in the lab environment. A simple manipulation of the parameters of the experiment can severely influence the results and essentially eradicate any altruistic behavior.

"Typical laboratory dictator games systematically exaggerate the amount of altruistic behavior relative to what we expect in most naturally occurring markets with symmetric action sets," says List.

People are not Bumblebees

In a typical lab experiment, subjects are keenly aware that their behavior is being monitored, recorded, and subsequently scrutinized. Social psychologists have observed that lab subjects naturally tend to infer from elaborate explanations that they are supposed to behave in a certain way.

As a result of increased scrutiny, Levitt and List predict that behavior in the lab will be more influenced by moral concerns and less aligned with maximizing wealth than in many real-world settings.

"In general, people care about doing the right thing," says Levitt. "Scrutiny does play a role in altruistic behavior though. If I'm in church and the Pope is passing around the hat, I'm likely to be a little more generous because the Pope is watching."

The model predicts that in experiments that have both morality and wealth components, financial concerns will become increasingly important as the stakes rise. Given the low stakes typically used in lab experiments, Levitt and List expect that lab results might overstate altruistic preferences relative to situations in the real world over greater stakes.

Human behavior also is heavily influenced by context: a complex set of relational situations, social norms, past experiences, and lessons gleaned. A researcher can control payoffs and descriptions of the way the game is played in a given experiment, but not the context that actors themselves bring to the game. Unlike the natural phenomena of bumble-bees, bacterial genes, or water, the phenomena of interest to researchers measuring human behavior is less likely to remain constant in a different context.

The beliefs and experiences that people bring into the lab greatly complicate the interpretation of lab results. A researcher would like to believe that it is possible to change one variable (such as switching the color of paint on a wall from yellow to red), observe how individual behavior changes, and thus measure the impact of having a red wall. The authors caution that interpreting such results depends on whether people in a particular society have a preference for red walls or whether people associate the color with religious meanings, among other possible explanations.

Most experiments in this field have been conducted using students who self-select into the experiments. Such students may readily cooperate with the researcher and seek social approval. In contrast, market participants are likely to be a highly-selected sample of individuals whose traits help them excel in the marketplace.

Overall, the lack of connection between moral and wealth-maximizing actions can lead lab experiments to yield quantitative insights that may not be readily extrapolated to the outside world.

Field Experiments

Based on their analysis, Levitt and List see an important role for traditional lab experiments in economics. The authors compare experimental economists to aerodynamicists who use wind tunnels to test models of proposed aircrafts, cars, and trains. The wind tunnel provides the engineer with valuable data on scale models much like the lab provides economists with important insights on economic phenomena.

The authors draw three conclusions regarding future research design and interpretation in experimental economics. First, combining lab analysis with a model of decision making expands the potential role of lab experiments. By anticipating the types of biases common to the lab, experiments can be designed to minimize such biases. Knowing the signs and likely impact of any biases induced by the lab, the researcher can extract useful information from a study, even if the results cannot be seamlessly applied to the real world. Second, much can be learned by focusing on qualitative rather than quantitative insights. Third, it is not helpful to draw a sharp line dividing lab experiments and data generated in natural settings. Each approach has its strengths and weaknesses, and a combination of the two is likely to provide more insights than either in isolation.

Field experiments incorporate the strengths of both approaches and can serve as a bridge connecting two empirical methods. List suggests that the most important element in constructing an experiment is the representativeness of the environment. A field experiment applies scientific methods to experimentally examine an intervention in a naturally occurring environment rather than the lab. These experiments generally randomize subjects into treatment and control groups and compare outcomes between the groups. Randomizing who receives the treatment and who does not is similar to tests of new drugs where some patients receive the drug and others receive a placebo. Such randomization is necessary to prevent other factors from influencing the outcome.

By allowing researchers to manipulate key elements of a decision problem, field experiments retain one of the most attractive features of lab experiments. However, because field experiments are conducted in naturally occurring environments, often with participants unaware that they are being scrutinized, the resulting findings are typically generalized more easily.

Another study conducted by List, "Toward an Understanding of the Economics of Charity: Evidence from a Field Experiment," highlights the field experimental method by exploring the economics of charity. Approaching nearly 5,000 households that are randomly divided into four treatments, the data shed light on key issues within charitable fundraising. List notes, however, that "even though these data are from a field experiment, to transfer these insights to another environment we need economic theory; the Levitt-List framework provides one structure to allow such generalizability."