Papers - Natural

John A List

Cited by*: None Downloads*: None

In 2019, I put together a summary of data from my field experiments website that pertained to natural field experiments (Harrison and List, 2024). Several people have asked me for updates. In this document I update all figures and numbers to show the details for 2023. I also include the description from the original paper below.

Glenn W Harrison, John A List

Cited by*: 23 Downloads*: 10

There has been a dramatic increase in the use of experimental methods in the past two decades. An oft-cited reason for this rise in popularity is that experimental methods provide the necessary control to estimate treatment effects in isolation of other confounding factors. We examine the relevance of experimental findings from laboratory settings that abstract from the field context of the task that theory purports to explain. Using common value auction theory as our guide, we identify naturally occurring settings in which one can test the theory. In our treatments the subjects are not picked at random, as in lab experiments with student subjects, but are deliberately identified by their trading roles in the natural field setting. We find that experienced agents bidding in familiar roles do not fall prey to the winner's curse. Yet, when experienced agents are observed bidding in an unfamiliar role, we find that they frequently fall prey to the winner's curse. We conclude that the theory predicts field behavior well when one is able to identify naturally occurring field counterparts to the key theoretical conditions.

Glenn W Harrison, John A List, Charles Towe

Cited by*: 1 Downloads*: 29

Does individual behavior in a laboratory setting provide a reliable indicator of behavior in a naturally occurring setting? We consider this general methodological question in the context of eliciting risk attitudes. The controls that are typically employed in laboratory settings, such as the use of abstract lotteries, could lead subjects to employ behavioral rules that differ from the ones they employ in the field. Because it is field behavior that we are interested in understanding, those controls might be a confound in themselves if they result in differences in behavior. We find that the use of artificial monetary prizes provides a reliable measure of risk attitudes when the natural counterpart outcome has minimal uncertainty, but that it can provide an unreliable measure when the natural counterpart outcome has background risk. Behavior tended to be moderately risk averse when artificial monetary prizes were used or when there was minimal uncertainty in the natural nonmonetary outcome, but subjects drawn from the same population were much more risk averse when their attitudes were elicited using the natural nonmonetary outcome that had some background risk. These results are consistent with conventional expected utility theory for the effects of background risk on attitudes to risk.

Paul Dolan, Robert D Metcalfe

Cited by*: 8 Downloads*: 6

There is increasing research on the exogenous impact of descriptive social norms on economic behavior. The research to date has a number of limitations: 1) it has not de-coupled the impact of the norm and the knowledge required to understand how to change behavior based upon it; 2) it has exclusively used offline but not online (i.e. emails) methods; and 3) it has not understood the impact of financial incentives in conjunction with norms. We address these three limitations using two natural field experiments. We find, firstly, that norms change energy behavior over a 15 month treatment period irrespective of whether information is provided or not. We find that social norms reduce consumption by around 6% (0.2 standard deviations). Norms have has their largest impact on the day that information on the social norm is received, and then decreases over time. Secondly, we do not find that social norms work online (even with experienced consumers who are used to online billing) - social norms de- livered online may have very little beneficial effects on reducing energy use. Thirdly, we find that large financial rewards work very well online in reducing consumption, with a 0.35 change in energy consumption over a four month period. Perhaps most interestingly, we find that the large effect of financial incentives is completely removed when information on social norms is added online.

Michael Kremer, Edward Miguel

Cited by*: 16 Downloads*: 19

We examine social learning using data from a program that promoted use of deworming medicine in Kenyan schools. These drugs kill worms in the body; although people are soon reinfected, treatment interferes with the cycle of transmission, generating positive externalities. Individuals randomly exposed to more information about deworming drugs through their social network were significantly less likely to take the drugs and more likely to believe the drugs are "not effective." This finding is consistent with the hypothesis that those exposed to the program had overly optimistic prior beliefs about net private drug benefits. The combination of strong social effects and extensive social networks among teenagers implies that a "child-to-child" public health approach focused on teenagers will speed social learning. There are large differences between social effect estimates relying on experimental variation (negative estimates) and nonexperimental methods (positive estimates).

John A List

Cited by*: None Downloads*: None

While empirical economics has made important strides over the past half century, there is a recent attack that threatens the foundations of the empirical approach in economics: external validity. Certain dogmatic arguments are not new, yet in some circles the generalizability question is beyond dispute, rendering empirical work as a passive enterprise based on frivolity. Such arguments serve to caution even the staunchest empirical advocates from even starting an empirical inquiry in a novel setting. In its simplest form, questions of external validity revolve around whether the results of the received study can be generalized to different people, situations, stimuli, and time periods. This study clarifies and places the external validity crisis into perspective by taking a unique glimpse into the grandest of trials: The External Validity Trial. A key outcome of the proceedings is an Author Onus Probandi, which outlines four key areas that every study should report to address external validity. Such an evaluative approach properly rewards empirical advances and justly recognizes inherent empirical limitations.

Junsoo Lee, John A List, Mark Strazicich

Cited by*: 0 Downloads*: 2

In this paper we examine temporal properties of eleven natural resource real price series from 1870-1990 by employing a Lagrangian Multiplier unit root test that allows for two endogenously determined structural breaks with and without a quadratic trend. Contrary to previous research, we find evidence against the unit root hypothesis for all price series. Our findings support characterizing natural resource prices as stationary around deterministic trends with structural breaks. This result is important in both a positive and normative sense. For example, without an appropriate understanding of the dynamics of a time series, empirical verification of theories, forecasting, and proper inference are potentially fruitless. More generally, we show that both pre-testing for unit roots with breaks and allowing for breaks in the forecast model can improve forecast accuracy.

Loukas Balafoutas, Nikos Nikiforakis

Cited by*: 22 Downloads*: 77

Extensive evidence from laboratory experiments indicates that many individuals are willing to use costly punishment to enforce social norms, even in one-shot interactions. However, there appears to be little evidence in the literature of such behavior in the field. We study the propensity to punish norm violators in a natural field experiment conducted in the main subway station in Athens, Greece. The large number of passengers ensures that strategic motives for punishing are minimized. We study violations of two distinct efficiency enhancing social norms. In line with laboratory evidence, we find that individuals punish norm violators. However, these individuals are a minority. Men are more likely than women to punish violators, while the decision to punish is unaffected by the violator's height and gender. Interestingly, we find that violations of the better known of the two norms are substantially less likely to trigger punishment. We present additional evidence from two surveys providing insights into the determinants of norm enforcement.

Indranil Goswami, Oleg Urminsky

Cited by*: None Downloads*: None

We present a complete empirical case study of fundraising decisions that demonstrates the importance of in-context field experiments. We first design novel matching-based fundraising appeals. We derive theory-based predictions from the standard impure altruism model and solicit expert opinion about the potential performance of our interventions. Both theory-based predictions and descriptive advice suggest improved fundraising performance from a framing intervention that credited donors for the matched funds (compared to a typical match framing). However, results from a natural field experiment with prior donors of a non-profit showed significantly poorer performance of this framing compared to a regularly framed matching intervention. This surprising finding was confirmed in a second natural field experiment, to establish the ground truth. Theoretically, our results highlight the limitations of both impure altruism models and of expert opinion in prediction complex "warm glow" motivation. More practically, our results question the availability of useful guidance, and suggest the indispensability of field testing for interventions in fundraising.

Pablo Celhay, Paul Gertler, Paula Giavagnoli, Christel Vermeersch

Cited by*: 0 Downloads*: 32

We show that fixed costs of adjustment as opposed to low returns likely explain why better quality care practices diffuse slowly in the medical industry. Using a randomized field experiment conducted in Argentina, we find that temporary financial incentives paid to health clinics for the early initiation of prenatal care 'nudged' providers to test and develop new data driven strategies to locate and encourage likely pregnant women to seek care in the first trimester of pregnancy. These innovations raised the rate of early initiation of prenatal care by 34% while the incentives were being paid in the treatment period. We follow health clinics over time and find that this increase persisted for at least 24 months after the incentives ended. In the absence of incentives, even though it is in the clinics' interest to stimulate early initiation of care, the presence of hard to change habits and cost of experimentation made it too expensive to develop and implement new methods to increase early initiation of care. Despite the large increases in early initiation of prenatal care, we find no effects on health outcomes.

Dean S Karlan, Jonathan Zinman

Cited by*: 2 Downloads*: 20

Information asymmetries are important in theory but difficult to identify in practice. We estimate the empirical importance of adverse selection and moral hazard in a consumer credit market using a new field experiment methodology. We randomized 58,000 direct mail offers issued by a major South African lender along three dimensions: 1) the initial "offer interest rate" appearing on direct mail solicitations; 2) a "contract interest rate" equal to or less than the offer interest rate and revealed to the over 4,000 borrowers who agreed to the initial offer rate; and 3) a dynamic repayment incentive that extends preferential pricing on future loans to borrowers who remain in good standing. These three randomizations, combined with complete knowledge of the Lender's information set, permit identification of specific types of private information problems. Specifically, our setup distinguishes adverse selection from moral hazard effects on repayment, and thereby generates unique evidence on the existence and magnitudes of specific credit market failures. We find evidence of both adverse selection (among women) and moral hazard (predominantly among men), and the findings suggest that about 20% of default is due to asymmetric information problems. This helps explain the prevalence of credit constraints even in a market that specializes in financing high-risk borrowers at very high rates.

Uri Gneezy, Andreas Leibbrandt, John A List

Cited by*: 1 Downloads*: 9

The functioning and well-being of any society and organization critically hinges on norms of cooperation that regulate social activities. Empirical evidence on how such norms emerge and in which environments they thrive remains a clear void in the literature. To provide an initial set of insights, we overlay a set of field experiments in a natural setting. Our approach is to compare behavior in Brazilian fishermen societies that differ along one major dimension: the workplace organization. In one society (located by the sea) fishermen are forced to work in groups whereas in the adjacent society (located on a lake) fishing is inherently an individual activity. We report sharp evidence that the sea fishermen trust and cooperate more and have greater ability to coordinate group actions than their lake fishermen counterparts. These findings are consistent with the argument that people internalize social norms that emerge from specific needs and support the idea that socio-ecological factors play a decisive role in the proliferation of pro-social behaviors.

Amee Kamdar, Steven D Levitt, John A List, Brian Mullaney, Chad Syverson

Cited by*: None Downloads*: None

In this paper, we present the results of a two-year series of large-scale natural field experiments involving hundreds of thousands of subjects.

John A List, Zacharias Maniadis, Fabio Tufano

Cited by*: 0 Downloads*: 3

In his comment, Mitesh Kataria (2014) makes three main points about a specific part of our paper (Maniadis, Tufano, and List 2014), namely about Tables 2 and 3. In our paper, we employ these tables in order to illustrate the idea that very inconclusive post-study probabilities that a tested phenomenon is true may result from novel, surprising findings. The main arguments in Kataria (2014) are the following: First, if P(H0) is unknown, as is often the case with economic applications, the post-study probability can lead to even worse inference than the Classical significance test, depending on the quality of the prior. Second, the simulation in Maniadis et al. (2014) ignores previous assessments of P(H0) and instead utilizes a selective empirical setup that favors the use of post-study probabilities. [Third,] contrary to what Maniadis et al. (2014) argue, their results do not allow for drawing general recommendations about which approach is the most appropriate. (Kataria 2014, abs.) We believe that our work might have been misunderstood by Kataria. Moreover, it seems that some of his claims are not supported by relevant empirical evidence.

Yan Chen, Xin Li, Jeffrey K MacKie-Mason

Cited by*: 13 Downloads*: 11

We implemented the first web-based online field experiments of fund-raising. We embedded our experiment in the Internet Public Library to test four mechanisms: Voluntary Contribution (VCM), Premium, Seed Money and Matching. Although the gift size is not significantly different across mechanisms, the Seed and Matching mechanisms each generate significantly higher user click-through response rate than the Premium mechanism. Because this is one of the earliest embedded, web-based field experiments, we report our methodology findings in some detail. Cookies work better as participant assignment techniques than pop-up windows and elicitation of geographic information. Participant clickstream data that nominally demonstrate a desire to donate is a poor predictor of actual giving.

Omar Al-Ubaydli, John A List

Cited by*: 2 Downloads*: 1

Economists are increasingly turning to the experimental method as a means to estimate causal effects. By using randomization to identify key treatment effects, theories previously viewed as untestable are now scrutinized, efficacy of public policies are now more easily verified, and stakeholders can swiftly add empirical evidence to aid their decision-making. This study provides an overview of experimental methods in economics, with a special focus on developing an economic theory of generalizability. Given that field experiments are in their infancy, our secondary focus pertains to a discussion of the various parameters that they identify, and how they add to scientific knowledge. We conclude that until we conduct more field experiments that build a bridge between the lab and the naturally-occurring settings of interest we cannot begin to make strong conclusions empirically on the crucial question of generalizability from the lab to the field.

John A List

Cited by*: None Downloads*: None

Social scientists have increasingly turned to the experimental method to understand human behavior. One critical issue that makes solving social problems difficult is "scaling" the idea from a small group to a larger group in more diverse situations. The urgency of scaling policies impacts us every day, whether it is protecting the health and safety of a community or enhancing the opportunities of future generations. Yet, a common result is that when we scale ideas most experience a "voltage drop": upon scaling, the benefit-cost profile depreciates considerably. To combat voltage drops, we must optimally generate policy-based evidence. Optimality requires answering two crucial questions: what information to generate and in what sequence. The economics underlying the science of scaling provides insights into these questions, which are in some cases at odds with conventional approaches. For example, there are important situations wherein I advocate flipping the traditional social science research model to an approach that, from the beginning, produces the type of policy-based evidence that the science of scaling demands. To do so, I propose augmenting efficacy trials by including relevant tests of scale in the original discovery process, which forces the scientist to naturally start with a recognition of the big picture: what information do I need to haves caling confidence?

John A List

Cited by*: None Downloads*: None

Presentation Slides

Alan S Gerber, Donald P Green, Matthew N Green

Cited by*: 6 Downloads*: 3

Political campaigns currently make extensive use of direct mail, particularly in state and local races, yet its effects on voter behavior are not well understood. This essay presents the results of large-scale randomized field experiments conducted in Connecticut and New Jersey during state and municipal elections of 1999. Tens of thousands of registered voters were sent from zero to nine pieces of direct mail. The target populations included party registrants with a strong history of voter participation, independents, and a random subset of registered voters. Our results indicate partisan campaign mail does little to stimulate voter turnout and may even dampen it when the mail is negative in tone.

Michael Kremer, Dan Levy

Cited by*: 16 Downloads*: 9

This paper examines a natural experiment in which students at a large state university were randomly assigned roommates through a lottery system. We find that on average, males assigned to roommates who reported drinking in the year prior to entering college had one quarter-point lower GPA than those assigned to non-drinking roommates. The 10th percentile of their college GPA is half a point lower than among males assigned non-drinking roommates. For males who themselves drank frequently prior to college, assignment to a roommate who drank frequently prior to college reduces GPA by two-thirds of a point. Since students who drink frequently are particularly influenced by frequent-drinking roommates, substance-free housing programs could potentially lower average GPA by segregating drinkers. The effect of initial assignment to a drinking roommate persists and possibly even grows over time. In contrast, students' college GPA is not influenced by roommates' high school grades, admission test scores, or family background. Females' GPAs are not affected by roommates' drinking prior to college. Overall, these findings are more consistent with models in which peers change preferences than models in which they change endowments.