Papers - All

Luigi Butera, Philip J Grossman, Daniel Houser, John A List, Marie-Claire Villeval

Cited by*: None Downloads*: None

Creation of empirical knowledge in economics has taken a dramatic turn in the past few decades. One feature of the new research landscape is the nature and extent to which scholars generate data. Today, in nearly every field the experimental approach plays an increasingly crucial role in testing theories and informing organizational decisions. Whereas there is much to appreciate about this revolution, recently a credibility crisis has taken hold across the social sciences, arguing that an important component of Fischer (1935)'s tripod has not been fully embraced: replication. Indeed, while the importance of replications is not debatable scientifically, current incentives are not sufficient to encourage replications from the individual researcher's perspective. We propose a novel mechanism that promotes replications by leveraging mutually beneficial gains between scholars and editors. We develop a model capturing the trade-offs involved in seeking independent replications before submission of a paper to journals. We showcase our method via an investigation of the effects of Knightian uncertainty on cooperation rates in public goods games, a pervasive and yet largely unexplored feature in the literature.

Omar Al-Ubaydli, John A List

Cited by*: None Downloads*: None

Natural field experiments investigating key labour market phenomena such as unemployment have only been used since the early 2000s. This paper reviews the literature and draws three primary conclusions that deepen our understanding of unemployment. First, the inability to monitor workers perfectly in many occupations complicates the hiring decision in a way that contributes to unemployment. Second, the inability to determine a worker's attributes precisely at the time of hiring leads to discrimination on the basis of factors such as race, gender, age and ethnicity. This can lead to systematically high and persistent levels of unemployment for groups that face discrimination. Third, the importance of social and personal dynamics in the workplace can lead to short-term unemployment. Much of the knowledge necessary for these conclusions could only be obtained using natural field experiments due to their ability to combine randomized control with an absence of experimenter demand effects.

John A List

Cited by*: None Downloads*: None

These are the slides from John A. List's keynote at the 2022 AFE conference.

Laura Forastiere, Patrizia Lattarulo, Marco Mariani, Fabrizia Mealli, Laura Razzolini

Cited by*: None Downloads*: None

This paper revisits results from a field experiment conducted in Florence, Italy to study the effects of incentives offered to high school teens to motivate them to visit art museums and to identify best practices to transform this behavior into a long run cultural consumption. Students belonging to a first group of classes receive a flier with basic information and opening hours of a main museum in Florence, Palazzo Vecchio. Students in a second group of classes receive the flyer and a short presentation conducted by an art expert. Students in a third group of classes, in addition to the flyer and the presentation, receive also a nonfinancial reward in the form of extra-credit points towards their school grade. Taking a Principal Stratification approach, we explore the causal pathways that may lead students to increase their future museum attendance. Within the strata defined by compliance to the three forms of encouragement, we estimate associative and dissociative principal causal effects, that is, effects of the encouragement on the primary outcome, long run cultural consumption, that are associative or dissociative with respect to the effects of the encouragements on the Palazzo Vecchio visit. This analysis allows to interpret these effects as ascribable either to the encouragements, or to the museum visits, or to classroom spillovers. To face identification issues, estimation is performed with Bayesian inferential methods using hierarchical models to account for clustering. The main findings of the analysis are as follows: what seems to matter the most is the motivational incentive (i.e., the presentation), rather than the induced experience, i.e., the Palazzo Vecchio visit.

Syon Bhanot

Cited by*: None Downloads*: None

Perception of peer rank, or how we can perform relative to out peers, can be a powerful motivator. While research exists on the effect of social information on decision making, there is less work on how ranked comparisons with our peers influence our behavior. This paper outlines a field experiment conducted with 3896 households in Castro Valley, California, which uses household mailers with various forms of social information and peer rank messaging to motivate water conservation. The experiment tests the effect of a visible peer rank on water use, and how the competitive framing of rank information influences behavioral response. The results show that households with relatively low or high water use in the pre-treatment period responded differently to how rank information was framed. I find that a neutrally-framed peer rank caused a small "boomerang effect" (i.e., an increase in average water use) for low water households, but this effect was eliminated by competitive framing. At the same time, a competitively-framed peer rank demotivated high water use households, increasing their average water use over the full period of the experiment. This result is supported by evidence that the competitive frame on rank information increased water use for households who ranked "last" in the peer group - a detrimental "last place effect" from competitively-framed rankings.

Syon Bhanot

Cited by*: None Downloads*: None

Social norms messaging campaigns are increasingly used to influence human behavior, with social science research generally finding that they have modest but meaningful effects. One aspect of these campaigns in practice has been the inclusion of injunctive norms messaging, designed to convey a social judgement about one's behaviors (often in the form of encouraging or discouraging language, or a visual smiley or frowny face). While some prominent research has provided support for the use of such messaging as a tool for positive behavior change, causal evidence on the effect of injunctive norms messaging as a motivator (as opposed to just one part of a multifaceted messaging campaign) is limited. This paper presents a field experiment on water conservation behavior conducted by an organization in California, involving over 40,000 households, which provides some of the most precise evidence to date regarding the effect of injunctive norms on decision making. I find that not only do injunctive norms encourage conservation behavior, there is also no evidence that they discourage individuals from further attending norms messaging-regardless of whether the social judgement conveyed is negative or positive. Taken together, this suggests that injunctive norms are a useful tool in "nudge"-style campaigns tackling behavior change.

Amee Kamdar, Steven D Levitt, John A List, Brian Mullaney, Chad Syverson

Cited by*: None Downloads*: None

In this paper, we present the results of a two-year series of large-scale natural field experiments involving hundreds of thousands of subjects.

John A List

Cited by*: None Downloads*: None

Last year I put together a summary of data from my field experiments website that pertained to artefactual field experiments. Several people have asked me if I have and update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper.

Erwin Bulte, John A List, Daan van Soest

Cited by*: None Downloads*: None

Social scientists have recently explored how framing of gains and losses affects productivity. We conducted a field experiment in peri-urban Uganda, and compared output levels across 1000 workers over isomorphic tasks and incentives, framed as either losses or gains. We find that loss aversion can be leveraged to increase the productivity of labor. The estimated welfare costs of using the loss contract are quite modest -- perhaps because the loss contract is viewed as a (soft) commitment device.

Syon Bhanot, Gordon Kraft-Todd, David Rand, Erez Yoeli

Cited by*: None Downloads*: None

We partnered with the School District of Philadelphia (SDP) to run a randomized experiment testing interventions to increase teacher participation in an annual feedback survey, an uncompensated task that requires a teacher's time but helps the educational system overall. Our experiment varied the nature of the incentive scheme used, and the associated messaging. In the experiment, all 8,062 active teachers in the SDP were randomly assigned to receive one of four emails using a 2x2 experimental design; specifically, teachers received a lottery-based financial incentive to complete the survey that was either "personal" (a chance to win one of fifteen $100 gift cards for themselves) or "social" (a chance to win one of fifteen $100 gift cards for supplies for their students), and also received email messaging that either did or did not make salient their identity as an educator. Despite abundant statistical power, we find no discernible differences across our conditions on survey completion rates. One implication of these null results is that from a public administration perspective, social rewards may be preferable since funds used for this purpose by school districts go directly to students (through increased expenditure on student supplies), and do not seem less efficacious than personal financial incentives for teachers.

Damon Clark, David Gill, Victoria Prowse , Mark Rush

Cited by*: None Downloads*: None

Will college students who set goals for themselves work harder and achieve better outcomes? In theory, setting goals can help present-biased students to mitigate their self-control problem. In practice, there is little credible evidence on the causal effects on goal setting for college students. We report the result of two field experiments that involved almost four thousand college students in total. One experiments asked treated students to set goals for performance in the course; the other asked treated students to set goals for a particular task (completing online practice exams). Task-based goals had robust positive effects on the level of task completion, and task-based goals also increased course performance. We also find that performance-based goals had positive but small effects on course performance. We use a theoretical framework that builds on present bias and loss aversion to interpret our results. Since task-based goal setting is low-cost, scalable and logistically simple, we conclude that our findings have important implications for educational practice and future research.

John A List

Cited by*: None Downloads*: None

Last year I put together a summary of data from my field experiments website that pertained to framed field experiments. Several people have asked if I have an update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper below.

Wojciech Hardy, Michal Krawczyk, Joanna Tyrowicz

Cited by*: None Downloads*: None

We report results of an experimental study analyzing the effects of Internet piracy on book sales. We conducted a year-long controlled large-scale field experiment with pre-treatment pair matching. Half of the book titles received experiment treatment, in which a specialized agency would immediately remove any unauthorized copy appearing on the Internet. For the other half we merely registered such occurrences, but no countermeasures were taken. For all the titles we obtained print and e-book sales statistics from the publishers. We find that removal of unauthorized copies was an effective method of curbing piracy, but this had no bearing on legal sales.

John A List, Jeffrey A Livingston, Susanne Neckermann

Cited by*: None Downloads*: None

In the face of worryingly low performance on standardized test, offering students financial incentives linked to academic performance has been proposed as a potentially cost-effective way to support improvement. However, a large literature across disciplines finds that extrinsic incentives, once removed, may crowd out intrinsic motivation on subsequent, similar tasks. We conduct a field experiment where students, parents, and tutors are offered incentives designed to encourage student preparation for a high-stakes state test. The incentives reward performance on a separate low-stakes assessment designed to measure the same skills as the high-stakes test. Performance on the high-stakes test, however, is not incentivized. We find substantial treatment effects on the incented tests but no effect on the non-incented test; if anything, the incentives result in worse performance on the non-incented test. We also find evidence supporting the conclusion that the incentives crowd out intrinsic motivation to perform well on the non-incented test, but this effect is only temporary. One year later, students who had been in the incentives treatments perform better than those in the control on the same non-incented test.

Omar Al-Ubaydli, John A List, Dana L Suskind

Cited by*: None Downloads*: None

Policymakers are increasingly turning to insights gained from the experimental method as a means of informing public policies. Whether-and to what extent-insights from a research study scale to the level of the broader public is, in many situations, based on blind faith. This scale-up problem can lead to a vast waste of resources, a missed opportunity to improve people's lives, and a diminution in the public's trust in the scientific method's ability to contribute to policymaking. This study provides a theoretical lens to deepen our understanding of the science of how to use science. Through a simple model, we highlight three elements of the scale-up problem: (1) when does evidence become actionable (appropriate statistical inference); (2) properties of the population; and (3) properties of the situation. We argue that until these three areas are fully understood and recognized by researchers and policymakers, the threats to scalability will render any scaling exercise as particularly vulnerable. In this way, our work represents a challenge to empiricists to estimate the nature and extent of how important the various threats to scalability are in practice, and to implement those in their original research.

Omar Al-Ubaydli, John A List, Claire Mackevicius, Min Sok Lee, Dana L Suskind

Cited by*: None Downloads*: None

Policymakers are increasingly turning to insights gained from the experimental method as a means to inform large scale public policies. Critics view this increased usage as premature, pointing to the fact that many experimentally-tested programs fail to deliver their promise at scale. Under this view, the experimental approach drives too much public policy. Yet, if policymakers could be more confident that the original research findings would be delivered at scale, even the staunchest critics would carve out a larger role for experiments to inform policy. Leveraging the economic framework of Al-Ubaydli et al. (2019), we put forward 12 simple proposals, spanning researchers, policymakers, funders, and stakeholders, which together tackle the most vexing scalability threats. The framework highlights that only after we deepen our understanding of the scale up problem will we be on solid ground to argue that scientific experiments should hold a more prominent place in the policymaker's quiver.

Chien-Yu Lai, John A List, Anya Samek

Cited by*: None Downloads*: None

The National School Lunch Program (NSLP) is a federal food assistance program that serves over 30 million children in the United States annually. Yet the impact of NSLP on nutritional intake may be limited because children frequently do not choose the healthier offerings or waste large portions of their meal. In this article, we study whether we can improve the impact of the NSLP on child food choice through low-cost nudges. We conduct a field experiment in a school lunchroom with 2500 children, evaluating the impact of informational prompts on milk choice and consumption over two weeks. We find that the prompts alone increase the proportion of children choosing and consuming the healthier white milk relative to sugar-sweetened chocolate milk from 20% in the control group to 30% in the treatment groups. Adding health or taste messaging to the prompt does not seem to make a difference. We survey students and find that most prompts affect perceived healthfulness of the milk, but not perceived taste. Finally, we find that the prompts are nearly as effective as a small nonmonetary incentive.

Rudolf Kerschbamer, Daniel Neururer, Matthias Sutter

Cited by*: None Downloads*: None

Honesty is a fundamental pillar for cooperation in human societies and thus for their economic welfare. However, humans do not always act in an honest way. Here, we examine how insurance coverage affects the degree of honesty in credence good markets. Such markets are plagued by strong incentives for fraudulent behavior of sellers, resulting in estimated annual costs of billions of dollars to costumers and the society as a whole. Prime examples of credence goods are all kinds of repair services, the provision of medical treatments, the sale of software programs, and the provision of taxi rides in unfamiliar cities. We examine in a natural field experiment how computer repair shops take advantage of costumers' insurance for repair costs. In a control treatment, the average repair price is about EUR 70, whereas the repair bill increases by more than 80% when the service provider is informed that an insurance would reimburse the bill. Our design allows decomposing the sources of this economically impressive difference, showing that it is mainly due to the overprovision of parts and overcharging of working time. A survey among repair shops shows that the higher bills are mainly ascribed to insured costumers being less likely to be concerned about minimizing costs because a third party (the insurer) pays the bill. Overall, our results strongly suggest that insurance coverage greatly increases the extent of dishonesty in important sectors of the economy with potentially huge costs to costumers and whole economies.

Syon Bhanot, Jiyoung Han, Chaning Jang

Cited by*: None Downloads*: None

Restrictions like work requirements and constraints on voucher transfers are often used in social welfare systems, but little empirical evidence exists on their impact on wellbeing. We conducted a 10-day randomized experiment with 432 individuals living below the poverty line in the Kawangware settlement of Nairobi, kenya, testing two elements of social welfare design: workfare versus welfare and restricted versus unrestricted vouchers. Participants were randomly assigned to a "Work" condition, involving daily work for unrestricted vouchers, or one of two "Wait" conditions, involving daily waiting for vouchers that were either unrestricted or partially restricted to staple foods. We find that working improved psychological wellbeing relative to waiting, suggesting that means of implementing welfare programs may have important effects on individuals beyond the impact of monetary benefit alone. Furthermore, although restrictions were inframarginal, partially restricted vouchers crowded-in spending on staple foods, suggesting the existence of a "flypaper effect" in spending from restricted vouchers.

Christopher Cotton, Brent R Hickman, John A List, Joseph Price, Sutanuka Roy

Cited by*: None Downloads*: None

We conduct a field experiment across three diverse school districts to structurally identify student motivation and study productivity parameters in a model of adolescent human capital development. By observing study time, homework task completion, and test results, we can identify individual and demographic variations in motivation and study time effectiveness. Struggling students typically do not lack motivation but rather struggle to convert study time into completed assignments and proficiency improvements. The study also attending a higher-performing school is associated with both higher productivity and higher motivation relative to peers with similar observables in lower-performing schools. Counterfactual analyses estimates that school quality differences account for a substantial share of the racial differences in test scores, and considers the impact of alternative policies aimed at reducing racial performance gaps.