Papers - Recent

Stefano Carattini, Robert Dur, John A. List

Cited by*: None Downloads*: None

Many policies that are generally considered socially desirable by the scientific community, based on modeling and causal empirical analyses, are not very widespread. The main driver is often lack of public support at baseline ("ex ante"). Yet, there is evidence that when voters hold biased beliefs ex ante about a given policy, experiencing the policy firsthand may lead them to correct their beliefs and increase public support. If it was widely documented that opposition to sound policies in part dissipates when voters experience a given policy, then more policy-makers may be inclined to experiment with policies that scientists recommend but that are unpopular ex ante. Systematically combining policy evaluation with causal analysis of public support would allow scholars to create a body of knowledge on the conditions under which policies become more (or less) popular after implementation and what are the drivers of changes in beliefs and public support.

John A. List, Haruka Uchida

Cited by*: None Downloads*: None

An unsettling stylized fact is that decorated early childhood education programs improve cognitive skills in the short-term, but lose their efficacy after a few years. We implement a field experiment with two stages of randomization to explore the underpinnings of the fade-out effect. We first randomly assign preschool access to children, and then partner with the local school district to randomly assign the same children to classmates throughout elementary school. We find that the fade-out effect is critically linked to the share of classroom peers assigned to preschool access-with enough treated peers the classic fade-out effect is muted. Our results highlight a paradoxical insight: while the fade-out effect has been viewed as a devastating critique of early childhood programs, our results highlight that fade-out is a key rational for providing early education to all children. This is because human capital accumulation is inherently a social activity, leading early education programs to deliver their largest benefits at scale when everyone receives such programs.

Samuel Chang, Andrew Kennedy, Aaron Leonard, John A. List

Cited by*: None Downloads*: None

We provide twelve best practices and discuss how each practice can help researchers accurately, credibly, and ethically use Generative AI (GenAI) to enhance experimental research. We split the twelve practices into four areas. First, in the pre-treatment stage, we discuss how GenAI can aid in pre-registration procedures, data privacy concerns, and ethical considerations specific to GenAI usage. Second, in the design and implementation stage, we focus on GenAI's role in identifying new channels of variation, piloting and documentation, and upholding the four exclusion restrictions. Third, in the analysis stage, we explore how prompting and training set bias can impact results as well as necessary steps to ensure replicability. Finally, we discuss forward-looking best practices that are likely to gain importance as GenAI evolves.

Omar Al-Ubaydli, John A. List

Cited by*: None Downloads*: None

The A/B testing approach invites promising early results that are unlikely to be realized in a larger setting. We argue that within the social sciences, a fundamentally different approach is needed; we call it option C thinking. Put simply, a twenty-first-century team of civil servants and social scientists should lead with experiments that anticipate likely causes of failure at scale, even if doing so requires more time, effort, and resources initially.

Stefano Carattini, Julia Blasch

Cited by*: None Downloads*: None

Social interventions are a popular tool to stimulate pro-social (including climate-friendly) behavior. Their use is, however, limited when the descriptive norm is low, i.e. when a desirable behavior is only practiced by a minority within the respective reference group. We tackle this issue by testing new strategies for social interventions, with an especially sophisticated target group. We implement a field experiment at two subsequent conferences in environmental economics, with which we examine the conference participants' proclivity to offset carbon emissions. For the two treatment conditions that we introduce, we document an average null effect. Yet, for one condition, we find that the intervention can be effective when the targeted individuals feel socially close to the referenced peer group. Further, we find suggestive evidence that the effectiveness of such interventions increases as individuals are exposed to repeated treatment, but with decreasing marginal returns.

Stefano Carattini, Kenneth Gillingham, Xiangyu Meng, Erez Yoeli

Cited by*: None Downloads*: None

Observability has been demonstrated to influence the adoption of pro-social behavior in a variety of contexts. This study implements a natural field experiment to examine the influence of observability in the context of a novel pro-social behavior: peer-to-peer solar. Peer-to-peer solar offers an opportunity to households who cannot have solar on their homes to access solar energy from their neighbors. However, unlike solar installations, peer-to-peer solar is an invisible form of pro-environmental behavior. We implemented a set of randomized campaigns using Facebook ads in the Massachusetts cities of Cambridge and Somerville, in partnership with a peer-to-peer company, to study social media users' interest in peer-to-peer solar through clicks on the ads. In the campaigns, treated customers were informed that they could share "green reports" online, providing information to others about their greenness. We find that interest in peer-to-peer solar increases by up to 30% when "green reports," which would make otherwise invisible behavior visible, are mentioned in the ads.

John A. List

Cited by*: None Downloads*: None

This document is meant to introduce my forthcoming book, titled "Experimental Economics: Theory and Practice," which is to be published in 2025 by The University of Chicago Press. The document first contains the book's outline followed by a Preface, which summarizes my inspiration for writing the book and my goals and aspirations for choosing the content contained in the book.

John A. List, Ioannis C. Pragidis, Michael K Price

Cited by*: None Downloads*: None

Prosumers are becoming increasingly important in global energy consumption and production. We partner with an energy service provider in Sweden to explore the economics facing such agents by conducting a natural field experiment over a 32-month period. As a policy instrument, we explore how simple nudges affect choices on both the consumption and production sides. Importantly, with the added flexibility to influence both sides of the market, and with a rich data set that permits an analysis of intraday, intraweek, and seasonal variation, we can detail effects on overall conservation efforts, intertemporal substitution, load shifting, and net purchases from the grid. The overarching theme is that nudges have the potential to have an even greater impact on the energy market with prosumers compared to their portmanteau components.

Katherine L. Milkman, Sean F. Ellis, Dena M. Gromet, Youngwoo Jung, Alex S. Luscher, Rayyan S. Mobarak, Madeline K. Paxson, Ramon A. Silvera Zumaran, Robert Kuan, Ron Berman, Neil A. Lewis Jr, John A. List, Mitesh S. Patel, Christophe Van den Bulte, Kevin G. Volpp, Maryann V. Beauvais, Jonathon K. Bellows, Cheryl A. Marandola, Angela L. Duckworth

Cited by*: None Downloads*: None

Encouraging routine COVID-19 vaccinations will probably be a crucial policy challenge for decades to come. To avert hundreds of thousands of unnecessary hospitalizations and deaths, adoption will need to be higher than it was in the autumn of 2022 or 2023, when less than one-fifth of Americans received booster vaccines. One approach to encourage vaccination is to eliminate the friction of transportation hurdles. Previous research has shown that friction can hinder follow-through and that individuals who live farther from COVID-19 vaccination sites are less likely to get vaccinated. However, the value of providing free round-trip transportation to vaccination sites is unknown. Here we show that offering people free round-trip Lyft rides to pharmacies has no benefit over and above sending them behaviourally informed text messages reminding them to get vaccinated. We determined this by running a megastudy with millions of CVS Pharmacy customers in the United States testing two effects: free round-trip Lyft rides to CVS Pharmacies for vaccination appointments, and seven different sets of behaviourally informed vaccine reminder messages. Our results suggest that offering previously vaccinated individuals free rides to vaccination sites is not a good investment in the United States, which is contrary to the high expectations of both expert and lay forecasters. Instead, people in the United States should be sent behaviourally informed COVID-19 vaccination reminders, which increased the 30-day COVID-19 booster uptake by 21% (1.05 percentage points) and spilled over to increase 30-day influenza vaccinations by 8% (0.34 percentage points) in our megastudy. More rigorous testing of interventions to promote vaccination is needed to ensure that evidence-based solutions are deployed widely and that ineffective but intuitively appealing tools are discontinued.

Erwin Bulte, Andreas Kontoleon, John A List, Ty Turley, Maarten Voors

Cited by*: None Downloads*: None

We use a field experiment in Sierra Leone to examine how the identity of the manager influences rent seeking and performance in participatory development projects. Specifically, we vary the composition of a committee responsible for implementing a development project-local elites or randomly selected villagers. The design is unique in that it permits us to explore the effectiveness of two alternative local governance modalities and the extent of elite capture in community projects. We find little evidence that local elites capture project resources. We do observe they are better managers of development projects. Improved performance covaries with a proxy for power of the local chief.

John A List

Cited by*: None Downloads*: None

Once believed to be an impossibility, field experiments in economics now occupy a central place in the empiricist's quiver. In the past few decades alone field experiments have taken on much greater import in academe, across organizations, as well as for policymakers. But is this emergence simply a fad that will soon return field experiments to obscurity? I argue in this article that there is something fundamental about the emergence of field experiments, as controlling the assignment mechanism in the field provides unparalleled power to both understand the "effects of causes" and the "causes of effects." This knowledge generation then begins to uncover the generalizability and scalability of knowledge. Quite the opposite of a withering tool that will be gone tomorrow, I urge economists to "double down" on this comparative advantage and in doing so I provide four methodological paths which I hope will cement the promise and growth of field experiments in the social sciences.

Omar Al-Ubaydli, Faith Fatchen, John A List

Cited by*: None Downloads*: None

Field experiments are a useful empirical tool that can be deployed in any sub-discipline - including institutional economics - to enhance the sub-discipline's empirical insights. However, we here argue that there exist fundamental barriers to the use of field experiments in understanding the impact of institutions on economic growth. Despite these obstacles, we present some significant scholarly contributions that merit exposition, while also proposing some future methods for using field experiments within institutional economics. While field experiments may be limited in answering questions in institutional economics with macroeconomic outcomes, there is great potential in employing field experiments to answer micro founded questions.

John A List, Lina Ramirez, Julia Seither, Jaime Unda, Beatriz Vallejo

Cited by*: None Downloads*: None

Misinformation represents a vital threat to the societal fabric of modern economies. While the supply side of the misinformation market has begun to receive increased scrutiny, the demand side has received scant attention. We explore the demand for misinformation through the lens of augmenting critical thinking skills in a field experiment during the 2022 Presidential election in Colombia. Data from roughly 2.000 individual suggest that our treatments enhance critical thinking, causing subjects to more carefully consider the truthfulness of potential misinformation. We furthermore provide evidence that reducing the demand of fake news can deliver on the dual goal of reducing the spread of fake news by encouraging reporting of misinformation.

John A List

Cited by*: None Downloads*: None

In 2019, I put together a summary of data from my field experiments website that pertained to natural field experiments (Harrison and List, 2024). Several people have asked me for updates. In this document I update all figures and numbers to show the details for 2023. I also include the description from the original paper below.

John A List

Cited by*: None Downloads*: None

In 2019 I put together a summary of data from my field experiments website that pertained to framed field experiments (see List 2024). Several people have asked me if I have an update. In this document I update all figures and numbers to show the details for 2023. I also include the description from the 2019 paper below.

John A List

Cited by*: None Downloads*: None

Social scientists have increasingly turned to the experimental method to understand human behavior. One critical issue that makes solving social problems difficult is "scaling" the idea from a small group to a larger group in more diverse situations. The urgency of scaling policies impacts us every day, whether it is protecting the health and safety of a community or enhancing the opportunities of future generations. Yet, a common result is that when we scale ideas most experience a "voltage drop": upon scaling, the benefit-cost profile depreciates considerably. To combat voltage drops, we must optimally generate policy-based evidence. Optimality requires answering two crucial questions: what information to generate and in what sequence. The economics underlying the science of scaling provides insights into these questions, which are in some cases at odds with conventional approaches. For example, there are important situations wherein I advocate flipping the traditional social science research model to an approach that, from the beginning, produces the type of policy-based evidence that the science of scaling demands. To do so, I propose augmenting efficacy trials by including relevant tests of scale in the original discovery process, which forces the scientist to naturally start with a recognition of the big picture: what information do I need to haves caling confidence?

John A List

Cited by*: None Downloads*: None

In 2019, I put together a summary of data from my field experiments website that pertained to artefactual field experiments. Several people have asked me if I have an update. In this document I update all figures and numbers to show the details for the year 2023. I also include the description from the 2019 paper below. The definition of artefactual field experiments comes originally from Harrison and List (2004) and is advanced in List (2024).

Amanda Agan, Bo Cowgill, Laura Gee

Cited by*: None Downloads*: None

Correspondence audit studies have sent almost one-hundred-thousand resumes without informing subjects they are in a study - increasing realism, but without being fully transparent. We study the potential trade-offs of this lack of transparency by running a hiring field experiment with recruiters in a natural setting. One group of recruiters is told they are screening for an employer, and another is told they are part of an academic study. Job applicants' gender is randomly assigned. When subjects are told they are in an experiment, callback rates and willingness-to-pay for male candidates decline relative to female candidates (with no detectable change for female candidates). This suggests that telling subjects they are in an experiment would underestimate gender inequality.

John A List

Cited by*: None Downloads*: None

ASSA 2023 presentation

John A List

Cited by*: None Downloads*: None

"Putting Economic Research into Practice at Businesses (to Inform Science and Major Social Challenges)" Slides from ASSA NABE