Papers - Natural

Omar Al-Ubaydli, John A List, Claire Mackevicius, Min Sok Lee, Dana L Suskind

Cited by*: None Downloads*: None

Policymakers are increasingly turning to insights gained from the experimental method as a means to inform large scale public policies. Critics view this increased usage as premature, pointing to the fact that many experimentally-tested programs fail to deliver their promise at scale. Under this view, the experimental approach drives too much public policy. Yet, if policymakers could be more confident that the original research findings would be delivered at scale, even the staunchest critics would carve out a larger role for experiments to inform policy. Leveraging the economic framework of Al-Ubaydli et al. (2019), we put forward 12 simple proposals, spanning researchers, policymakers, funders, and stakeholders, which together tackle the most vexing scalability threats. The framework highlights that only after we deepen our understanding of the scale up problem will we be on solid ground to argue that scientific experiments should hold a more prominent place in the policymaker's quiver.

Maria De Paola, Francesca Gioia, Vincenzo Scoppa

Cited by*: None Downloads*: None

We ran a field experiment to investigate whether competing in rank-order tournaments with different prize spreads affects individual performance. Our experiment involved students from an Italian University who took an exam that was partly evaluated on the basis of relative performance. Students were matched in pairs on the basis of their high school grades and each pair was randomly assigned to one of three different tournaments. Random assignment neutralizes selection effects and allows us to investigate if larger prize spreads increase individual effort. We do not find any positive effect of larger prizes on performance. Furthermore, we show that the effect of prize spreads on students' performance depends on their degree of risk-aversion: competing in tournaments with large spreads negatively affects the performance of risk-averse students, while it does not produce any effect on students who are more prone to taking risks.

Eszter Czibor, David Jimenez-Gomez, John A List

Cited by*: None Downloads*: None

What was once broadly viewed as an impossibility - learning from experimental data in economics - has now become commonplace. Governmental bodies, think tanks, and corporations around the world employ teams of experimental researchers to answer their most pressing questions. For their part, in the past two decades academics have begun to more actively partner with organizations to generate data via field experimentation. While this revolution in evidence-based approaches has served to deepen the economic science, recently a credibility crisis has caused even the most ardent experimental proponents to pause. This study takes a step back from the burgeoning experimental literature and introduces 12 actions that might help to alleviate this credibility crisis and raise experimental economics to an even higher level. In this way, we view our "12 action wish list" as discussion points to enrich the field.

Andrew Dustan, Juan Manuel Hernandez-Agramonte, Stanislao Maldonado

Cited by*: None Downloads*: None

We study how non-monetary incentives, motivated by recent advances in behavioral economics, affect civil servant performance in a context where state capacity is weak. We collaborated with a government agency in Peru to experimentally vary the context of text messages targeted to civil servants in charge of a school maintenance program. These messages incorporated behavioral insights in dimensions related to information provision, social norms, and weak forms of monitoring and auditing. We find that these messages are a very cost-effective strategy to enforce compliance with national policies among civil servants. We further study the role of social norms and the salience of social benefits in a follow-up experiment and explore the external validity or our original results by implementing a related experiment with civil servants from a different national program. The findings of these new experiments support our original results and provide additional insights regarding the context in which these incentives may work. Our results highlight the importance of carefully designed non-monetary incentives as a tool to improve civil servant performance when the state lacks institutional mechanisms to enforce compliance.

Maya Haran Rosen , Orly Sade

Cited by*: None Downloads*: None

Define contribution mechanism combined with a dynamic job market can affect the sum of retirement savings and the choices of plans and products. Hence, it is important for regulators to engage servers to manage the accounts they accumulate over the years. In 2013-2014 the Israeli regulator reached out to the population, recommending the use of a website to help individuals find inactive retirement savings accounts and close them (withdraw the savings or transfer them to active accounts). The government's efforts did not result in the closure of most of the inactive accounts. Proprietary data indicate that those who closed the inactive accounts live in central locations with a higher socioeconomic index. Survey data indicate that those who lacked financial literacy and confidence in their financial knowledge were less likely to take financial actions. Using a controlled field experiment, we also provide evidence that an intervention with a human touch can promote greater involvement.

Alec Brandon, John A List, Robert D Metcalfe, Michael K Price, Florian Rundhammer

Cited by*: None Downloads*: None

This study considers the response of household electricity consumption to social nudges during peak load events. Our investigation considers two social nudges. The first targets conservation during peak load events, while the second promotes aggregate conservation. Using data from a natural field experiment with 42,100 households, we find that both social nudges reduce peak load electricity consumption by 2 to 4% when implemented in isolation and by nearly 7% when implemented in combination. These findings suggest an important role for social nudges in the regulation of electricity markets and a limited role for crowd out effects.

Syon Bhanot

Cited by*: None Downloads*: None

Across domains, people struggle to follow through on their commitments. This can happen for many reasons, including dishonesty, forgetfulness, or insufficient intrinsic motivation. Social scientists have explored the reasons for persistent failures to follow through. suggesting that eliciting explicit promises can be an effective way to motivate action. This paper presents a field experiment that tests the effect of explicit promises, in the form of "honor pledges", on loan repayment rates. The experiment was conducted with LendUp, an online lender, and targeted 4,883 first time-borrowers with the firm. Individuals were randomized into four groups, with the following experimental treatments: (1) having no honor pledge to complete (control); (2) signing a given honor pledge; (3) re-typing the same honor pledge as in (2) before signing; and (4) coming up with a personal honor pledge to type and sign. I also randomized whether or not borrowers were reminded of the honor pledge they signed prior to repayment deadline. The results suggest that the honor pledge treatments had minimal impacts on repayment, and that reminders of the pledges were similarly ineffective. This suggests that borrowers who fail to repay loans do so not because of dishonesty or behavioral biases, but because they suffer from true financial hardship and are simply unable to repay.

Loukas Balafoutas, Nikos Nikiforakis, Bettina Rockenbach

Cited by*: 0 Downloads*: None

The degree of human cooperation among strangers is a major evolutionary puzzle. A prominent explanation is that cooperation maintained because many individuals have a predisposition to punish those violating group-beneficial norms. A critical condition for cooperation to evolve in evolutionary models is that punishment increases with the severity of the violation. Here we present evidence from a field experiment with real-life interactions that, unlike in lab experiments, altruistic punishment does not increase with the severity of the violation, regardless of whether it is direct (confronting a violator) or indirect (withholding help). We also document growing concerns for counter-punishment as the severity of the violation increases, indicating that the marginal cost of direct punishment increases with the severity of violations. The evidence suggests that altruistic punishment may not provide appropriate incentives to deter large violations. Our findings thus offer a rationale for the emergence of formal institutions for prompting large-scale cooperation among strangers.

Syon Bhanot

Cited by*: None Downloads*: None

Social norms messaging campaigns are increasingly used to influence human behavior, with social science research generally finding that they have modest but meaningful effects. One aspect of these campaigns in practice has been the inclusion of injunctive norms messaging, designed to convey a social judgement about one's behaviors (often in the form of encouraging or discouraging language, or a visual smiley or frowny face). While some prominent research has provided support for the use of such messaging as a tool for positive behavior change, causal evidence on the effect of injunctive norms messaging as a motivator (as opposed to just one part of a multifaceted messaging campaign) is limited. This paper presents a field experiment on water conservation behavior conducted by an organization in California, involving over 40,000 households, which provides some of the most precise evidence to date regarding the effect of injunctive norms on decision making. I find that not only do injunctive norms encourage conservation behavior, there is also no evidence that they discourage individuals from further attending norms messaging-regardless of whether the social judgement conveyed is negative or positive. Taken together, this suggests that injunctive norms are a useful tool in "nudge"-style campaigns tackling behavior change.

Syon Bhanot, Gordon Kraft-Todd, David Rand, Erez Yoeli

Cited by*: None Downloads*: None

We partnered with the School District of Philadelphia (SDP) to run a randomized experiment testing interventions to increase teacher participation in an annual feedback survey, an uncompensated task that requires a teacher's time but helps the educational system overall. Our experiment varied the nature of the incentive scheme used, and the associated messaging. In the experiment, all 8,062 active teachers in the SDP were randomly assigned to receive one of four emails using a 2x2 experimental design; specifically, teachers received a lottery-based financial incentive to complete the survey that was either "personal" (a chance to win one of fifteen $100 gift cards for themselves) or "social" (a chance to win one of fifteen $100 gift cards for supplies for their students), and also received email messaging that either did or did not make salient their identity as an educator. Despite abundant statistical power, we find no discernible differences across our conditions on survey completion rates. One implication of these null results is that from a public administration perspective, social rewards may be preferable since funds used for this purpose by school districts go directly to students (through increased expenditure on student supplies), and do not seem less efficacious than personal financial incentives for teachers.

Eric Floyd, Michael Hallsworth, John A List, Robert D Metcalfe, Kristian Rotaru, Ivo Vlaev

Cited by*: None Downloads*: None

In this study, we first present a large natural field experiment that tested messages aimed at increasing tax compliance. We find that the main drivers of changes in compliance are messages describing the monitoring and enforcement behavior of the tax collector. A second natural field experiment built on the results of the first experiment to further investigate what kinds of costs resulting from tax collector oversight are salient to taxpayers. Specific time and cognitive incentives did not significantly increase payment rates, whereas stating non-specific costs of inaction did. Additional analyses suggest the increase in compliance is likely due to a 'fill in the blank' effect in which taxpayers assume the consequence is a fine. Interestingly, specifically stating maximum fine or jailtime consequences have the largest effect in a laboratory setting but only if the consequences are interpreted as realistic. Overall, our study reinforces that tax authorities can use short messages to increase tax compliance; the estimated accelerated revenue from the two field studies amounts to 9.9m GBP.

John A List

Cited by*: None Downloads*: None

No abstract available

Pradhi Aggarwal, Alec Brandon, Ariel Goldszmidt, Justin Holz, John A List, Ian Muir, Gregory Sun, Thomas Yu

Cited by*: None Downloads*: None

Prior research finds that, conditional on an encounter, minority civilians are more likely to be punished by police than white civilians. An open question is whether the actual encounter is related to race. Using high-frequency location data of rideshare drivers operating on the Lyft platform in Florida, we estimate the effect of driver race on traffic stops and fines for speeding. Estimates obtained across traditional and machine learning approaches show that, relative to a white driver traveling the same speed, minorities are 24 to 33 percent more likely to be stopped for speeding and pay 23 to 34 percent more in fines. We find no evidence that these estimates can be explained by racial differences in accident and re-offense rates. Our study provides key insights into the total effect of civilian race on outcomes of interest and highlights the potential value of private sector data to help inform major social challenges.

Omar Isaac Asensio, Magali A Delmas

Cited by*: 3 Downloads*: None

Little is known about the effect of message framing on conservation behavior over time. In a randomized controlled trial with residential households, we use advanced metering and information technologies to test how different messages about household energy use impact the dynamics of conservation behavior down to the appliance level. Our results, based on 374 million panel observations of kilowatt-hour (kWh) electricity consumption for 118 households over 9 months, show that differences in behavioral responses due to message framing become more significant over time. We find that a health-based frame, in which households consider the human health effects of their marginal electricity use, induced persistent energy savings behavior of 8-10% over 100 days; whereas a more traditional cost savings frame, drove sharp attenuation of treatment effects after 2 weeks with no significant savings versus control after 7 weeks. We discuss the implications for the design of effective information campaigns to engage households in conservation behavior.

John A List

Cited by*: None Downloads*: None

While empirical economics has made important strides over the past half century, there is a recent attack that threatens the foundations of the empirical approach in economics: external validity. Certain dogmatic arguments are not new, yet in some circles the generalizability question is beyond dispute, rendering empirical work as a passive enterprise based on frivolity. Such arguments serve to caution even the staunchest empirical advocates from even starting an empirical inquiry in a novel setting. In its simplest form, questions of external validity revolve around whether the results of the received study can be generalized to different people, situations, stimuli, and time periods. This study clarifies and places the external validity crisis into perspective by taking a unique glimpse into the grandest of trials: The External Validity Trial. A key outcome of the proceedings is an Author Onus Probandi, which outlines four key areas that every study should report to address external validity. Such an evaluative approach properly rewards empirical advances and justly recognizes inherent empirical limitations.

Chien-Yu Lai, John A List, Anya Samek

Cited by*: None Downloads*: None

The National School Lunch Program (NSLP) is a federal food assistance program that serves over 30 million children in the United States annually. Yet the impact of NSLP on nutritional intake may be limited because children frequently do not choose the healthier offerings or waste large portions of their meal. In this article, we study whether we can improve the impact of the NSLP on child food choice through low-cost nudges. We conduct a field experiment in a school lunchroom with 2500 children, evaluating the impact of informational prompts on milk choice and consumption over two weeks. We find that the prompts alone increase the proportion of children choosing and consuming the healthier white milk relative to sugar-sweetened chocolate milk from 20% in the control group to 30% in the treatment groups. Adding health or taste messaging to the prompt does not seem to make a difference. We survey students and find that most prompts affect perceived healthfulness of the milk, but not perceived taste. Finally, we find that the prompts are nearly as effective as a small nonmonetary incentive.

Amanda Bayer, Syon Bhanot, Fernando Lozano

Cited by*: None Downloads*: None

No abstract available

Omar Al-Ubaydli, John A List, Dana L Suskind

Cited by*: None Downloads*: None

Policymakers are increasingly turning to insights gained from the experimental method as a means of informing public policies. Whether-and to what extent-insights from a research study scale to the level of the broader public is, in many situations, based on blind faith. This scale-up problem can lead to a vast waste of resources, a missed opportunity to improve people's lives, and a diminution in the public's trust in the scientific method's ability to contribute to policymaking. This study provides a theoretical lens to deepen our understanding of the science of how to use science. Through a simple model, we highlight three elements of the scale-up problem: (1) when does evidence become actionable (appropriate statistical inference); (2) properties of the population; and (3) properties of the situation. We argue that until these three areas are fully understood and recognized by researchers and policymakers, the threats to scalability will render any scaling exercise as particularly vulnerable. In this way, our work represents a challenge to empiricists to estimate the nature and extent of how important the various threats to scalability are in practice, and to implement those in their original research.

Syon Bhanot

Cited by*: None Downloads*: None

Perception of peer rank, or how we can perform relative to out peers, can be a powerful motivator. While research exists on the effect of social information on decision making, there is less work on how ranked comparisons with our peers influence our behavior. This paper outlines a field experiment conducted with 3896 households in Castro Valley, California, which uses household mailers with various forms of social information and peer rank messaging to motivate water conservation. The experiment tests the effect of a visible peer rank on water use, and how the competitive framing of rank information influences behavioral response. The results show that households with relatively low or high water use in the pre-treatment period responded differently to how rank information was framed. I find that a neutrally-framed peer rank caused a small "boomerang effect" (i.e., an increase in average water use) for low water households, but this effect was eliminated by competitive framing. At the same time, a competitively-framed peer rank demotivated high water use households, increasing their average water use over the full period of the experiment. This result is supported by evidence that the competitive frame on rank information increased water use for households who ranked "last" in the peer group - a detrimental "last place effect" from competitively-framed rankings.

John A. List, Ioannis C. Pragidis, Michael K Price

Cited by*: None Downloads*: None

Prosumers are becoming increasingly important in global energy consumption and production. We partner with an energy service provider in Sweden to explore the economics facing such agents by conducting a natural field experiment over a 32-month period. As a policy instrument, we explore how simple nudges affect choices on both the consumption and production sides. Importantly, with the added flexibility to influence both sides of the market, and with a rich data set that permits an analysis of intraday, intraweek, and seasonal variation, we can detail effects on overall conservation efforts, intertemporal substitution, load shifting, and net purchases from the grid. The overarching theme is that nudges have the potential to have an even greater impact on the energy market with prosumers compared to their portmanteau components.