Papers - All

Syon Bhanot, Gordon Kraft-Todd, David Rand, Erez Yoeli

Cited by*: None Downloads*: None

We partnered with the School District of Philadelphia (SDP) to run a randomized experiment testing interventions to increase teacher participation in an annual feedback survey, an uncompensated task that requires a teacher's time but helps the educational system overall. Our experiment varied the nature of the incentive scheme used, and the associated messaging. In the experiment, all 8,062 active teachers in the SDP were randomly assigned to receive one of four emails using a 2x2 experimental design; specifically, teachers received a lottery-based financial incentive to complete the survey that was either "personal" (a chance to win one of fifteen $100 gift cards for themselves) or "social" (a chance to win one of fifteen $100 gift cards for supplies for their students), and also received email messaging that either did or did not make salient their identity as an educator. Despite abundant statistical power, we find no discernible differences across our conditions on survey completion rates. One implication of these null results is that from a public administration perspective, social rewards may be preferable since funds used for this purpose by school districts go directly to students (through increased expenditure on student supplies), and do not seem less efficacious than personal financial incentives for teachers.

Andrew Dustan, Juan Manuel Hernandez-Agramonte, Stanislao Maldonado

Cited by*: None Downloads*: None

We study how non-monetary incentives, motivated by recent advances in behavioral economics, affect civil servant performance in a context where state capacity is weak. We collaborated with a government agency in Peru to experimentally vary the context of text messages targeted to civil servants in charge of a school maintenance program. These messages incorporated behavioral insights in dimensions related to information provision, social norms, and weak forms of monitoring and auditing. We find that these messages are a very cost-effective strategy to enforce compliance with national policies among civil servants. We further study the role of social norms and the salience of social benefits in a follow-up experiment and explore the external validity or our original results by implementing a related experiment with civil servants from a different national program. The findings of these new experiments support our original results and provide additional insights regarding the context in which these incentives may work. Our results highlight the importance of carefully designed non-monetary incentives as a tool to improve civil servant performance when the state lacks institutional mechanisms to enforce compliance.

Indranil Goswami, Oleg Urminsky

Cited by*: None Downloads*: None

We present a complete empirical case study of fundraising decisions that demonstrates the importance of in-context field experiments. We first design novel matching-based fundraising appeals. We derive theory-based predictions from the standard impure altruism model and solicit expert opinion about the potential performance of our interventions. Both theory-based predictions and descriptive advice suggest improved fundraising performance from a framing intervention that credited donors for the matched funds (compared to a typical match framing). However, results from a natural field experiment with prior donors of a non-profit showed significantly poorer performance of this framing compared to a regularly framed matching intervention. This surprising finding was confirmed in a second natural field experiment, to establish the ground truth. Theoretically, our results highlight the limitations of both impure altruism models and of expert opinion in prediction complex "warm glow" motivation. More practically, our results question the availability of useful guidance, and suggest the indispensability of field testing for interventions in fundraising.

Loukas Balafoutas, Nikos Nikiforakis, Bettina Rockenbach

Cited by*: 0 Downloads*: None

Many interactions in modern human societies are among strangers. Explaining cooperation in such interactions is challenging. The two most prominent explanations critically depend on individuals' willingness to punish defectors: In models of direct punishment, individuals punish antisocial behavior at a personal cost, whereas in models of indirect reciprocity, they punish indirectly by withholding rewards. We investigate these competing explanations in a field experiment with real-life interactions among strangers. We find clear evidence of both direct and indirect punishment. Direct punishment is not rewarded by strangers and, in line with models of indirect reciprocity, is crowded out by indirect punishment opportunities. The existence of direct and indirect punishment in daily life indicates the importance of both means of understanding the evolution of cooperation.

Loukas Balafoutas, Nikos Nikiforakis, Bettina Rockenbach

Cited by*: 0 Downloads*: None

The degree of human cooperation among strangers is a major evolutionary puzzle. A prominent explanation is that cooperation maintained because many individuals have a predisposition to punish those violating group-beneficial norms. A critical condition for cooperation to evolve in evolutionary models is that punishment increases with the severity of the violation. Here we present evidence from a field experiment with real-life interactions that, unlike in lab experiments, altruistic punishment does not increase with the severity of the violation, regardless of whether it is direct (confronting a violator) or indirect (withholding help). We also document growing concerns for counter-punishment as the severity of the violation increases, indicating that the marginal cost of direct punishment increases with the severity of violations. The evidence suggests that altruistic punishment may not provide appropriate incentives to deter large violations. Our findings thus offer a rationale for the emergence of formal institutions for prompting large-scale cooperation among strangers.

Eszter Czibor, David Jimenez-Gomez, John A List

Cited by*: None Downloads*: None

What was once broadly viewed as an impossibility - learning from experimental data in economics - has now become commonplace. Governmental bodies, think tanks, and corporations around the world employ teams of experimental researchers to answer their most pressing questions. For their part, in the past two decades academics have begun to more actively partner with organizations to generate data via field experimentation. While this revolution in evidence-based approaches has served to deepen the economic science, recently a credibility crisis has caused even the most ardent experimental proponents to pause. This study takes a step back from the burgeoning experimental literature and introduces 12 actions that might help to alleviate this credibility crisis and raise experimental economics to an even higher level. In this way, we view our "12 action wish list" as discussion points to enrich the field.

John A List, Jeffrey A Livingston, Susanne Neckermann

Cited by*: None Downloads*: None

In the face of worryingly low performance on standardized test, offering students financial incentives linked to academic performance has been proposed as a potentially cost-effective way to support improvement. However, a large literature across disciplines finds that extrinsic incentives, once removed, may crowd out intrinsic motivation on subsequent, similar tasks. We conduct a field experiment where students, parents, and tutors are offered incentives designed to encourage student preparation for a high-stakes state test. The incentives reward performance on a separate low-stakes assessment designed to measure the same skills as the high-stakes test. Performance on the high-stakes test, however, is not incentivized. We find substantial treatment effects on the incented tests but no effect on the non-incented test; if anything, the incentives result in worse performance on the non-incented test. We also find evidence supporting the conclusion that the incentives crowd out intrinsic motivation to perform well on the non-incented test, but this effect is only temporary. One year later, students who had been in the incentives treatments perform better than those in the control on the same non-incented test.

John A List, Zacharias Maniadis, Fabio Tufano

Cited by*: None Downloads*: None

The sciences are in an era o fan alleged "credibility crisis'. In this study, we discuss the reproducibility of empirical results, focusing on economics research. By combining theory and empirical evidence, we discuss the import of replication studies, and whether they improve our confidence in novel findings. The theory sheds light on the importance of replications, even when replications are subject to bias. We then present a pilot meta-study of replication in experimental economics, a subfield serving as a positive benchmark for investigating the credibility of economics. Our meta-study highlights certain difficulties when applying meta-research (Ioannidis et al., 2015) and systematizing the economics literature.

John A List

Cited by*: None Downloads*: None

Once believed to be an impossibility, field experiments in economics now occupy a central place in the empiricist's quiver. In the past few decades alone field experiments have taken on much greater import in academe, across organizations, as well as for policymakers. But is this emergence simply a fad that will soon return field experiments to obscurity? I argue in this article that there is something fundamental about the emergence of field experiments, as controlling the assignment mechanism in the field provides unparalleled power to both understand the "effects of causes" and the "causes of effects." This knowledge generation then begins to uncover the generalizability and scalability of knowledge. Quite the opposite of a withering tool that will be gone tomorrow, I urge economists to "double down" on this comparative advantage and in doing so I provide four methodological paths which I hope will cement the promise and growth of field experiments in the social sciences.

Ran Kivetz, Oleg Urminsky, Yuhuang Zheng

Cited by*: None Downloads*: None

The goal-gradient hypothesis denotes the classic finding from behaviorism that animals expend more effort as they approach a reward. Building on this hypothesis, the authors generate new propositions for the human psychology of rewards. They test these propositions using a field experiment, secondary customer data, paper-and-pencil problems, and Tobit and logit models. The key finding indicate that (1) participants in a real cafe reward program purchase coffee more frequently the closer they are to earning a free coffee; (2) Internet users who rate songs in return for reward certificates visit the rating Web site more often, rate more songs per visit, and persist longer in the rating effort as they approach the reward goal; (3) the illusion of progress toward the goal induces purchase acceleration (e.g., customers who receive a 12-stamp coffee card with 2 preexisting "bonus" stamps complete the 10 required purchases faster than customers who receive a "regular" 10-stamp card) and (4) a stronger tendency to accelerate toward the goal predicts greater retention and faster reengagement in the program. The conceptualization and empirical findings are captured by a parsimonious goal distance model, in which effort investment is a function of the proportion of original distance remaining to the goal. In addition, using statistical and experimental controls, the authors rule out alternative explanations for the observed goal gradients. They discuss the theoretical significance of their findings and the managerial implications for incentive systems, promotions, and customer retention.

Damon Clark, David Gill, Victoria Prowse , Mark Rush

Cited by*: None Downloads*: None

Will college students who set goals for themselves work harder and achieve better outcomes? In theory, setting goals can help present-biased students to mitigate their self-control problem. In practice, there is little credible evidence on the causal effects on goal setting for college students. We report the result of two field experiments that involved almost four thousand college students in total. One experiments asked treated students to set goals for performance in the course; the other asked treated students to set goals for a particular task (completing online practice exams). Task-based goals had robust positive effects on the level of task completion, and task-based goals also increased course performance. We also find that performance-based goals had positive but small effects on course performance. We use a theoretical framework that builds on present bias and loss aversion to interpret our results. Since task-based goal setting is low-cost, scalable and logistically simple, we conclude that our findings have important implications for educational practice and future research.

Basil Halperin, Benjamin Ho, John A List, Ian Muir

Cited by*: None Downloads*: None

We use a theory of apologies to analyze a nationwide field experiment involving 1.5 million Uber ridesharing consumers who experienced late rides. Several insights emerge. First, apologies are not a panacea: the efficacy of an apology and whether it may backfire depend on how the apology is made. Second, across treatments, money speaks louder than words - the best form of apology is to include a coupon for a future trip. Third, in some cases sending an apology is worse than sending nothing at all, particularly for repeated apologies. For firms, caveat venditor should be the rule when considering apologies.

Alec Brandon, John A List, Robert D Metcalfe, Michael K Price, Florian Rundhammer

Cited by*: None Downloads*: None

This study considers the response of household electricity consumption to social nudges during peak load events. Our investigation considers two social nudges. The first targets conservation during peak load events, while the second promotes aggregate conservation. Using data from a natural field experiment with 42,100 households, we find that both social nudges reduce peak load electricity consumption by 2 to 4% when implemented in isolation and by nearly 7% when implemented in combination. These findings suggest an important role for social nudges in the regulation of electricity markets and a limited role for crowd out effects.

Omar Isaac Asensio, Magali A Delmas

Cited by*: 3 Downloads*: None

Little is known about the effect of message framing on conservation behavior over time. In a randomized controlled trial with residential households, we use advanced metering and information technologies to test how different messages about household energy use impact the dynamics of conservation behavior down to the appliance level. Our results, based on 374 million panel observations of kilowatt-hour (kWh) electricity consumption for 118 households over 9 months, show that differences in behavioral responses due to message framing become more significant over time. We find that a health-based frame, in which households consider the human health effects of their marginal electricity use, induced persistent energy savings behavior of 8-10% over 100 days; whereas a more traditional cost savings frame, drove sharp attenuation of treatment effects after 2 weeks with no significant savings versus control after 7 weeks. We discuss the implications for the design of effective information campaigns to engage households in conservation behavior.

Indranil Goswami, Oleg Urminsky

Cited by*: None Downloads*: None

How does setting a donation option as the default in a charitable appeal affect people's decisions? In eight studies, comprising 11,508 participants making 2,423 donation decisions in both experimental settings and a large scale-natural field experiment, we investigate the effect of "choice-option" defaults on the donation rate, average donation amount, and the resulting revenue. We find (1) a "lower-bar" effect, where defaulting a low amount increases donation rate, (2) a "scale-back" effect where low defaults reduce average donation amounts and (3) a "default-distraction' effect, where introducing any defaults reduces the effect of other cues, such as positive charity information. Contrary to the view that setting defaults will backfire, defaults increased revenue in our field study. However, our findings suggest that defaults can sometimes be a "self-cancelling" intervention, with countervailing effects of default option magnitude on decisions and resulting in no net effect on revenue. We discuss the implications of our findings for research on fundraising specifically, for choice architecture and behavioral interventions more generally, as well as for the use of "nudges" in policy decisions.

John A List

Cited by*: None Downloads*: None

"Putting Economic Research into Practice at Businesses (to Inform Science and Major Social Challenges)" Slides from ASSA NABE

John A List

Cited by*: None Downloads*: None

These are the slides from John A. List's keynote at the 2022 AFE conference.

Hansika Kapoor, Savita Kulkarni, Anirudh Tagat

Cited by*: None Downloads*: None

This study aims to investigate intra-household bargaining outcomes elicited in an artefactual field experiment design where participants completed a purchase task of real commodities. Married couples separately expressed their initial preferences over commodities. The bargaining process in the experiment was exogenously introduced by sharing information about partners' preferences in the treatment group. We hypothesized that the spouse with weaker bargaining position at the household level would consider the information of their partner's preferences while making own consumption decisions more compared to their partner. Therefore, they may deviate from their own preferences when purchasing commodities. More than 230 married couples from two villages in the Tamil Nadu state of India participated in the experiment. It was observed that information about partners' spending preferences resulted in reduced final allocation for female participants. However, the deviation was not significantly different from the original intention to spend. Therefore, information about partners' preferences may not be an effective medium to elicit bargaining power in the context of jointly-consumed household commodities. Subgroup analyses were performed to identify any heterogeneous treatment effects.

Syon Bhanot

Cited by*: None Downloads*: None

Across domains, people struggle to follow through on their commitments. This can happen for many reasons, including dishonesty, forgetfulness, or insufficient intrinsic motivation. Social scientists have explored the reasons for persistent failures to follow through. suggesting that eliciting explicit promises can be an effective way to motivate action. This paper presents a field experiment that tests the effect of explicit promises, in the form of "honor pledges", on loan repayment rates. The experiment was conducted with LendUp, an online lender, and targeted 4,883 first time-borrowers with the firm. Individuals were randomized into four groups, with the following experimental treatments: (1) having no honor pledge to complete (control); (2) signing a given honor pledge; (3) re-typing the same honor pledge as in (2) before signing; and (4) coming up with a personal honor pledge to type and sign. I also randomized whether or not borrowers were reminded of the honor pledge they signed prior to repayment deadline. The results suggest that the honor pledge treatments had minimal impacts on repayment, and that reminders of the pledges were similarly ineffective. This suggests that borrowers who fail to repay loans do so not because of dishonesty or behavioral biases, but because they suffer from true financial hardship and are simply unable to repay.

Erwin Bulte, John A List, Daan van Soest

Cited by*: None Downloads*: None

Social scientists have recently explored how framing of gains and losses affects productivity. We conducted a field experiment in peri-urban Uganda, and compared output levels across 1000 workers over isomorphic tasks and incentives, framed as either losses or gains. We find that loss aversion can be leveraged to increase the productivity of labor. The estimated welfare costs of using the loss contract are quite modest -- perhaps because the loss contract is viewed as a (soft) commitment device.