Papers - Framed

Amanda Chuan, John A List, Anya Samek, Shreemayi Samujjwala

Cited by*: Downloads*:

Parental investments shape children's educational specializations. Using a longitudinal study, we find that parents invest more in daughters than sons at ages 3-5. We find that early parental investment can explain persistently higher English scores for girls than boys 4-6 years later. However, there is no gender gap in Math. Parental investments at ages 3-5 appear to contribute to girls' advantage in English but have little impact on Math. Our results suggest that parental investments at early ages contributes to girls' comparative advantage in English.

Roland Fryer , Steven D Levitt, John A List, Anya Samek

Cited by*: Downloads*:

We present the results of a novel early childhood intervention in which disadvantaged 3-4-year-old children were randomized to receive a new preschool and parents education program focused on cognitive and non-cognitive skills (CogX) or to a control group that did not receive preschool education. In addition to a typical academic year (9 month) program, we also evaluated a shortened summer version of the program (2 months) in which children were treated immediately prior to the start of Kindergarten. Both programs, including the shortened version, significantly improved cognitive test scores by about one quarter of a standard deviation relative to the control group at the end of the year. The shortened version of the program was equally as effective as the academic-year program because most of the gains in the academic-year program occurred within the first few months.

Alec Brandon, Christopher M Clapp, John A List, Robert D Metcalfe, Michael K Price

Cited by*: Downloads*:

Smart-home technologies have been heralded as an important way to increase energy conservation. While in vitro engineering estimates provide broad optimism, little has been done to explore whether such estimates scale beyond the lab. We estimate the causal impact of smart thermostats on energy use via two novel framed field experiments in which a random subset of treated households have a smart thermostat installed in their home. Examining 18 months of associated high-frequency data on household energy consumption, yielding more than 16 million hourly electricity and daily natural gas observations, we find little evidence that smart thermostats have a statistically or economically significant effect on energy use. We explore potential mechanisms using almost four million observations of system events including human interactions with their smart thermostat. Results indicate that user behavior dampens energy savings and explains the discrepancy between estimates from engineering models, which assume a perfectly compliant subject, and actual households, who are occupied by users acting in accord with behavioral economists' conjectures. In this manner, our data document a keen threat to the scalability of new user-based technologies.

John A List

Cited by*: Downloads*:

Last year I put together a summary of data from my field experiments website that pertained to framed field experiments. Several people have asked if I have an update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper below.

Erwin Bulte, John A List, Daan van Soest

Cited by*: Downloads*:

Incomplete contracts are the rule rather than the exception, and any incentive scheme faces the risk of improving performance on incented aspects of a task at the detriment of performance on non-incented aspects. Recent research documents the effect of loss-framed versus gain-framed incentives on incentivized behavior, but how do such incentives affect overall performance? We explore potential trade-offs by conducting field experiments in an artificial "workplace". We explore two types of incentive spillovers: those contemporaneous to the incented task and those subsequent to the incented task. We report three main results. First, consonant with the extant literature, a loss aversion incentive induces greater effort on the incented task. Second, offsetting this productivity gain, we find that the quality of work decreases if quality is not specified in the incentive contract. Third, we find no evidence of harmful spillover effects to subsequent tasks; if anything, the loss aversion incentive induces more effort in subsequent tasks. Taken together, our results highlight that measuring and accounting for incentive spillovers are important when considering their overall impact.

Michal Krawczyk

Cited by*: Downloads*:

Several studies have identified the "better than average" effect - the tendency of most people to think they are better than most other people on most dimensions. The effect would have profound consequences (see e.g. Barber and Odean (2001)). These findings are predominantly based on non-incentivized, non-verifiable self-reports. The current study looks at the impact of incentives to judge one's abilities accurately in a framed field experiment. Nearly 400 students were asked to predict whether they would do better or worse than average in an exam. The most important findings are that subjects tend to show more confidence when incentivized and when asked before the exam rather than afterwards. The first effect shows particularly in females.

Kentaro Asai, Seda Ertac, Ali Hortacsu, John A List, Howard Nusbaum, Lester Tong, Karen Ye

Cited by*: Downloads*:

People often demand a greater price when selling goods that they own than they would pay to purchase the same goods- a well-known economic bias called the endowment effect. The endowment effect has been found to be muted among experienced traders, but little is known about how trading experience reduces the endowment effect. We show that when selling, experienced traders exhibit lower right anterior insula activity, but no differences in nucleus accumbens or orbitofrontal activation, compared with inexperienced traders. Furthermore, insula activation mediates the effect of experience on the endowment effect. Similar results are obtained for inexperienced traders who are incentivized to gain trading experience. This finding indicates that frequent trading likely mitigates the endowment effect indirectly by modifying negative affective responses in the context of selling.

John A List, Jeffrey A Livingston, Susanne Neckermann

Cited by*: Downloads*:

In the face of worryingly low performance on standardized test, offering students financial incentives linked to academic performance has been proposed as a potentially cost-effective way to support improvement. However, a large literature across disciplines finds that extrinsic incentives, once removed, may crowd out intrinsic motivation on subsequent, similar tasks. We conduct a field experiment where students, parents, and tutors are offered incentives designed to encourage student preparation for a high-stakes state test. The incentives reward performance on a separate low-stakes assessment designed to measure the same skills as the high-stakes test. Performance on the high-stakes test, however, is not incentivized. We find substantial treatment effects on the incented tests but no effect on the non-incented test; if anything, the incentives result in worse performance on the non-incented test. We also find evidence supporting the conclusion that the incentives crowd out intrinsic motivation to perform well on the non-incented test, but this effect is only temporary. One year later, students who had been in the incentives treatments perform better than those in the control on the same non-incented test.

John A List, Dana L Suskind

Cited by*: Downloads*:

Op-ed

John A List

Cited by*: Downloads*:

In 2019 I put together a summary of data from my field experiments website that pertained to framed field experiments (see List 2024). Several people have asked me if I have an update. In this document I update all figures and numbers to show the details for 2023. I also include the description from the 2019 paper below.

Greg Allenby, Russell Belk, Catherine Eckel, Robert Fisher, Ernan Haruvy, John A List, Yu Ma, Peter Popkowski Leszczyc, Yu Wang, Sherry Xin Li

Cited by*: Downloads*:

We offer a unified conceptual, behavioral, and econometric framework for optimal fundraising that deals with both synergies and discrepancies between approaches from economics, consumer behavior, and sociology. The purpose is to offer a framework that can bridge differences and open a dialogue between disciplines in order to facilitate optimal fundraising design. The literature is extensive, and our purpose is to offer a brief background and perspective on each of the approaches, provide an integrated framework leading to new insights, and discuss areas of future research.

Matthew A. Kraft, John A List, Jeffrey A Livingston, Sally Sadoff

Cited by*: Downloads*:

In-person tutoring programs can have large impacts on K-12 student achievement, but high program costs and limited local supply of tutors have hampered scale-up. Online tutoring provided by volunteers can potentially reach more students in need. We implemented a randomized pilot program of online tutoring that paired college volunteers with middle school students. We estimate consistently positive but statistically insignificant effects on student achievement, 0.07s for math and 0.04s for reading. While our estimated effects are smaller than those for many higher-dosage in-person programs, they are from a significantly lower-cost program delivered within the challenging context of the COVID-19 pandemic.

Christopher Cotton, Brent R Hickman, John A List, Joseph Price, Sutanuka Roy

Cited by*: Downloads*:

We conduct a field experiment across three diverse school districts to structurally identify student motivation and study productivity parameters in a model of adolescent human capital development. By observing study time, homework task completion, and test results, we can identify individual and demographic variations in motivation and study time effectiveness. Struggling students typically do not lack motivation but rather struggle to convert study time into completed assignments and proficiency improvements. The study also attending a higher-performing school is associated with both higher productivity and higher motivation relative to peers with similar observables in lower-performing schools. Counterfactual analyses estimates that school quality differences account for a substantial share of the racial differences in test scores, and considers the impact of alternative policies aimed at reducing racial performance gaps.

John A List

Cited by*: Downloads*:

In 2019 I put together a summary of data from my field experiments website that pertained to framed field experiments. Several people have asked me if I have update. In this document I update all figures and numbers to show the details for 2021. I also include the description from the 2019 paper below.

Omar Al-Ubaydli, John A List, Dana L Suskind

Cited by*: Downloads*:

Policymakers are increasingly turning to insights gained from the experimental method as a means of informing public policies. Whether-and to what extent-insights from a research study scale to the level of the broader public is, in many situations, based on blind faith. This scale-up problem can lead to a vast waste of resources, a missed opportunity to improve people's lives, and a diminution in the public's trust in the scientific method's ability to contribute to policymaking. This study provides a theoretical lens to deepen our understanding of the science of how to use science. Through a simple model, we highlight three elements of the scale-up problem: (1) when does evidence become actionable (appropriate statistical inference); (2) properties of the population; and (3) properties of the situation. We argue that until these three areas are fully understood and recognized by researchers and policymakers, the threats to scalability will render any scaling exercise as particularly vulnerable. In this way, our work represents a challenge to empiricists to estimate the nature and extent of how important the various threats to scalability are in practice, and to implement those in their original research.

Rocco Caferra, Roberto Dell'Anno, Andrea Morone

Cited by*: Downloads*:

This paper aims to unmask the inadequacy of the review process of a sample of fee-charging journals in economics. We submitted a bait-manuscript to 104 academic economic journals to test whether there is a difference in the peer-review process between Article Processing Charges (APC) journals and Traditional journals which do not require a publication fee. The submitted bait-article was based on completely made-up data, with evident errors in terms of methodology, literature, reporting of results, and quality of language. Nevertheless, about half of the APC journals fell in the trap. Their editors accepted the article in the journals and required to pay the publication fee. We conclude that the Traditional model has a more effective incentive-mechanism in selecting articles, based on quality standards. Otherwise, we confirm that the so-called "Predatory Journals" - i.e. academic journals which accept papers without a quality check - exploit the APC scheme to increase their profits. They are also able to enter whitelists (e.g. Scopus, COPE). Accordingly, poor-quality articles published on APC journals shed the lights on the weakness of methodologies based on a mechanical inclusion of academic journals in scientific database indexes, succeeding in being considered for bibliometric evaluations of research institutions or scholars' productivity.

John A List

Cited by*: Downloads*:

While empirical economics has made important strides over the past half century, there is a recent attack that threatens the foundations of the empirical approach in economics: external validity. Certain dogmatic arguments are not new, yet in some circles the generalizability question is beyond dispute, rendering empirical work as a passive enterprise based on frivolity. Such arguments serve to caution even the staunchest empirical advocates from even starting an empirical inquiry in a novel setting. In its simplest form, questions of external validity revolve around whether the results of the received study can be generalized to different people, situations, stimuli, and time periods. This study clarifies and places the external validity crisis into perspective by taking a unique glimpse into the grandest of trials: The External Validity Trial. A key outcome of the proceedings is an Author Onus Probandi, which outlines four key areas that every study should report to address external validity. Such an evaluative approach properly rewards empirical advances and justly recognizes inherent empirical limitations.

Omar Al-Ubaydli, John A List, Claire Mackevicius, Min Sok Lee, Dana L Suskind

Cited by*: Downloads*:

Policymakers are increasingly turning to insights gained from the experimental method as a means to inform large scale public policies. Critics view this increased usage as premature, pointing to the fact that many experimentally-tested programs fail to deliver their promise at scale. Under this view, the experimental approach drives too much public policy. Yet, if policymakers could be more confident that the original research findings would be delivered at scale, even the staunchest critics would carve out a larger role for experiments to inform policy. Leveraging the economic framework of Al-Ubaydli et al. (2019), we put forward 12 simple proposals, spanning researchers, policymakers, funders, and stakeholders, which together tackle the most vexing scalability threats. The framework highlights that only after we deepen our understanding of the scale up problem will we be on solid ground to argue that scientific experiments should hold a more prominent place in the policymaker's quiver.

Eszter Czibor, David Jimenez-Gomez, John A List

Cited by*: Downloads*:

What was once broadly viewed as an impossibility - learning from experimental data in economics - has now become commonplace. Governmental bodies, think tanks, and corporations around the world employ teams of experimental researchers to answer their most pressing questions. For their part, in the past two decades academics have begun to more actively partner with organizations to generate data via field experimentation. While this revolution in evidence-based approaches has served to deepen the economic science, recently a credibility crisis has caused even the most ardent experimental proponents to pause. This study takes a step back from the burgeoning experimental literature and introduces 12 actions that might help to alleviate this credibility crisis and raise experimental economics to an even higher level. In this way, we view our "12 action wish list" as discussion points to enrich the field.

John A List, Lina Ramirez, Julia Seither, Jaime Unda, Beatriz Vallejo

Cited by*: Downloads*:

Misinformation represents a vital threat to the societal fabric of modern economies. While the supply side of the misinformation market has begun to receive increased scrutiny, the demand side has received scant attention. We explore the demand for misinformation through the lens of augmenting critical thinking skills in a field experiment during the 2022 Presidential election in Colombia. Data from roughly 2.000 individual suggest that our treatments enhance critical thinking, causing subjects to more carefully consider the truthfulness of potential misinformation. We furthermore provide evidence that reducing the demand of fake news can deliver on the dual goal of reducing the spread of fake news by encouraging reporting of misinformation.