Papers - Framed

John A List

Cited by*: None Downloads*: None

Last year I put together a summary of data from my field experiments website that pertained to framed field experiments. Several people have asked if I have an update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper below.

Andreas Loschel, Michael K Price, Laura Razzolini, Madeline Werthschulte

Cited by*: None Downloads*: None

This study explores whether negative income shocks from the COVID-19 pandemic affect the demand for environmental policy. By running a survey in Germany in May 2020, we show that there is a large and negative correlation between the COVID-19 income shocks and the willingness to support green policies. Importantly, this relation is separate from the effect of long-run income. Building on the first evidence, our study provides directions for future valuation studies. Specifically, our results provide a proof of concept that welfare analyses based on willingness-to-pay estimates to assess the benefits of an environmental good or the cost of an environmental damage may be downward biased if temporary changes in income are not considered.

Andrea Morone, Paola Tiranzoni

Cited by*: None Downloads*: None

This study presents an analysis of hypothetical bias in WTA valuation connected with a bargaining game setting, in a field experiment context. The field of the experiment is the History Channel television "Pawn Stars". We collected a unique dataset that allowed us to analyze not only the gap between real and hypothetical WTA but also how they affect the bargaining game and vice versa. The general aim of this paper is to study the hypothetical bias related to subjects' WTA, and the factors that mostly affect it. The main results, of our paper, show that the hypothetical bias is positive, and it depends mainly on the price range and the type of good.

Michal Krawczyk

Cited by*: None Downloads*: None

This study is a field experiment on loss aversion. The framing of scoring rules was differentiated in two exams at the University of Warsaw, with only half the students facing explicit penalty points in the case of giving an incorrect answer. Loss aversion predicts that less risk will be taken (less questions will be answered) when losses are possible but in fact, no treatment effect was observed.

Rocco Caferra, Roberto Dell'Anno, Andrea Morone

Cited by*: None Downloads*: None

This paper aims to unmask the inadequacy of the review process of a sample of fee-charging journals in economics. We submitted a bait-manuscript to 104 academic economic journals to test whether there is a difference in the peer-review process between Article Processing Charges (APC) journals and Traditional journals which do not require a publication fee. The submitted bait-article was based on completely made-up data, with evident errors in terms of methodology, literature, reporting of results, and quality of language. Nevertheless, about half of the APC journals fell in the trap. Their editors accepted the article in the journals and required to pay the publication fee. We conclude that the Traditional model has a more effective incentive-mechanism in selecting articles, based on quality standards. Otherwise, we confirm that the so-called "Predatory Journals" - i.e. academic journals which accept papers without a quality check - exploit the APC scheme to increase their profits. They are also able to enter whitelists (e.g. Scopus, COPE). Accordingly, poor-quality articles published on APC journals shed the lights on the weakness of methodologies based on a mechanical inclusion of academic journals in scientific database indexes, succeeding in being considered for bibliometric evaluations of research institutions or scholars' productivity.

Gary Charness, John A List, Aldo Rustichini, Anya Samek, Jeroen van de Ven

Cited by*: None Downloads*: None

Theory of Mind (ToM), the ability to correctly attribute mental states to others, is important in social interactions. We evaluate the development of ToM in about 800 mostly disadvantaged children. We next conduct a field experiment with about 160 children in which we find that the low ToM rates for these disadvantaged children improve substantially in environments where the presence of other children is made salient. We see that ToM performance increases for both younger and older children in the treatment with strong salience, but that the treatment with weaker salience seems to be only effective in improving the ToM rates for older children.

Greg Allenby, Russell Belk, Catherine Eckel, Robert Fisher, Ernan Haruvy, John A List, Yu Ma, Peter Popkowski Leszczyc, Yu Wang, Sherry Xin Li

Cited by*: None Downloads*: None

We offer a unified conceptual, behavioral, and econometric framework for optimal fundraising that deals with both synergies and discrepancies between approaches from economics, consumer behavior, and sociology. The purpose is to offer a framework that can bridge differences and open a dialogue between disciplines in order to facilitate optimal fundraising design. The literature is extensive, and our purpose is to offer a brief background and perspective on each of the approaches, provide an integrated framework leading to new insights, and discuss areas of future research.

Eliot Abrams, Jonathan Libgober, John A List

Cited by*: None Downloads*: None

The past few decades have ushered in an experimental revolution in economics whereby scholars are now much more likely to generate their own data. While there are virtues associated with this movement, there are concomitant difficulties. Several scientific disciplines, including economics, have launched research registries in an effort to attenuate key inferential issues. This study assesses registries both empirically and theoretically, with a special focus on the AEA registry. We find that over 90% of randomized controlled trials (RCTs) in economics do not register, only 50% of the RCTs that register do so before the intervention begins, and the majority of these preregistrations are not detailed enough to significantly aid inference. Our empirical analysis further shows that using other scientific registries as aspirational examples is misguided, as their perceived success in tackling the main issues is largely a myth. In light of these facts, we advance a simple economic model to explore potential improvements. A key insight from the model is that removal of the (current) option to register completed RCTs could increase the fraction of trials that register. We also argue that linking IRB applications to registrations could further increase registry effectiveness.

Michal Krawczyk

Cited by*: None Downloads*: None

Several studies have identified the "better than average" effect - the tendency of most people to think they are better than most other people on most dimensions. The effect would have profound consequences (see e.g. Barber and Odean (2001)). These findings are predominantly based on non-incentivized, non-verifiable self-reports. The current study looks at the impact of incentives to judge one's abilities accurately in a framed field experiment. Nearly 400 students were asked to predict whether they would do better or worse than average in an exam. The most important findings are that subjects tend to show more confidence when incentivized and when asked before the exam rather than afterwards. The first effect shows particularly in females.

Wojciech Hardy, Michal Krawczyk, Joanna Tyrowicz

Cited by*: None Downloads*: None

In this project, we investigate downloading and sharing behaviour in a novel 'piracy game' modelled after standard public good games. We find that willingness to share correlates positively with the sharing by others. By contrast, actual behaviour in the 'piracy game' is not correlated with self-reported behaviour.

Eszter Czibor, David Jimenez-Gomez, John A List

Cited by*: None Downloads*: None

What was once broadly viewed as an impossibility - learning from experimental data in economics - has now become commonplace. Governmental bodies, think tanks, and corporations around the world employ teams of experimental researchers to answer their most pressing questions. For their part, in the past two decades academics have begun to more actively partner with organizations to generate data via field experimentation. While this revolution in evidence-based approaches has served to deepen the economic science, recently a credibility crisis has caused even the most ardent experimental proponents to pause. This study takes a step back from the burgeoning experimental literature and introduces 12 actions that might help to alleviate this credibility crisis and raise experimental economics to an even higher level. In this way, we view our "12 action wish list" as discussion points to enrich the field.

Shubha Chakravarty, Mattias Lundberg, Plamen Nikolov, Juliane Zenker

Cited by*: None Downloads*: None

Lack of skills is arguably one of the most important determinants of high levels of unemployment and poverty. In response, policymakers often initiate vocational training programs in effort to enhance skill formation among the youth. Using a regression-discontinuity design, we examine a large youth training intervention in Nepal. We find, twelve months after the start of the training program, that the intervention generated an increase in non-farm employment of 10 percentage points (ITT estimates) and up to 31 percentage points for program compliers (LATE estimates). We also detect sizable gains in monthly earnings. Women who start self-employment activities inside their homes largely drive these impacts. We argue that low baseline educational levels and non-farm employment levels and Nepal's social and cultural norms towards women drive our large program impacts. Our results suggest that the program enables otherwise underemployed women to earn an income while staying at home - close to household errands and in line with the socio-cultural norms that prevent them from taking up employment outside the house.

John A List, Zacharias Maniadis, Fabio Tufano

Cited by*: None Downloads*: None

The sciences are in an era o fan alleged "credibility crisis'. In this study, we discuss the reproducibility of empirical results, focusing on economics research. By combining theory and empirical evidence, we discuss the import of replication studies, and whether they improve our confidence in novel findings. The theory sheds light on the importance of replications, even when replications are subject to bias. We then present a pilot meta-study of replication in experimental economics, a subfield serving as a positive benchmark for investigating the credibility of economics. Our meta-study highlights certain difficulties when applying meta-research (Ioannidis et al., 2015) and systematizing the economics literature.

John A List, Jeffrey A Livingston, Susanne Neckermann

Cited by*: None Downloads*: None

In the face of worryingly low performance on standardized test, offering students financial incentives linked to academic performance has been proposed as a potentially cost-effective way to support improvement. However, a large literature across disciplines finds that extrinsic incentives, once removed, may crowd out intrinsic motivation on subsequent, similar tasks. We conduct a field experiment where students, parents, and tutors are offered incentives designed to encourage student preparation for a high-stakes state test. The incentives reward performance on a separate low-stakes assessment designed to measure the same skills as the high-stakes test. Performance on the high-stakes test, however, is not incentivized. We find substantial treatment effects on the incented tests but no effect on the non-incented test; if anything, the incentives result in worse performance on the non-incented test. We also find evidence supporting the conclusion that the incentives crowd out intrinsic motivation to perform well on the non-incented test, but this effect is only temporary. One year later, students who had been in the incentives treatments perform better than those in the control on the same non-incented test.

Greer K Gosnell, John A List, Robert D Metcalfe

Cited by*: None Downloads*: None

Increasing evidence indicates the importance of management in determining firms' productivity. Yet, causal evidence regarding the effectiveness of management practices is scarce, especially for high-skilled workers in the developed world. In an eight-month field experiment measuring the productivity of captains in the commercial aviation sector, we test four distinct management practices: (i) performance monitoring; (ii) performance feedback; (iii) target setting; and (iv) prosocial incentives. We find that these management practices -particularly performance monitoring and target setting- significantly increase captains' productivity with respect to the targeted fuel-saving dimensions. We identify positive spillovers of the tested management practices on job satisfaction and carbon dioxide emissions, and captains overwhelmingly express desire for deeper managerial engagement. Both the implementation and the results of the study reveal an uncharted opportunity for management researchers to delve into the black box of firms and rigorously examine the determinants of productivity amongst skilled labor.

Omar Al-Ubaydli, John A List, Claire Mackevicius, Min Sok Lee, Dana L Suskind

Cited by*: None Downloads*: None

Policymakers are increasingly turning to insights gained from the experimental method as a means to inform large scale public policies. Critics view this increased usage as premature, pointing to the fact that many experimentally-tested programs fail to deliver their promise at scale. Under this view, the experimental approach drives too much public policy. Yet, if policymakers could be more confident that the original research findings would be delivered at scale, even the staunchest critics would carve out a larger role for experiments to inform policy. Leveraging the economic framework of Al-Ubaydli et al. (2019), we put forward 12 simple proposals, spanning researchers, policymakers, funders, and stakeholders, which together tackle the most vexing scalability threats. The framework highlights that only after we deepen our understanding of the scale up problem will we be on solid ground to argue that scientific experiments should hold a more prominent place in the policymaker's quiver.

Omar Al-Ubaydli, John A List, Dana L Suskind

Cited by*: None Downloads*: None

Policymakers are increasingly turning to insights gained from the experimental method as a means of informing public policies. Whether-and to what extent-insights from a research study scale to the level of the broader public is, in many situations, based on blind faith. This scale-up problem can lead to a vast waste of resources, a missed opportunity to improve people's lives, and a diminution in the public's trust in the scientific method's ability to contribute to policymaking. This study provides a theoretical lens to deepen our understanding of the science of how to use science. Through a simple model, we highlight three elements of the scale-up problem: (1) when does evidence become actionable (appropriate statistical inference); (2) properties of the population; and (3) properties of the situation. We argue that until these three areas are fully understood and recognized by researchers and policymakers, the threats to scalability will render any scaling exercise as particularly vulnerable. In this way, our work represents a challenge to empiricists to estimate the nature and extent of how important the various threats to scalability are in practice, and to implement those in their original research.

Anouar El Haji, Michal Krawczyk, Marta Sylwestrzak, Ewa Zawojska

Cited by*: None Downloads*: None

Auctions often require risk taking under time pressure. However, little is known about how time pressure moderates the relationship between uncertainty of outcomes and bidding behavior. This study consists of a field experiment in which participants are invited to a Vickrey auction to elicit their willingness to pay for a lottery ticket. The time available to place a bid and also the skewness of the lottery (holding the expected value constant) are systematically manipulated. We find that under high time pressure participants are less likely to place a bid at all. Furthermore, participants who do place a bid under high time pressure bid significantly less than participants under low time pressure. The main finding is thus that increased time pressure significantly decreases risk taking. The effect seems to be particularly strong for the lottery with a high probability of winning and for female subjects.

Omar Al-Ubaydli, John A List

Cited by*: None Downloads*: None

This is a review of the literature of field experimental studies of markets. The main results covered by the review are as follows: (1) Generally speaking, markets organize the efficient exchange of commodities; (2) There are some behavioral anomalies that impede efficient exchange; (3) Many behavioral anomalies disappear when traders are experienced.

John A List

Cited by*: None Downloads*: None

While empirical economics has made important strides over the past half century, there is a recent attack that threatens the foundations of the empirical approach in economics: external validity. Certain dogmatic arguments are not new, yet in some circles the generalizability question is beyond dispute, rendering empirical work as a passive enterprise based on frivolity. Such arguments serve to caution even the staunchest empirical advocates from even starting an empirical inquiry in a novel setting. In its simplest form, questions of external validity revolve around whether the results of the received study can be generalized to different people, situations, stimuli, and time periods. This study clarifies and places the external validity crisis into perspective by taking a unique glimpse into the grandest of trials: The External Validity Trial. A key outcome of the proceedings is an Author Onus Probandi, which outlines four key areas that every study should report to address external validity. Such an evaluative approach properly rewards empirical advances and justly recognizes inherent empirical limitations.