Papers - Artefactual

John A List

Cited by*: Downloads*:

ASSA 2023 presentation

John A List

Cited by*: Downloads*:

A summary of artefactual field experiments on fieldexperiments.com.

Omar Al-Ubaydli, John A List, Dana L Suskind

Cited by*: Downloads*:

Policymakers are increasingly turning to insights gained from the experimental method as a means of informing public policies. Whether-and to what extent-insights from a research study scale to the level of the broader public is, in many situations, based on blind faith. This scale-up problem can lead to a vast waste of resources, a missed opportunity to improve people's lives, and a diminution in the public's trust in the scientific method's ability to contribute to policymaking. This study provides a theoretical lens to deepen our understanding of the science of how to use science. Through a simple model, we highlight three elements of the scale-up problem: (1) when does evidence become actionable (appropriate statistical inference); (2) properties of the population; and (3) properties of the situation. We argue that until these three areas are fully understood and recognized by researchers and policymakers, the threats to scalability will render any scaling exercise as particularly vulnerable. In this way, our work represents a challenge to empiricists to estimate the nature and extent of how important the various threats to scalability are in practice, and to implement those in their original research.

Uri Gneezy, Moshe Hoffman, Mark A. Lane, John A List, Jeffrey A Livingston, Michael J. Seiler

Cited by*: Downloads*:

Recent theoretical work shows that the better-than-average effect, where a majority believes their ability to be better than average, can be perfectly consistent with Bayesian updating. However, later experiments that account for this theoretical advance still find behavior consistent with overconfidence. The literature notes that overoptimism can be caused by either overconfidence (optimism about performance), wishful thinking (optimism about outcomes), or both. To test whether the better-than-average effect might be explained by wishful thinking instead of overconfidence, we conduct an experiment that is similar to those used in the overconfidence literature, but removes performance as a potential channel. We find evidence that wishful thinking might explain overconfidence only among the most optimistic subjects, and that conservatism is possibly more of a worry; if unaccounted for, overconfidence might be underestimated.

John A List, Ragan Petrie, Anya Samek

Cited by*: Downloads*:

In the past several decades the experimental method has lent deep insights into economics. One surprising area that has contributed is the experimental study of children, where advances as varied as the evolution of human behaviors that shape markets and institutions to how early life influences shape later life outcomes have been explored. We first develop a framework for economic preference measurement that provides a lens into how to interpret data from experiments with children. Next, we survey work that provides general empirical insights within our framework. Finally, we provide 10 tips for pulling off experiments with children, including factors such as taking into account child competencies, causal identification, and logistical issues related to recruitment and implementation. We envision the experimental study of children as a high growth research area in the coming decades as social scientists begin to more fully appreciate that children are active participants in markets who (might) respond predictably to economic incentives.

Omar Al-Ubaydli, John A List, Claire Mackevicius, Min Sok Lee, Dana L Suskind

Cited by*: Downloads*:

Policymakers are increasingly turning to insights gained from the experimental method as a means to inform large scale public policies. Critics view this increased usage as premature, pointing to the fact that many experimentally-tested programs fail to deliver their promise at scale. Under this view, the experimental approach drives too much public policy. Yet, if policymakers could be more confident that the original research findings would be delivered at scale, even the staunchest critics would carve out a larger role for experiments to inform policy. Leveraging the economic framework of Al-Ubaydli et al. (2019), we put forward 12 simple proposals, spanning researchers, policymakers, funders, and stakeholders, which together tackle the most vexing scalability threats. The framework highlights that only after we deepen our understanding of the scale up problem will we be on solid ground to argue that scientific experiments should hold a more prominent place in the policymaker's quiver.

Syon Bhanot, Jiyoung Han, Chaning Jang

Cited by*: Downloads*:

Restrictions like work requirements and constraints on voucher transfers are often used in social welfare systems, but little empirical evidence exists on their impact on wellbeing. We conducted a 10-day randomized experiment with 432 individuals living below the poverty line in the Kawangware settlement of Nairobi, kenya, testing two elements of social welfare design: workfare versus welfare and restricted versus unrestricted vouchers. Participants were randomly assigned to a "Work" condition, involving daily work for unrestricted vouchers, or one of two "Wait" conditions, involving daily waiting for vouchers that were either unrestricted or partially restricted to staple foods. We find that working improved psychological wellbeing relative to waiting, suggesting that means of implementing welfare programs may have important effects on individuals beyond the impact of monetary benefit alone. Furthermore, although restrictions were inframarginal, partially restricted vouchers crowded-in spending on staple foods, suggesting the existence of a "flypaper effect" in spending from restricted vouchers.

Yan Chen, Peter Cramton, John A List, Axel Ockenfels

Cited by*: Downloads*:

We review past research and discuss future directions on how the vibrant research areas of market design and behavioral economics have influenced and will continue to impact the science and practice of management in both the private and public sectors. Using examples from various auction markets, reputation and feedback systems in online markets, matching markets in education, and labor markets, we demonstrate that combining market design theory, behavioral insights, and experimental methods can lead to fruitful implementation of superior market designs in practice.

Ori Heffetz, John A List

Cited by*: Downloads*:

Carefully designed scientific experiments have been an engine of economic, technological, and social progress for well over a century, which is why the public generally trusts such methods. Unfortunately, governments around the world still routinely oppose controlled trials of public policies.

Eszter Czibor, David Jimenez-Gomez, John A List

Cited by*: Downloads*:

What was once broadly viewed as an impossibility - learning from experimental data in economics - has now become commonplace. Governmental bodies, think tanks, and corporations around the world employ teams of experimental researchers to answer their most pressing questions. For their part, in the past two decades academics have begun to more actively partner with organizations to generate data via field experimentation. While this revolution in evidence-based approaches has served to deepen the economic science, recently a credibility crisis has caused even the most ardent experimental proponents to pause. This study takes a step back from the burgeoning experimental literature and introduces 12 actions that might help to alleviate this credibility crisis and raise experimental economics to an even higher level. In this way, we view our "12 action wish list" as discussion points to enrich the field.

Isabelle Brocas, Juan D Carrillo

Cited by*: Downloads*:

Adults do not play the Nash equilibrium in the well known centipede game. While Palacios-Huerta and Volij (2009) argued that behavior results from the failure of backward induction logic, Levitt et al. (2011) found that players who know how to backward induct still do not play Nash. Here, we ask children and adolescents (ages 8 to 16) to play the centipede game in the laboratory and we leverage knowledge about developing abilities to assess the contribution of backward induction logic. In line with the literature, we find that the ability to perform backward induction increases with age. However, it predicts behavior only in elementary school children: those with advanced logical abilities over-apply their skills. Starting in middle school, students who reason logically know that the unraveling argument should not be applied blindly. They utilize Theory-of-Mind (ToM) abilities to form beliefs about others' play and (optimally) refrain from stopping immediately. Their behavior is in line with the deviations observed in adults. Interestingly, developing ToM leads to a gradual decrease in stopping stages with age, which is accompanied by a decrease in payoffs with age. The results indicate that ToM is the key contributor of behavior that helps departing from backward induction when beneficial.

Wojciech Hardy, Michal Krawczyk, Joanna Tyrowicz

Cited by*: Downloads*:

We report results of an experimental study analyzing the effects of Internet piracy on book sales. We conducted a year-long controlled large-scale field experiment with pre-treatment pair matching. Half of the book titles received experiment treatment, in which a specialized agency would immediately remove any unauthorized copy appearing on the Internet. For the other half we merely registered such occurrences, but no countermeasures were taken. For all the titles we obtained print and e-book sales statistics from the publishers. We find that removal of unauthorized copies was an effective method of curbing piracy, but this had no bearing on legal sales.

Greg Allenby, Russell Belk, Catherine Eckel, Robert Fisher, Ernan Haruvy, John A List, Yu Ma, Peter Popkowski Leszczyc, Yu Wang, Sherry Xin Li

Cited by*: Downloads*:

We offer a unified conceptual, behavioral, and econometric framework for optimal fundraising that deals with both synergies and discrepancies between approaches from economics, consumer behavior, and sociology. The purpose is to offer a framework that can bridge differences and open a dialogue between disciplines in order to facilitate optimal fundraising design. The literature is extensive, and our purpose is to offer a brief background and perspective on each of the approaches, provide an integrated framework leading to new insights, and discuss areas of future research.

John A List

Cited by*: Downloads*:

While empirical economics has made important strides over the past half century, there is a recent attack that threatens the foundations of the empirical approach in economics: external validity. Certain dogmatic arguments are not new, yet in some circles the generalizability question is beyond dispute, rendering empirical work as a passive enterprise based on frivolity. Such arguments serve to caution even the staunchest empirical advocates from even starting an empirical inquiry in a novel setting. In its simplest form, questions of external validity revolve around whether the results of the received study can be generalized to different people, situations, stimuli, and time periods. This study clarifies and places the external validity crisis into perspective by taking a unique glimpse into the grandest of trials: The External Validity Trial. A key outcome of the proceedings is an Author Onus Probandi, which outlines four key areas that every study should report to address external validity. Such an evaluative approach properly rewards empirical advances and justly recognizes inherent empirical limitations.

Greta List, John A List, Lina Ramirez, Anya Samek

Cited by*: Downloads*:

We conduct experiments with 720 children ages 9-11 to evaluate the relationship of time and risk preferences with health. Children who are more patient report consuming fewer unhealthy calories and spending less time on sedentary activities such as video games. Children who are more risk seeking report engaging in more exercise and more screen time. However, time and risk preferences are not predictive of body mass index (BMI). Moreover, some of the negative health behaviors, such as screen time, are associated with lower - rather than higher - BMI.

Annika List, John A List, Anya Samek

Cited by*: Downloads*:

Social scientists for years have documented the pervasiveness of discrimination in product and labor markets. While the literature has recently attempted to measure the nature of such discrimination, much less work has been done exploring the origins of discrimination. We make a modest step in this direction by reporting data from a field experiment attempting to measure discrimination amongst 3-5 years olds. Using a design that isolated discriminatory behaviors in economic games, we find that both White and Hispanic children send more resources to Black children than White children, whereas black children send equal amounts. This provides a first glimpse that suggests preferences amongst the young do not show similar patterns as preferences of adults.

John A List

Cited by*: Downloads*:

Last year I put together a summary of data from my field experiments website that pertained to artefactual field experiments. Several people have asked me if I have and update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper.

Snigdha Gupta, John A List, Lauren Supplee, Dana L Suskind

Cited by*: Downloads*:

Failed to Scale: Embracing the Challenge of Scaling in Early Childhood

Rocco Caferra, Alessia Casamassima, Alessandro Cascavilla, Andrea Morone, Paola Tiranzoni

Cited by*: Downloads*:

This work aims to identify and quantify the biases behind the anomalous behavior of people when they deal with the Three Doors dilemma, which is a really simple but counterintuitive game. Carrying out an artefactual field experiment and proposing eight different treatments to isolate the anomalies, we provide new interesting experimental evidence on the reasons why subjects fail to take the optimal decision. According to the experimental results, we can quantify the size and the impact of three main biases that explain the anomalous behavior of participants: Bayesian updating, illusion of control, and status quo bias.

Luigi Butera, Philip J Grossman, Daniel Houser, John A List, Marie-Claire Villeval

Cited by*: Downloads*:

Creation of empirical knowledge in economics has taken a dramatic turn in the past few decades. One feature of the new research landscape is the nature and extent to which scholars generate data. Today, in nearly every field the experimental approach plays an increasingly crucial role in testing theories and informing organizational decisions. Whereas there is much to appreciate about this revolution, recently a credibility crisis has taken hold across the social sciences, arguing that an important component of Fischer (1935)'s tripod has not been fully embraced: replication. Indeed, while the importance of replications is not debatable scientifically, current incentives are not sufficient to encourage replications from the individual researcher's perspective. We propose a novel mechanism that promotes replications by leveraging mutually beneficial gains between scholars and editors. We develop a model capturing the trade-offs involved in seeking independent replications before submission of a paper to journals. We showcase our method via an investigation of the effects of Knightian uncertainty on cooperation rates in public goods games, a pervasive and yet largely unexplored feature in the literature.