Papers - Artefactual
Marco Castillo,
John A List,
Ragan Petrie,
Anya Samek
Cited by*: None Downloads*: None 
We use field experiments with nearly 900 children to investigate how skills developed at ages 3-5 drive later-life outcomes. We find that skills map onto three distinct factors - cognitive skills, executive functions, and economic preferences. Returning to the children up to 7 years later, we find that executive functions, but not cognitive skills, predict the likelihood of receiving disciplinary referrals. Economic preferences have an independent effect: children who displayed impatience at ages 3-5 were more likely to receive disciplinary referrals. Random assignment to a parenting program reduced disciplinary referrals. This effect was not mediated by skills or preferences.
John A List
Cited by*: None Downloads*: None
Last year I put together a summary of data from my field experiments website that pertained to artefactual field experiments. Several people have asked me if I have and update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper.
John A List
Cited by*: None Downloads*: None
All happy families are alike; each unhappy family is unhappy in its own way.
-Leo Tolstoy, Anna Karenina
Eliot Abrams,
Jonathan Libgober,
John A List
Cited by*: None Downloads*: None
The past few decades have ushered in an experimental revolution in economics whereby scholars are now much more likely to generate their own data. While there are virtues associated with this movement, there are concomitant difficulties. Several scientific disciplines, including economics, have launched research registries in an effort to attenuate key inferential issues. This study assesses registries both empirically and theoretically, with a special focus on the AEA registry. We find that over 90% of randomized controlled trials (RCTs) in economics do not register, only 50% of the RCTs that register do so before the intervention begins, and the majority of these preregistrations are not detailed enough to significantly aid inference. Our empirical analysis further shows that using other scientific registries as aspirational examples is misguided, as their perceived success in tackling the main issues is largely a myth. In light of these facts, we advance a simple economic model to explore potential improvements. A key insight from the model is that removal of the (current) option to register completed RCTs could increase the fraction of trials that register. We also argue that linking IRB applications to registrations could further increase registry effectiveness.
Avner Ben-Ner,
John A List,
Louis Putterman,
Anya Samek
Cited by*: None Downloads*: None
An active area of research within the social sciences concerns the underlying motivation for sharing resources and engaging in other pro-social actions. In this paper we ask: do parents model social preference behavior to children, and do children emulate this behavior? We develop a theoretical framework to examine this question, and conduct an experiment with 147 3 to 5 year old children and their parents, using dictator games to measure generosity. We find (1) evidence of parental teaching/modeling in the case of fathers and in that of parents of relatively generous children, and (2) an emulation effect such that children who initially share less than half of their endowment subsequently share more the more they see a parent or other adult share. We find little correlation between baseline sharing of children and the parents, with the possible exception of the oldest children.
Brian Albrecht,
Omar Al-Ubaydli,
Peter Boettke
Cited by*: None Downloads*: None
Economists well understand that the work of Friedrich Hayek contains important theoretical insights. It is less often acknowledged that his work contains testable predictions about the nature of market processes. Vernon Smith termed the most important one the 'Hayek hypothesis': that gains from trade can be realized in the presence of diffuse, decentralized information, and in the absence of price-taking behavior and centralized market direction. Vernon Smith tested this prediction by surveying data on laboratory experimental markets and found strong support. We extend Smith's work first by showing how subsequent theoretical advances provide a theoretical foundation for the Hayek Hypothesis. We then test the hypothesis using recent field experimental market data. Using field experiments allows us to test several other predictions from Hayek, such as that market experience increases the realized gains from trade. Generally speaking, we find support for Hayek's theories.
Yan Chen,
Peter Cramton,
John A List,
Axel Ockenfels
Cited by*: None Downloads*: None
We review past research and discuss future directions on how the vibrant research areas of market design and behavioral economics have influenced and will continue to impact the science and practice of management in both the private and public sectors. Using examples from various auction markets, reputation and feedback systems in online markets, matching markets in education, and labor markets, we demonstrate that combining market design theory, behavioral insights, and experimental methods can lead to fruitful implementation of superior market designs in practice.
John A List
Cited by*: None Downloads*: None
No abstract available
John A List
Cited by*: None Downloads*: None
All happy families are alike; each unhappy family is unhappy in its own way.
-Leo Tolstoy, Anna Karenina
Eduardo Fe,
David Gill,
Victoria Prowse
Cited by*: None Downloads*: None
We investigate how childhood cognitive skills affect strategic sophistication and adult outcomes. In particular, we emphasize the importance of childhood theory-of-mind as a cognitive skill. We collected experimental data from more than seven hundred children in a variety of strategic interactions. First, we find that theory-of-mind ability and cognitive ability both predict level-k behavior. Second, older children respond to information about the cognitive ability of their opponent, which provides support for the emergence of a sophisticated strategic theory-of-mind. Third, theory-of-mind and age strongly predict whether children respond to intentions in a gift-exchange game, while cognitive ability has no influence, suggesting that different measures of cognitive skill correspond to different cognitive processes in strategic situations that involve understanding intentions. Using the ALSPAC birth-cohort study, we find that childhood theory-of-mind and cognitive ability are both associated with enhanced adult social skills, higher educational participation, better educational attainment, and lower fertility in young adulthood. Finally, we provide evidence that school spending improves theory-of-mind in childhood.
Uri Gneezy,
Moshe Hoffman,
Mark A. Lane,
John A List,
Jeffrey A Livingston,
Michael J. Seiler
Cited by*: None Downloads*: None
Recent theoretical work shows that the better-than-average effect, where a majority
believes their ability to be better than average, can be perfectly consistent with Bayesian updating. However, later experiments that account for this theoretical advance still find behavior consistent with overconfidence. The literature notes that overoptimism can be caused by either overconfidence (optimism about performance), wishful thinking (optimism about outcomes), or both. To test whether the better-than-average effect might be explained by wishful thinking instead of overconfidence, we conduct an experiment that is similar to those used in the overconfidence literature, but removes performance as a potential channel. We find evidence that wishful thinking might explain overconfidence only among the most optimistic subjects, and that conservatism is possibly more of a worry; if unaccounted for, overconfidence might be underestimated.
Snigdha Gupta,
John A List,
Lauren Supplee,
Dana L Suskind
Cited by*: None Downloads*: None
Failed to Scale: Embracing the Challenge of Scaling in Early Childhood
Snigdha Gupta,
Maggie C. Kane,
John A List,
Liz Sablich,
Lauren Supplee,
Dana L Suskind
Cited by*: None Downloads*: None
Recommendations for Mitigating Threats to Scaling
Daniel J Benjamin,
James O Berger,
Magnus Johannesson,
Brian A Nosek,
E. J Wagenmakers,
Richard Berk,
Kenneth A Bollen,
Bjorn Brembs,
Lawrence Brown,
Colin F Camerer,
David Cesarini,
Christopher D. Chambers,
Merlise Clyde,
Thomas D Cook,
Paul De Boeck,
Zoltan Dienes,
Anna Dreber,
Kenny Easwaran,
Charles Efferson,
Ernst Fehr,
Fiona Fidler,
Andy P. Field,
Malcom Forster,
Edward I. George,
Tarun Ramadorai,
Richard Gonzalez,
Steven Goodman,
Edwin Green,
Donald P Green,
Anthony Greenwald,
Jarrod D. Hadfield,
Larry V. Hedges,
Leonhard Held,
Teck Hau Ho,
Herbert Hoijtink,
James Holland Jones,
Daniel J Hruschka,
Kosuke Imai,
Guido Imbens,
John P.A. Ioannidis,
Minjeong Jeon,
Michael Kirchler,
David Laibson ,
John A List,
Roderick Little,
Arthur Lupia,
Edouard Machery,
Scott E. Maxwell,
Michael McCarthy,
Don Moore,
Stephen L. Morgan,
Marcus Munafo,
Shinichi Nakagawa,
Brendan Nyhan,
Timothy H Parker,
Luis Pericchi,
Marco Perugini,
Jeff Rouder,
Judith Rousseau,
Victoria Savalei,
Felix D. Schonbrodt,
Thomas Sellke,
Betsy Sinclair,
Dustin Tingley,
Trisha Van Zandt,
Simine Vazire,
Duncan J. Watts,
Christopher Winship,
Robert L. Wolpert,
Yu Xie,
Cristobal Young,
Jonathan Zinman,
Valen E. Johnson
Cited by*: 1 Downloads*: 965
We propose to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005.
Luigi Butera,
John A List
Cited by*: 1 Downloads*: 355
Novel empirical insights by their very nature tend to be unanticipated, and in some cases at odds with the current state of knowledge on the topic. The mechanics of statistical inference suggest that such initial findings, even when robust and statistically significant within the study, should not appreciably move priors about the phenomenon under investigation. Yet, a few well-conceived independent replications dramatically improve the reliability of novel findings. Nevertheless, the incentives to replicate are seldom in place in the sciences, especially within the social sciences. We propose a simple incentive-compatible mechanism to promote replications, and use experimental economics to highlight our approach. We begin by reporting results from an experiment in which we investigate how cooperation in allocation games is affected by the presence of Knightian uncertainty (ambiguity), a pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find that adding uncertainty enhances cooperation. This surprising result serves as a test case for our mechanism: instead of sending this paper to a peer-reviewed journal, we make it available online as a working paper, but we commit never to submit it to a journal for publication. We instead offered co-authorship for a second, yet to be written, paper to other scholars willing to independently replicate our study. That second paper will reference this working paper, will include all replications, and will be submitted to a peer- reviewed journal for publication. Our mechanism allows mutually-beneficial gains from trade between the original investigators and other scholars, alleviates the publication bias problem that often surrounds novel experimental results, and accelerates the advancement of economic science by leveraging the mechanics of statistical inference.
Glenn W Harrison,
Morten I Lau,
Elisabet E Rutstrom,
Melonie B Williams
Cited by*: 29 Downloads*: 303
We design experiments to jointly elicit risk and time preferences for the adult Danish population. The experimental procedures build on laboratory experiments that have been evaluated using traditional subject pools. The field experiments utilize field sampling designs that we developed, and procedures that were chosen to be relatively transparent in the field with non-standard subject pools. Our overall design was also intended to be a general template for such field experiments in other countries. We examine the characterization of risk over a wider domain for each subject than previous experiments, allowing more precise estimates of risk attitudes. We also examine individual discount rates over six time horizons, as the first stage in a panel experiment in which we revisit subjects to test consistency and stability of responses over time. Risk and time preferences are heterogeneous, varying by observable individual characteristics. On a methodological level, we implement a refinement of existing procedures which elicits much more precise estimates, and also mitigates framing effects.
John A List,
Azeem M Shaikh,
Yang Xu
Cited by*: 33 Downloads*: 278
Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives-multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogenous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate - the probability of one false rejection - and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.
John A List,
Anya Samek,
Dana L Suskind
Cited by*: 0 Downloads*: 258
Behavioral economics and field experiments within the social sciences have advanced well beyond academic curiosum. Governments around the globe as well as the most powerful firms in modern economies employ staffs of behavioralists and experimentalists to advance and test best practices. In this study, we combine behavioral economics with field experiments to reimagine a new model of early childhood education. Our approach has three distinct features. First, by focusing public policy dollars on prevention rather than remediation, we call for much earlier educational programs than currently conceived. Second, our approach has parents at the center of the education production function rather than at its periphery. Third, we advocate attacking the macro education problem using a public health methodology, rather than focusing on piecemeal advances.
Anthony Heyes,
John A List
Cited by*: 2 Downloads*: 157
No abstract available
John A List
Cited by*: 49 Downloads*: 123
Laboratory experiments have been used extensively in economics in the past several decades to lend both positive and normative insights into a myriad of important economic issues. This study discusses a related approach that has increasingly grown in prominence of late--field experiments. I argue that field experiments serve as a useful bridge between data generated in the lab and empirical studies using naturally-occurring data. In discussing this relationship, I highlight that field experiments can yield important insights into economic theory and provide useful guidance to policymakers. I also draw attention to an important methodological contribution of field experiments: they provide an empirical account of behavioral principles that are shared across different domains. In this regard, at odds with conventional wisdom, I argue that representativeness of the environment, rather than representative of the sampled population, is the most crucial variable in determining generalizability of results for a large class of experimental laboratory games.