Papers - Artefactual

Samuel Chang, Andrew Kennedy, Aaron Leonard, John A. List

Cited by*: None Downloads*: None

We provide twelve best practices and discuss how each practice can help researchers accurately, credibly, and ethically use Generative AI (GenAI) to enhance experimental research. We split the twelve practices into four areas. First, in the pre-treatment stage, we discuss how GenAI can aid in pre-registration procedures, data privacy concerns, and ethical considerations specific to GenAI usage. Second, in the design and implementation stage, we focus on GenAI's role in identifying new channels of variation, piloting and documentation, and upholding the four exclusion restrictions. Third, in the analysis stage, we explore how prompting and training set bias can impact results as well as necessary steps to ensure replicability. Finally, we discuss forward-looking best practices that are likely to gain importance as GenAI evolves.

John A List

Cited by*: None Downloads*: None

Last year I put together a summary of data from my field experiments website that pertained to artefactual field experiments. Several people have asked me if I have and update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper.

John A List

Cited by*: None Downloads*: None

In 2019, I put together a summary of data from my field experiments website that pertained to artefactual field experiments. Several people have asked me if I have an update. In this document I update all figures and numbers to show the details for 2021. I also include the description from the 2019 paper below.

John A List

Cited by*: None Downloads*: None

2022 Summary of Artefactual Experiments

Charles Bellemare, Sabine Kroger, Arthur van Soest

Cited by*: 3 Downloads*: 9

We combine the choice data of proposers and responders in the ultimatum game, their expectations elicited in the form of subjective probability questions, and the choice data of proposers (dictators) in a dictator game to estimate a structural model of decision making under uncertainty. We use a large and representative sample of subjects drawn from the Dutch population. Our results indicate that there is considerable heterogeneity in preferences for equity in the population. Changes in preferences have an important impact on decisions of dictators in the dictator game and responders in the ultimatum game, but a smaller impact on decisions of proposers in the ultimatum game, a result due to proposers subjective expectations about responders decisions. The model which uses subjective data on expectations has better predictive power and lower noise level than a model which assumes that players have rational expectations.

Luigi Butera, John A List

Cited by*: 1 Downloads*: 355

Novel empirical insights by their very nature tend to be unanticipated, and in some cases at odds with the current state of knowledge on the topic. The mechanics of statistical inference suggest that such initial findings, even when robust and statistically significant within the study, should not appreciably move priors about the phenomenon under investigation. Yet, a few well-conceived independent replications dramatically improve the reliability of novel findings. Nevertheless, the incentives to replicate are seldom in place in the sciences, especially within the social sciences. We propose a simple incentive-compatible mechanism to promote replications, and use experimental economics to highlight our approach. We begin by reporting results from an experiment in which we investigate how cooperation in allocation games is affected by the presence of Knightian uncertainty (ambiguity), a pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find that adding uncertainty enhances cooperation. This surprising result serves as a test case for our mechanism: instead of sending this paper to a peer-reviewed journal, we make it available online as a working paper, but we commit never to submit it to a journal for publication. We instead offered co-authorship for a second, yet to be written, paper to other scholars willing to independently replicate our study. That second paper will reference this working paper, will include all replications, and will be submitted to a peer- reviewed journal for publication. Our mechanism allows mutually-beneficial gains from trade between the original investigators and other scholars, alleviates the publication bias problem that often surrounds novel experimental results, and accelerates the advancement of economic science by leveraging the mechanics of statistical inference.

Michael J. Seiler, Eric Walden

Cited by*: 0 Downloads*: 0

This study examines strategic mortgage default on a neurological level. Specifically, we test two mainstream behavioral finance/economic theories: sunk cost fallacy and cognitive dissonance. Using fMRI technology, we identify a number of substrates within the brain that provide a neurobiological explanation for why some homeowners exercise their mortgage put option while others do not. We find that borrowers rationally do not suffer from the sunk cost fallacy as it relates to strategic default in that stye significantly prioritize their negative equity position over the amount of their initial down payment. We do, however, find neurological support that cognitive dissonance is relevant in homeowners' thought processes as they toil with the hesitancy brought on by the belief that strategic default is immortal against strong financial incentive to walk away from a substantially underwater mortgage.

Luigi Butera, Philip J Grossman, Daniel Houser, John A List, Marie-Claire Villeval

Cited by*: None Downloads*: None

Creation of empirical knowledge in economics has taken a dramatic turn in the past few decades. One feature of the new research landscape is the nature and extent to which scholars generate data. Today, in nearly every field the experimental approach plays an increasingly crucial role in testing theories and informing organizational decisions. Whereas there is much to appreciate about this revolution, recently a credibility crisis has taken hold across the social sciences, arguing that an important component of Fischer (1935)'s tripod has not been fully embraced: replication. Indeed, while the importance of replications is not debatable scientifically, current incentives are not sufficient to encourage replications from the individual researcher's perspective. We propose a novel mechanism that promotes replications by leveraging mutually beneficial gains between scholars and editors. We develop a model capturing the trade-offs involved in seeking independent replications before submission of a paper to journals. We showcase our method via an investigation of the effects of Knightian uncertainty on cooperation rates in public goods games, a pervasive and yet largely unexplored feature in the literature.

Hansika Kapoor, Savita Kulkarni, Anirudh Tagat

Cited by*: None Downloads*: None

This study aims to investigate intra-household bargaining outcomes elicited in an artefactual field experiment design where participants completed a purchase task of real commodities. Married couples separately expressed their initial preferences over commodities. The bargaining process in the experiment was exogenously introduced by sharing information about partners' preferences in the treatment group. We hypothesized that the spouse with weaker bargaining position at the household level would consider the information of their partner's preferences while making own consumption decisions more compared to their partner. Therefore, they may deviate from their own preferences when purchasing commodities. More than 230 married couples from two villages in the Tamil Nadu state of India participated in the experiment. It was observed that information about partners' spending preferences resulted in reduced final allocation for female participants. However, the deviation was not significantly different from the original intention to spend. Therefore, information about partners' preferences may not be an effective medium to elicit bargaining power in the context of jointly-consumed household commodities. Subgroup analyses were performed to identify any heterogeneous treatment effects.

Inkyoung Hur, Sung-Hee Kim, Anya Samek, Ji Soo Yi

Cited by*: 0 Downloads*: 50

We investigate the effect of different interactive technologies on the decision-making process in an information search laboratory experiment. In our experiment, the participant makes a selection from a list of differently-valued objects with multiple attributes. We compare presenting information in static form to two methods of interactive presentation. In the first, the participant can manually sort objects by attribute, a capability similar to that found in spreadsheet software. In the second, we present an interactive visual tool that (1) automatically sorts all objects by attribute and (2) uses visual cues for comparisons. Manual sorting capability does not cause an improvement in decisions in this context. On the other hand, the visual tool increases the value of the objects selected by the participant and decreases time spent deliberating. We also find that our interactive presentations affect the decision-making process of participants by changing the number of intermediate options considered. Our results highlight the importance of investigating the effect of technology on information search, and suggest that appropriate interactive visual displays may improve search in practice.

Anya Samek

Cited by*: 0 Downloads*: 3

Reputation systems provide decision support for e-commerce. A shortcoming of existing systems is that all transactions are rated equally, and the impact of reputation systems for differently valued goods is not well understood. In an experiment, we study a heterogeneous good market. We find that the reputation system increases surplus by increasing transactions in the high value good. Allowing for heterogeneous goods reduces information, as buyers cannot determine whether the seller previously transacted in low/high value goods. We test a new system, which displays reputation separately for each good. We provide evidence that this additional information is utilized in decisions.

Douglas V DeJong, Robert Forsythe, Wilfred C Uecker

Cited by*: 13 Downloads*: 12

Using the data from sealed offer laboratory markets, we compare the price and quality choices of student subjects with those of businessmen subjects. The businessmen subjects were public accounting firm partners and corporate financial officers. This is of interest since the financial officer-auditor relationship is one particular application of the elementary principal-agent model which the laboratory environment was designed to test. Using several different performance measures we are unable to reject the null hypothesis that the average performance of the two subject pools were the same. However, the market using businessmen subjects generally exhibited greater variance than the market using students.

Cannon Koo, John A List, Michael Margolis, Jason F Shogren

Cited by*: 31 Downloads*: 9

Second-price auctions are designed to induce people to reveal their private preferences for a good. Laboratory evidence suggests that while these auctions do a reasonable job on aggregate, they fall short at the individual level, especially for bidders who are off-margin of the market-clearing price. Herein we introduce and explore whether a random nth-price auction can engage all bidders to bid sincerely. Our results first show that the random nth-price auction can induce sincere bidding in theory and practice. We then compare the random nth-price to the second-price auction. We find that the second-price auction works better on-margin, and the random nth-price auction works better off-margin.

John A List, Charles F Mason

Cited by*: 0 Downloads*: 20

Are individuals expected utility maximizers? This question represents much more than academic curiosity. In a normative sense, at stake are the fundamental underpinnings of the bulk of the last half-century's models of choice under uncertainty. From a positive perspective, the ubiquitous use of benefit-cost analysis across government agencies renders the expected utility maximization paradigm literally the only game in town. In this study, we advance the literature by exploring CEO's preferences over small probability, high loss lotteries. Using undergraduate students as our experimental control group, we find that both our CEO and student subject pools exhibit frequent and large departures from expected utility theory. In addition, as the extreme payoffs become more likely CEOs exhibit greater aversion to risk. Our results suggest that use of the expected utility paradigm in decision making substantially underestimates society's willingness to pay to reduce risk in small probability, high loss events.

Jayson L Lusk, Ted C Schroeder

Cited by*: 125 Downloads*: 80

This study compares hypothetical and nonhypothetical responses to choice experiment questions. We test for hypothetical bias in a choice experiment involving beef ribeye steaks with differing quality attributes. In general, hypothetical responses predicted higher probabilities of purchasing beef steaks than nonhypothetical resposnes. Thus, hypothetical choices overestimate total willingness-to-pay for beef steaks. However, marginal willingness-to-pay for a change in steak quality is, in general, not statistically different across hypothetical and actual payment settings.

Richard O Biel, David N Laband

Cited by*: 17 Downloads*: 71

There is considerable professional disagreement among economists about whether economists are less cooperative than non-economists. It has been argued that once an individual has been schooled in the self-interest model of individual human behavior (s)he exhibits more selfish behavior than other, ostensibly similar individuals who have not been taught to fully appreciate Homo economicus. Heretofore, the empirical debate has centered around classroom experiments designed to compare the "honesty" of undergraduate economics majors versus non economics majors. However, methodological problems have plagued these studies, leaving both sides at an impasse. We offer unique and compelling real-world evidence that suggests economists are no less cooperative than non-economists. Indeed, after comparing the incidence of "cheating" on their Association dues, we find that professional economists are significantly more honest/cooperative than professional political scientists, and especially, professional sociologists.

David J Cooper

Cited by*: 6 Downloads*: 13

This paper studies experiments set in a corporate environment where a manager attempts to overcome a history of coordination failure by employees using either financial incentives or communication. I compare the choices of subject managers drawn from a standard undergraduate population with subject managers drawn from the executive MBA (EMBA) program at Case?s Weatherhead School of Management. The EMBA subjects are a group of experienced, successful managers; all of the EMBA subjects have at least ten years of work experience, including at least five years in a supervisory role, and have average annual earnings in excess of $120,000. The EMBA subject managers are able to overcome a history of coordination failure significantly faster than the undergraduate subject managers. This superior performance is driven neither by differences in the financial incentives offered to the employees nor by use of an inherently different communications strategy. Instead, EMBA subject managers are significantly more likely to use the same "good" communication strategy as is identified for undergraduate subject managers through systematic coding of managers messages to employees.

Joseph Henrich, Richard McElreath

Cited by*: 32 Downloads*: 33

Evidence shows that real-effort investments can affect bilateral bargaining outcomes. This paper investigates whether similar investments can inhibit equilibrium convergence of experimental markets. In one treatment, sellers' relative effort affects the allocation of production costs, but a random productivity shock ensures that the allocation is not necessarily equitable. In another treatment, sellers' effort increases the buyers' valuation of a good. We find that effort investments have a short-lived impact on trading behavior when sellers' effort benefits buyers, but no effect when effort determines cost allocation. Efficiency rates are high and do not differ across treatments.

Andreas Leibbrandt

Cited by*: 0 Downloads*: 5

This paper combines experimental with field data from professional sellers to study whether social preferences are related to performance in natural markets. The data show that sellers who are more pro-social in a laboratory experiment are also more successful in natural markets: they achieve higher prices, have superior trade relations and better abilities to signal trustworthiness to buyers. These findings suggest that social preferences play a significant role for outcomes in natural markets.

Omar Al-Ubaydli, Jason Chien-Yu, John A List

Cited by*: None Downloads*: None

The "voltage effect" is defined as the tendency for a program's efficacy to change when it is scaled up, which in most cases results in the absolute size of a program's treatment effects to diminish when the program is scaled. Understanding the scaling problem and taking steps to diminish voltage drops are important because if left unaddressed, the scaling problem can weaken the public's faith in science, and it can lead to a misallocation of public resources. There exists a growing literature illustrating the prevalence of the scaling problem, explaining its causes, and proposing countermeasures. This paper adds to the literature by providing a simple model of the scaling problem that is consistent with rational expectations by the key stakeholders. Our model highlights that asymmetric information is a key contributor to the voltage effect.