Search Papers

John A List, Azeem M Shaikh, Yang Xu

Cited by*: 33 Downloads*: 278

Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives-multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogenous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate - the probability of one false rejection - and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.

Ufuk Akcigit, Fernando Alvarez, Stephane Bonhomme, George M Constantinides, Douglas W Diamond, Eugene F Fama, David W Galenson, Michael Greenstone, Lars Peter Hansen, Uhlig Harald, James J Heckman, Ali Hortacsu, Emir Kamenica, Greg Kaplan, Anil K Kashyap, Steven D Levitt, John A List, Robert E Lucas Jr., Magne Mogstad, Roger Myerson, Derek Neal, Canice Prendergast, Raghuram G Rajan, Philip J Reny, Azeem M Shaikh, Robert Shimer, Hugo F Sonnenschein, Nancy L Stokey, Richard H Thaler, Robert H Topel, Robert Vishny, Luigi Zingales

Cited by*: 0 Downloads*: 207

No abstract available

John A List, Azeem M Shaikh, Atom Vayalinkal

Cited by*: None Downloads*: None

List et al. (2019) provides a framework for testing multiple null hypotheses simultaneously using experimental data in which simple random sampling is used to assign treatment status to units. As in List et al. (2019), we rely on general results in Romano and Wolf (2010) to develop under weak assumptions a procedure that (i) asymptotically controls the familywise error rate – the probability of one or more false rejections – and (ii) is asymptotically balanced in that the marginal probability of rejecting any true null hypothesis is approximately equal in large samples. Our analysis departs from List et al. (2019) in that it further exploits observed, baseline covariates. The precise way in which these covariates are incorporated is based upon results in Ye et al. (2022) in order to ensure that inferences are typically more powerful in large samples.