A manifesto for reproducible science
Marcus R. Munafò, Brian A. Nosek, Dorothy V. M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware & John P. A. Ioannidis
Nature Human Behaviour 1, Article number: 0021 (2017)
Download Citation
Social sciences
Published online: 10 January 2017
Figure 1: Threats to reproducible science.
Abstract
Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.
What proportion of published research is likely to be false? Low sample size, small effect sizes, data dredging (also known as P-hacking), conflicts of interest, large numbers of scientists working competitively in silos without combining their efforts, and so on, may conspire to dramatically increase the probability that a published finding is incorrect1. The field of metascience — the scientific study of science itself — is flourishing and has generated substantial empirical evidence for the existence and prevalence of threats to efficiency in knowledge accumulation (refs 2,3,4,5,6,7; Fig. 1).
Acknowledgements
M.R.M. is a member of the UK Centre for Tobacco Control Studies, a UKCRC Public Health Research Centre of Excellence. Funding from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council, Medical Research Council, and the National Institute for Health Research, under the auspices of the UK Clinical Research Collaboration, is gratefully acknowledged. This work was supported by the Medical Research Council Integrative Epidemiology Unit at the University of Bristol (MC_UU_12013/6). D.V.M.B. is funded by a Wellcome Trust Principal Research Fellowship and Programme (grant number 082498/Z/07/Z). N.P.d.S. is employed by the NC3Rs, which is primarily funded by the UK government. J.P.A.I. is funded by an unrestricted gift from S. O'Donnell and B. O'Donnell to the Stanford Prevention Research Center. METRICS is supported by a grant by the Laura and John Arnold Foundation. The authors are grateful to Don van den Bergh for preparing Fig. 2.
Author information
Affiliations
MRC Integrative Epidemiology Unit, University of Bristol, Bristol BS8 2BN, UK.
Marcus R. Munafò
UK Centre for Tobacco and Alcohol Studies, School of Experimental Psychology, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK.
Marcus R. Munafò
Department of Psychology, University of Virginia, Charlottesville, Virginia 22904, USA.
Brian A. Nosek
Center for Open Science, Charlottesville, Virginia 22903, USA.
Brian A. Nosek
Department of Experimental Psychology, University of Oxford, 9 South Parks Road, Oxford OX1 3UD, UK.
Dorothy V. M. Bishop
Department of Psychology, University of Bath, Bath BS2 7AY, UK.
Katherine S. Button
Cardiff University Brain Research Imaging Centre, School of Psychology, Cardiff University, Cardiff CF24 4HQ, UK.
Christopher D. Chambers
National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), London NW1 2BE, UK.
Nathalie Percie du Sert
The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA.
Uri Simonsohn
Department of Psychology, University of Amsterdam, Amsterdam 1018 WT, Netherlands.
Eric-Jan Wagenmakers
CHDI Management/CHDI Foundation, New York, New York 10001, USA.
Jennifer J. Ware
Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford 94304, California, USA.
John P. A. Ioannidis
Stanford Prevention Research Center, Department of Medicine and Department of Health Research and Policy, Stanford University School of Medicine, Stanford 94305, California, USA.
John P. A. Ioannidis
Department of Statistics, Stanford University School of Humanities and Sciences, Stanford 94305, California, USA.
John P. A. Ioannidis
Competing interests
M.R.M, together with C.D.C and D.V.M.B., has received funding from the BBSRC (grant number BB/N019660/1) to convene a workshop on advanced methods for reproducible science, and is chair of the CHDI Foundation Independent Statistical Standing Committee. B.A.N. is executive director of the non-profit Center for Open Science with a mission to increase openness, integrity and reproducibility of research. N.P.d.S. leads the NC3Rs programme of work on experimental design, which developed the ARRIVE guidelines and Experimental Design Assistant. J.J.W. is director , experimental design, at CHDI Management/CHDI Foundation, a non-profit biomedical research organization exclusively dedicated to developing therapeutics for Huntington's disease. The other authors declare no competing interests.
Corresponding author
Correspondence to Marcus R. Munafò.
FREE PDF GRATIS: Nature Human Behavior
