1. Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
- Author
-
Ari Huusko, William J. Sutherland, Tyler D. Eddy, Doriane Stagnol, Deborah A. Buhl, Matt Rinella, Tatsuya Amano, Teppo Vehanen, Thomas R. Stanley, C. Roland Pitcher, David Abecasis, Ricardo S. Ceia, Anjali Pande, Brendan P. Kelaher, Shailesh Sharma, Juan C. Alonso, Adrià López-Baucells, Luciana Cibils-Martina, Heather L. Major, Jill A. Shaffer, Monica Montefalcone, Bradley P. Harris, Jake E. Bicknell, Annelies De Backer, Ian L. Jones, Aki Mäki-Petäys, Juan J. Schmitter-Soto, María C. Ruiz-Delgado, Corinne Watts, Oliver Tully, Norbertas Noreika, Kade Mills, Christoph F. J. Meyer, Just Cebrian, Michele Meroni, Qingyuan Zhao, Rafael Barrientos, Dominique Davoult, Michael D. Craig, Carlos Ponce, Mary K. Donovan, Alec P. Christie, Jonathan P. A. Gardner, Carlos Palacín, Alvaro Antón, Robert A. McConnaughey, Beatriz Martín, Aurora Torres, Daniel Mateos-Molina, Filipe França, Sarah Clarke, Kevin D. E. Stokesbury, Janne S. Kotiaho, Mehdi Adjeroud, Ricardo Rocha, Anna A. Sher, Barry P. Baldigo, Carlos A. Martín, Philip A. Martin, Joachim Claudet, Christie, Alec P [0000-0002-8465-8410], Abecasis, David [0000-0002-9802-8153], Alonso, Juan C [0000-0003-0450-7434], Amano, Tatsuya [0000-0001-6576-3410], Anton, Alvaro [0000-0003-4108-6122], Baldigo, Barry P [0000-0002-9862-9119], Barrientos, Rafael [0000-0002-1677-3214], Bicknell, Jake E [0000-0001-6831-627X], Cebrian, Just [0000-0002-9916-8430], Ceia, Ricardo S [0000-0001-7078-0178], Cibils-Martina, Luciana [0000-0002-2101-4095], Claudet, Joachim [0000-0001-6295-1061], De Backer, Annelies [0000-0001-9129-9009], Donovan, Mary K [0000-0001-6855-0197], França, Filipe M [0000-0003-3827-1917], Gardner, Jonathan PA [0000-0002-6943-2413], Kotiaho, Janne S [0000-0002-4732-784X], López-Baucells, Adrià [0000-0001-8446-0108], Major, Heather L [0000-0002-7265-1289], Mateos-Molina, Daniel [0000-0002-9383-0593], McConnaughey, Robert A [0000-0002-8537-3695], Meyer, Christoph FJ [0000-0001-9958-8913], Noreika, Norbertas [0000-0002-3853-7677], Pitcher, C Roland [0000-0003-2075-4347], Rocha, Ricardo [0000-0003-2757-7347], Schmitter-Soto, Juan J [0000-0003-4736-8382], Shaffer, Jill A [0000-0003-3172-0708], Sharma, Shailesh [0000-0002-7918-4070], Sher, Anna A [0000-0002-6433-9746], Vehanen, Teppo [0000-0003-3441-6787], Apollo - University of Cambridge Repository, Ecologie marine tropicale des océans Pacifique et Indien (ENTROPIE [Perpignan]), Institut de Recherche pour le Développement (IRD), Centre de recherches insulaires et observatoire de l'environnement (CRIOBE), Université de Perpignan Via Domitia (UPVD)-École Pratique des Hautes Études (EPHE), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Centre National de la Recherche Scientifique (CNRS), Université de Perpignan Via Domitia (UPVD)-École pratique des hautes études (EPHE), Christie, Alec P. [0000-0002-8465-8410], Alonso, Juan C. [0000-0003-0450-7434], Baldigo, Barry P. [0000-0002-9862-9119], Bicknell, Jake E. [0000-0001-6831-627X], Ceia, Ricardo S. [0000-0001-7078-0178], Donovan, Mary K. [0000-0001-6855-0197], França, Filipe M. [0000-0003-3827-1917], Gardner, Jonathan P. A. [0000-0002-6943-2413], Kotiaho, Janne S. [0000-0002-4732-784X], Major, Heather L. [0000-0002-7265-1289], McConnaughey, Robert A. [0000-0002-8537-3695], Meyer, Christoph F. J. [0000-0001-9958-8913], Pitcher, C. Roland [0000-0003-2075-4347], Schmitter-Soto, Juan J. [0000-0003-4736-8382], Shaffer, Jill A. [0000-0003-3172-0708], Sher, Anna A. [0000-0002-6433-9746], and Repositório da Universidade de Lisboa
- Subjects
0106 biological sciences ,Research design ,Scientific community ,SCIENTIFIC COMMUNITY ,Medio ambiente natural ,sosiaalitieteet ,Psychological intervention ,General Physics and Astronomy ,Social Sciences ,QH75 ,01 natural sciences ,Environmental impact ,purl.org/becyt/ford/1 [https] ,010104 statistics & probability ,706/648 ,Credibility ,Prevalence ,Social science ,ComputingMilieux_MISCELLANEOUS ,GE ,Multidisciplinary ,Ecology ,article ,Sampling (statistics) ,Biodiversity ,näyttöön perustuvat käytännöt ,satunnaistetut vertailukokeet ,ENVIRONMENTAL IMPACT ,Research Design ,Scale (social sciences) ,[SDE]Environmental Sciences ,H1 ,Science ,Environment ,010603 evolutionary biology ,General Biochemistry, Genetics and Molecular Biology ,Social sciences ,Bias ,tutkimusmenetelmät ,QH541 ,704/172/4081 ,Humans ,0101 mathematics ,purl.org/becyt/ford/1.6 [https] ,ympäristötieteet ,poliittinen päätöksenteko ,Clinical study design ,metodologia ,706/689 ,General Chemistry ,15. Life on land ,Ecología ,Literature ,Pairwise comparison ,Observational study ,631/158 ,luotettavuus - Abstract
Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs., Randomised controlled experiments are the gold standard for scientific inference, but environmental and social scientists often rely on different study designs. Here the authors analyse the use of six common study designs in the fields of biodiversity conservation and social intervention, and quantify the biases in their estimates.
- Published
- 2020
- Full Text
- View/download PDF