Search

Your search keyword '"Mark Verschoor"' showing total 5 results
Start Over Author "Mark Verschoor" Topic psychology
5 results on '"Mark Verschoor"'

1. Crowdsourcing hypothesis tests:Making transparent how design choices shape research results

Author

Landy, Justin F., Miaolei (Liam) Jia, Isabel, Ding, Domenico, Viganola, Warren, Tierney, Anna, Dreber, Magnus, Johannesson, Thomas, Pfeiffer, Ebersole, Charles R., Gronau, Quentin F., Alexander, Ly, Don van den Bergh, Maarten, Marsman, Eric-Jan, Wagenmakers, Bartels, Daniel M., Bauman, Christopher W., William, Brady, Felix, Cheung, Andrei, Cimpian, Simone, Dohle, Brent Donnellan, M., Adam, Hahn, Michael, Hall, William, Jiménez-Leal, Johnson, David J., Lucas, Richard E., Benoît, Monin, Andres, Montealegre, Elizabeth, Mullen, Jun, Pang, Jennifer, Ray, Reinero, Diego A., Jesse, Reynolds, Walter, Sowden, Daniel, Storage, Runkun, Su, Tworek, Christina M., Van Bavel, Jay J., Daniel, Walco, Julian, Wills, Xiaobing, Xu, Kai Chi Yam, Xiaoyu, Yang, Martin, Schweinsberg, Molly, Urwitz, Matúš, Adamkovič, Ravin, Alaei, Albers, Casper J., Aurélien, Allard, Anderson, Ian A., Andreychik, Michael R., Peter, Babinčák, Baker, Bradley J., Gabriel, Baník, Ernest, Baskin, Jozef, Bavolar, Berkers, Ruud M. W. J., Michał, Białek, Joel, Blanke, Johannes, Breuer, Ambra, Brizi, Brown, Stephanie E. V., Florian, Brühlmann, Hendrik, Bruns, Leigh, Caldwell, Jean-François, Campourcy, Chan, Eugene Y., Yen-Ping, Chang, Cheung, Benjamin Y., Alycia, Chin, Cho, Kit W., Simon, Columbus, Paul, Conway, Corretti, Conrad A., Craig, Adam W., Curran, Paul G., Danvers, Alexander F., Dawson, Ian G. J., Day, Martin V., Erik, Dietl, Doerflinger, Johannes T., Alice, Dominici, Vilius, Dranseika, Edelsbrunner, Peter A., Edlund, John E., Matthew, Fisher, Anna, Fung, Oliver, Genschow, Timo, Gnambs, Goldberg, Matthew H., Lorenz, Graf-Vlachy, Hafenbrack, Andrew C., Sebastian, Hafenbrädl, Andree, Hartanto, Heck, Patrick R., Heffner, Joseph P., Joseph, Hilgard, Felix, Holzmeister, Horchak, Oleksandr V., Huang, Tina S. -T., Joachim, Hüffmeier, Sean, Hughes, Ian, Hussey, Roland, Imhoff, Bastian, Jaeger, Konrad, Jamro, Johnson, Samuel G. B., Andrew, Jones, Lucas, Keller, Olga, Kombeiz, Krueger, Lacy E., Anthony, Lantian, Laplante, Justin P., Lazarevic, Ljiljana B., Jonathan, Leclerc, Nicole, Legate, Leonhardt, James M., Leung, Desmond W., Levitan, Carmel A., Hause, Lin, Qinglan, Liu, Marco Tullio Liuzza, Locke, Kenneth D., Albert L., Ly, Maceacheron, Melanie D., Madan, Christopher R., Harry, Manley, Silvia, Mari, Marcel, Martončik, Mclean, Scott L., Jonathon, Mcphetres, Mercier, Brett G., Corinna, Michels, Mullarkey, Michael C., Musser, Erica D., Ladislas, Nalborczyk, Gustav, Nilsonne, Otis, Nicholas G., Otner, Sarah M. G., Otto, Philipp E., Oscar, Oviedo-Trespalacios, Mariola Paruzel- Czachura, Francesco, Pellegrini, Pereira, Vitor M. D., Hannah, Perfecto, Gerit, Pfuhl, Phillips, Mark H., Ori, Plonsky, Pozzi, Maura, Purić, Danka B., Brett, Raymond-Barker, Redman, David E., Reynolds, Caleb J., Ivan, Ropovik, Lukas, Röseler, Ruessmann, Janna K., Ryan, William H., Nika, Sablaturova, Schuepfer, Kurt J., Astrid, Schütz, Miroslav, Sirota, Matthias, Stefan, Stocks, Eric L., Strosser, Garrett L., Suchow, Jordan W., Anna, Szabelska, Tey, Kian-Siong S., Leonid, Tiokhin, Jais, Troian, Till, Utesch, Alejandro, Vásquez-Echeverría, Leigh Ann Vaughn, Mark, Verschoor, Bettina von Helversen, Pascal, Wallisch, Weissgerber, Sophia C., Wichman, Aaron L., Woike, Jan K., Iris, Žeželj, Zickfeld, Janis H., Yeonsin, Ahn, Blaettchen, Philippe F., Kang, Xi, Yoo Jin Lee, Parker, Philip M., Parker, Paul A., Song, Jamie S., May-Anne, Very, Lynn, Wong, Uhlmann, Eric L., Psychometrics and Statistics, The Crowdsourcing Hypothesis Tests Collaboration [Member of the MPIB: Jan K. Woike], Laboratoire Parisien de Psychologie Sociale (LAPPS), Université Paris Nanterre (UPN)-Université Paris 8 Vincennes-Saint-Denis (UP8), Human Technology Interaction, Psychologische Methodenleer (Psychologie, FMG), Psychology Other Research (FMG), Université Paris 8 Vincennes-Saint-Denis (UP8)-Université Paris Nanterre (UPN), Organizational Psychology, Department of Social Psychology, Landy, J, Jia, M, Ding, I, Viganola, D, Tierney, W, Dreber, A, Johannesson, M, Pfeiffer, T, Ebersole, C, Gronau, Q, Ly, A, van den Bergh, D, Marsman, M, Derks, K, Wagenmakers, E, Proctor, A, Bartels, D, Bauman, C, Brady, W, Cheung, F, Cimpian, A, Dohle, S, Donnellan, M, Hahn, A, Hall, M, Jiménez-Leal, W, Johnson, D, Lucas, R, Monin, B, Montealegre, A, Mullen, E, Pang, J, Ray, J, Reinero, D, Reynolds, J, Sowden, W, Storage, D, Su, R, Tworek, C, Van Bavel, J, Walco, D, Wills, J, Xu, X, Yam, K, Yang, X, Cunningham, W, Schweinsberg, M, Urwitz, M, The Crowdsourcing Hypothesis Tests, C, Uhlmann, E, Mari, S, and Imperial College London

Subjects
Research design, 1ST OFFERS, 1702 Cognitive Sciences, Social Sciences, 050109 social psychology, Ciências Sociais::Psicologia [Domínio/Área Científica], CONCEPTUAL REPLICATIONS, Random Allocation, Empirical research, Crowdsourcing Hypothesis Tests Collaboration, Psychology, research robustness, Psychology(all), General Psychology, ComputingMilieux_MISCELLANEOUS, Marketing, 05 social sciences, SCIENCE, Settore M-PSI/05 - PSICOLOGIA SOCIALE, scientific transparency, Research Design, VDP::Samfunnsvitenskap: 200::Psykologi: 260, [SCCO.PSYC]Cognitive science/Psychology, Crowdsourcing, Cognitive Sciences, crowdsourcing, Cognitive psychology, Adult, replication, Conceptual replications, Social Psychology, Implicit cognition, VDP::Social science: 200::Psychology: 260, Bayesian probability, forecasting, stimulus sampling, INDIVIDUAL-DIFFERENCES, Scientific transparency, Consistency (negotiation), History and Philosophy of Science, Psychology, Multidisciplinary, IMPLICIT, Humans, conceptual replications, crowdsourcing, forecasting, research robustness, scientific transparency, 0501 psychology and cognitive sciences, ATTITUDES, 1505 Marketing, METAANALYSIS, M-PSI/05 - PSICOLOGIA SOCIALE, Statistical hypothesis testing, CONSEQUENCES, business.industry, Crowdsourced testing, SOCIAL-PSYCHOLOGY, M-PSI/03 - PSICOMETRIA, 1701 Psychology, REPLICABILITY, Research robustness, business, Forecasting
Abstract

©American Psychological Association, 2020. This paper is not the copy of record and may not exactly replicate the authoritative document published in the APA journal. Please do not copy or cite without author's permission. The final article is available, upon publication, at: https://doi.apa.org/doi/10.1037/bul0000220 To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = 0.37 to 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.

Published
2020

2. When worry about climate change leads to climate action

Author

Casper J. Albers, Wouter Poortinga, Stephen D. Fisher, Lorraine Whitmarsh, Mark Verschoor, Gisela Böhm, Linda Steg, Thijs Bouman, Environmental Psychology, and Psychometrics and Statistics

Subjects
010504 meteorology & atmospheric sciences, media_common.quotation_subject, Geography, Planning and Development, Climate change, 050109 social psychology, klimabekymring, Management, Monitoring, Policy and Law, Affect (psychology), global warming, 01 natural sciences, klimaendringer, global oppvarming, values, 0501 psychology and cognitive sciences, Moral responsibility, 0105 earth and related environmental sciences, media_common, Global and Planetary Change, European Social Survey, Ecology, Public economics, climate action, 05 social sciences, Global warming, climate worry, personlig ansvar, personal responsibility, climate change, Action (philosophy), Feeling, VDP::Samfunnsvitenskap: 200::Psykologi: 260, Worry, Psychology
Abstract

Dette er den aksepterte versjonen av en artikkel publisert i Global Environmental Change. Du finner den publiserte artikkelen her: https://doi.org/10.1016/j.gloenvcha.2020.102061 // This is the postprint version of the article published in Global Environmental Change. You will find the published article here: https://doi.org/10.1016/j.gloenvcha.2020.102061 The IPCC's report on Global Warming of 1.5°C positioned climate change as one of the most worrying issues mankind has ever faced. Although many people worry about climate change, there is still much unknown about the origins and outcomes of worry about climate change; particularly, whether and how it can motivate specific and personal climate actions. The current paper investigates this critical relationship with data from the European Social Survey Round 8 (44,387 respondents from 23 countries). As expected, the more individuals worried about climate change, the more likely they were to take and support climate action. Yet, the process through which this association occurred differed between actions. Specifically, worry was both directly and indirectly, via feelings of personal responsibility to reduce climate change, associated with climate policy support; whereas worry was mostly indirectly associated with personal climate mitigation behaviours, via personal responsibility. In addition, worry about climate change appears partly rooted in biospheric values (i.e., caring about nature and the environment), and biospheric values were also clearly, directly and positively related to personal climate mitigation behaviours. The relationships were highly consistent across countries but varied somewhat in size. The results show how generic feelings about climate change can directly and indirectly affect both climate policy support and personal climate mitigation behaviours, thereby providing critical insights for science and policy making.

Published
2020

3. Who is more likely to influence others? A value-based approach to pro-environmental social influence behavior (Preprint)

Author

Rebecca J. Sargisson, Mark Verschoor, and Andrea Kis

Subjects
Value (ethics), Negative correlation, Psychology, Positive correlation, Social psychology, Social influence
Abstract

Biospheric values can promote, and egoistic values inhibit, a broad range of pro-environmental behaviors. However, people who strongly endorse egoistic values might undertake pro-environmental behavior involving attempts to influence others. We used a questionnaire to assess the relationship between values and the likelihood that with 193 students will attempt to influence their housemates to engage in pro-environmental behavior. To measure this type of influence behavior, we developed and used the Environmental Social Influence Behavior (E-SIB) questionnaire. Both biospheric and egoistic values promoted influence behaviors. Biospheric values more strongly predicted the likelihood of social-influence actions as egoistic values decreased, except when egoistic values were high. We discuss the connections between values and social-influence behaviors, and current knowledge on the role of egoistic values in environmental actions.

Published
2019

4. The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network

Author

Christopher M. Castille, Zoltan Kekecs, Randy J. McCarthy, Ana María Fernández, Sraddha Pradhan, Sara Álvarez Solas, Hannah Moshontz, Ceylan Okan, Christopher R. Chartier, Armand Chatard, Asil Ali Özdoğru, Babita Pande, Gerit Pfuhl, Bastian Jaeger, Tripat Gill, Martin Voracek, Michelangelo Vianello, Heather L. Urry, Vilius Dranseika, Kathleen Schmidt, Pratibha Kujur, William J. Chopik, Monica A. Koehn, Daniel Storage, S. Mason Garrison, Justin Robert Keene, Gwendolyn Gardiner, Susann Fiedler, Carlota Batres, Jerome Olsen, Sau-Chin Chen, Jon Grahe, Gorka Navarrete, Katherine S. Corker, Diego A. Forero, Kai T. Horstmann, Michael C. Mensink, Cameron Brick, Gwenaël Kaminski, Blair Saunders, David Clarance, Hans IJzerman, Melissa Kline, John Paul Wilson, Patrick S. Forscher, Jack Arnal, Daniel Ansari, Nicholas A. Coles, Noorshama Parveen, Priyanka Chandel, Miguel A. Vadillo, Crystal N. Steltenpohl, Charles R. Ebersole, Lorne Campbell, Stefan Stieger, Burak Aydin, Janis Zickfeld, Erica D. Musser, Miroslav Sirota, Margaret Messiah Singh, Gavin Brent Sullivan, Dana Awlia, Peder M. Isager, Jeremy K. Miller, Evie Vergauwe, Philipp Kanske, Yarrow Dunham, Oscar Oviedo-Trespalacios, Christian K. Tamnes, Martha Lucia Borras-Guevara, Anna Szabelska, Glenn Patrick Williams, John Protzko, Thomas Rhys Evans, Marco Antonio Correa Varella, Steve M. J. Janssen, Nicholas O. Rule, Balazs Aczel, Wolf Vanpaemel, Arti Parganiha, Aycan Kapucu, Jan Antfolk, Nicholas W. Fox, Ivan Ropovik, Barnaby J. W. Dixson, Jessica Kay Flake, Silan Ma, Pavol Kačmár, Lisa M. DeBruine, Hause Lin, Amanda C. Hahn, Benedict C. Jones, Carmel A. Levitan, Ernest Baskin, Vidar Schei, Jaroslava Varella Valentova, Mark Verschoor, Laboratoire Inter-universitaire de Psychologie : Personnalité, Cognition, Changement Social (LIP-PC2S ), Université Savoie Mont Blanc (USMB [Université de Savoie] [Université de Chambéry])-Université Grenoble Alpes [2016-2019] (UGA [2016-2019]), Molecular Genetics, VIB-UA, Department of Medical History and Ethics, Vilnius University [Vilnius], Cognition, Langues, Langage, Ergonomie (CLLE-LTC), École pratique des hautes études (EPHE), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Université Toulouse - Jean Jaurès (UT2J)-Centre National de la Recherche Scientifique (CNRS), Queensland University of Technology [Brisbane] (QUT), Norwegian School of Economics and Business Administration, Universidad de Deusto (DEUSTO), FPSE, Université de Genève (UNIGE), World Bank, Centro de Investigaciones Energéticas Medioambientales y Tecnológicas [Madrid] (CIEMAT), Human Technology Interaction, IJzerman, Hans, Department of Social Psychology, and Organizational Psychology

Subjects
Open science, Collaborative network, VDP::Social science: 200::Psychology: 260, BF, [SHS.PSY]Humanities and Social Sciences/Psychology, 050109 social psychology, Development theory, Crowdsourcing, 050105 experimental psychology, Article, theory development, [SHS.PSY] Humanities and Social Sciences/Psychology, SDG 17 - Partnerships for the Goals, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], PSICOLOGIA BASEADA EM EVIDÊNCIAS, Psychology, 0501 psychology and cognitive sciences, Generalizability theory, generalizability, General Psychology, ComputingMilieux_MISCELLANEOUS, Structure (mathematical logic), [STAT.ME] Statistics [stat]/Methodology [stat.ME], business.industry, Psychological research, 05 social sciences, Neurosciences, Data science, [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Social science: 200::Economics: 210::Economics: 212 [VDP], Transparency (graphic), VDP::Samfunnsvitenskap: 200::Psykologi: 260, large-scale collaboration, Psychological Science Accelerator, crowdsourcing, Social science: 200::Psychology: 260 [VDP], business, [STAT.ME]Statistics [stat]/Methodology [stat.ME]
Abstract

Source at https://doi.org/10.1177/2515245918797607. Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability.

Published
2019

5. Estimating the reproducibility of psychological science

Author

Yoram K. Kunkels, Dylan Selterman, Denise J. Humphries, Kristina G. Brown, David G. Dobolyi, David J. Johnson, Mark A. Roebke, Andy T. Woods, John Hodsoll, Marije van der Hulst, Alexander A. Aarts, Kim Kelso, Erin C. Westgate, James A. Grange, Jesse Chandler, Jenelle Feather, Annick Bosch, Olivia Devitt, Benjamin T. Brown, Megan M. Kyc, Štěpán Bahník, Alissa Melinger, Michael Conn, Rebecca S. Frazier, Marc Jekel, Sara Bowman, Michael J. Wood, Erica Baranski, Sining Wu, Milan Valášek, Anna E. Van't Veer, Jeanine L. M. Skorinko, Joeri Wissink, Sara Steegen, Michael C. Pitts, Douglas Gazarian, Steve N.H. Tsang, Matthew W. Kirkhart, Jennifer S. Beer, Nathali Immelman, Elizabeth Chagnon, Robbie C. M. van Aert, Maya B. Mathur, Magnus Johannesson, Joshua D. Foster, Frank J. Farach, Gandalf Nicolas, Ian S. Penton-Voak, Rebecca M. Goldberg, Sarah L. Thomas, Kathleen Schmidt, Stephanie C. Lin, Linda Cillessen, Belén Fernández-Castilla, Taru Flagan, René Schlegelmilch, Joanneke Weerdmeester, Cyril Pernet, Andreas Cordes, Onur Sahin, Jolanda J. Kossakowski, Samuel Shaki, David Santos, Sabine Scholz, Jeremy R. Gray, Frank Renkewitz, Key Jung Lee, Gillian M. Sandstrom, Marie K. Deserno, Melissa Vazquez, Ed Cremata, Rebecca Saxe, Manuela Thomae, Johannes M. Meixner, Emma Heikensten, Sylvia Eboigbe, Carmel A. Levitan, Natalia Vélez, James G. Field, Riet van Bork, Vivien Estel, Michèle B. Nuijten, Lin Lin, Kate M. Johnson, Bobby Den Bezemer, Jennifer A. Joy-Gaba, Francis Tuerlinckx, Frits Traets, Ilse Luteijn, Christopher R. Chartier, Denise C. Marigold, Denny Borsboom, Elizabeth Gilbert, Jeff Galak, Shannon P. Callahan, E. J. Masicampo, Thomas Talhelm, Chris H.J. Hartgerink, Patrick T. Goodbourn, Stephanie M. Müller, Taylor Nervi, Marcus Möschl, Katherine Moore, Wolf Vanpaemel, Seung K. Kim, Elizabeth Bartmess, Heather N. Mainard, Martin Voracek, Gea Hoogendoorn, Sean P. Mackinnon, Ryan Donohue, Kate A. Ratliff, Jin X. Goh, Anastasia E. Rigney, Andreas Glöckner, Marieke Vermue, Angela S. Attwood, Michelle A. DeGaetano, Nick Spencer, Heather Bentley, Nina Strohminger, Geneva T. Dodson, R. Nathan Pipitone, Hayley M. D. Cleary, Matt Motyl, Amanda L. Forest, Marcus R. Munafò, Marcel Zeelenberg, Susann Fiedler, Ann Calhoun-Sauls, Mallorie Miller, Anondah R. Saide, Ljiljana B. Lazarević, Hilmar Brohmer, Mallory C. Kidwell, Pranjal H. Mehta, Jessie Gorges, Russ Clay, Jeffrey R. Spies, Joanna E. Anderson, Johnny van Doorn, Ashley A. Ricker, Elizabeth W. Dunn, Erin L Braswell, Jamie DeCoster, Larissa Seibel, Matthias Lippold, Lutz Ostkamp, William B. Simpson, Cathy On-Ying Hung, Carina Sonnleitner, Emily M. Wright, Laura Dewitte, Koen Ilja Neijenhuijs, Tim Kuhlmann, Job Krijnen, Leah Beyan, Jesse Graham, Andrew M Rivers, Sacha Epskamp, Aamir Laique, Christopher J. Anderson, Peter Raymond Attridge, Eric-Jan Wagenmakers, Agnieszka Slowik, Michael C. Frank, Bryan Gorges, Alejandro Vásquez Echeverría, Gina Vuu, Giulio Costantini, Eskil Forsell, Michelangelo Vianello, Don van den Bergh, Anna Fedor, Courtney K. Soderberg, M. Brent Donnellan, Kayleigh E Easey, Shauna Gordon-McKeon, Raoul Bell, William J. Johnston, Brian A. Nosek, Ashlee Welsh, Melissa Lewis, Anna Dreber, Simon Columbus, Frank A. Bosco, Pia Tio, Joshua K. Hartshorne, Lars Goellner, Elisa Maria Galliani, Etienne P. Le Bel, Kellylynn Zuni, Olivia Perna, Kristi M. Lemm, Marco Perugini, Anniek M. te Dorsthorst, Hedderik van Rijn, Timothy M. Errington, Bennett Kleinberg, Vanessa C. Irsik, Frank Jäkel, Timothy Hayes, Mark Verschoor, Mark D. Cloud, Bethany Lassetter, Justin Goss, Paul J. Turchan, Gavin Brent Sullivan, Darren Loureiro, Jo Embley, Robert S. Ryan, Jovita Brüning, Jan Crusius, Joel S. Snyder, Larissa Gabrielle Johnson, Nicolás Delia Penna, Grace Binion, Calvin K. Lai, Gustav Nilsonne, Heather M. Fuchs, Angela Rachael Dorrough, Michelle Dugas, Johanna Cohoon, Minha Lee, Robert Krause, David Reinhard, Goran Knežević, Jason M. Prenoveau, Kristin A. Lane, Stanka A. Fitneva, Rima-Maria Rahal, Mathijs Van De Ven, Anup Gampa, Marcel A.L.M. van Assen, Jordan Axt, Felix Henninger, Misha Pavel, Daniel Lakens, Jeremy K. Miller, Sara García, Leslie Cramblet Alvarez, Colleen Osborne, Kai J. Jonas, Taylor Holubar, Stefan Stieger, Heather Barry Kappes, Felix Cheung, Daan R. van Renswoude, Catherine Olsson, Roel van Dooren, Tylar Martinez, Megan Tapia, Philip A. Gable, Cody D. Christopherson, Franziska Plessow, Roger Giner-Sorolla, Abraham M. Rutchick, Michael Barnett-Cowan, Mark J. Brandt, Rebecca A. Dore, Michael May, H. Colleen Sinclair, Georg Jahn, Daniel P. Martin, Fred Hasselman, Casey Eggleston, Nicole Mechin, Joshua J. Matacotta, Molly Babel, Franziska Maria Kolorz, Social & Organizational Psychology, IBBA, Clinical Psychology, EMGO+ - Mental Health, Social Networks, Solidarity and Inequality, Department of Social Psychology, Department of Methodology and Statistics, Aarts, A, Anderson, J, Anderson, C, Attridge, P, Attwood, A, Axt, J, Babel, M, Bahník, Š, Baranski, E, Barnett Cowan, M, Bartmess, E, Beer, J, Bell, R, Bentley, H, Beyan, L, Binion, G, Borsboom, D, Bosch, A, Bosco, F, Bowman, S, Brandt, M, Braswell, E, Brohmer, H, Brown, B, Brown, K, Brüning, J, Calhoun Sauls, A, Callahan, S, Chagnon, E, Chandler, J, Chartier, C, Cheung, C, Cd, Cillessen, L, Clay, R, Cleary, H, Cloud, M, Cohn, M, Cohoon, J, Columbus, S, Cordes, A, Costantini, G, Cramblet Alvarez, L, Cremata, E, Crusius, J, Decoster, J, Degaetano, M, Della Penna, N, den Bezemer, B, Deserno, M, Devitt, O, Dewitte, L, Dobolyi, D, Dodson, G, Donnellan, M, Donohue, R, Dore, R, Dorrough, A, Dreber, A, Dugas, M, Dunn, E, Easey, K, Eboigbe, S, Eggleston, C, Embley, J, Epskamp, S, Errington, T, Estel, V, Farach, F, Feather, J, Fedor, A, Fernández Castilla, B, Fiedler, S, Field, J, Fitneva, S, Flagan, T, Forest, A, Forsell, E, Foster, J, Frank, M, Frazier, R, Fuchs, H, Gable, P, Galak, J, Galliani, E, Gampa, A, Garcia, S, Gazarian, D, Gilbert, E, Giner Sorolla, R, Glöckner, A, Goellner, L, Goh, J, Goldberg, R, Goodbourn, P, Gordon McKeon, S, Gorges, B, Gorges, J, Goss, J, Graham, J, Grange, J, Gray, J, Hartgerink, C, Hartshorne, J, Hasselman, F, Hayes, T, Heikensten, E, Henninger, F, Hodsoll, J, Holubar, T, Hoogendoorn, G, Humphries, D, Hung, C, Immelman, N, Irsik, V, Jahn, G, Jäkel, F, Jekel, M, Johannesson, M, Johnson, L, Johnson, D, Johnson, K, Johnston, W, Jonas, K, Joy Gaba, J, Kappes, H, Kelso, K, Kidwell, M, Kim, S, Kirkhart, M, Kleinberg, B, Kneževic, G, Kolorz, F, Kossakowski, J, Krause, R, Krijnen, J, Kuhlmann, T, Kunkels, Y, Kyc, M, Lai, C, Laique, A, Lakens, D, Lane, K, Lassetter, B, Lazarevic, L, Lebel, E, Lee, K, Lee, M, Lemm, K, Levitan, C, Lewis, M, Lin, L, Lin, S, Lippold, M, Loureiro, D, Luteijn, I, Mackinnon, S, Mainard, H, Marigold, D, Martin, D, Martinez, T, Masicampo, E, Matacotta, J, Mathur, M, May, M, Mechin, N, Mehta, P, Meixner, J, Melinger, A, Miller, J, Miller, M, Moore, K, Möschl, M, Motyl, M, Müller, S, Munafo, M, Neijenhuijs, K, Nervi, T, Nicolas, G, Nilsonne, G, Nosek, B, Nuijten, M, Olsson, C, Osborne, C, Ostkamp, L, Pavel, M, Penton Voak, I, Perna, O, Pernet, C, Perugini, M, Pipitone, N, Pitts, M, Plessow, F, Prenoveau, J, Rahal, R, Ratliff, K, Reinhard, D, Renkewitz, F, Ricker, A, Rigney, A, Rivers, A, Roebke, M, Rutchick, A, Ryan, R, Sahin, O, Saide, A, Sandstrom, G, Santos, D, Saxe, R, Schlegelmilch, R, Schmidt, K, Scholz, S, Seibel, L, Selterman, D, Shaki, S, Simpson, E, Sinclair, H, Skorinko, J, Slowik, A, Snyder, J, Soderberg, C, Sonnleitner, C, Spencer, N, Spies, J, Steegen, S, Stieger, S, Strohminger, N, Sullivan, G, Talhelm, T, Tapia, M, te Dorsthorst, A, Thomae, M, Thomas, S, Tio, P, Traets, F, Tsang, S, Tuerlinckx, F, Turchan, P, Valášek, M, van 't Veer, A, Van Aert, R, van Assen, M, van Bork, R, van de Ven, M, van den Bergh, D, van der Hulst, M, van Dooren, R, van Doorn, J, van Renswoude, D, van Rijn, H, Vanpaemel, W, Vásquez Echeverría, A, Vazquez, M, Velez, N, Vermue, M, Verschoor, M, Vianello, M, Voracek, M, Vuu, G, Wagenmakers, E, Weerdmeester, J, Welsh, A, Westgate, E, Wissink, J, Wood, M, Woods, A, Wright, E, Wu, S, Zeelenberg, M, Zuni, K, Sociology/ICS, Experimental Psychology, Human Technology Interaction, Sociale Psychologie (Psychologie, FMG), Ontwikkelingspsychologie (Psychologie, FMG), and Brein en Cognitie (Psychologie, FMG)

Subjects
Research design, Department Psychologie, BF Psychology, media_common.quotation_subject, POWER, Learning and Plasticity, Reproducibility Project, Q1, Experimental Psychopathology and Treatment, Replication (statistics), Statistics, TRUTH, Psychology, General, Mathematics, media_common, Selection bias, Replication crisis, Behaviour Change and Well-being, Multidisciplinary, PUBLICATION, Publication bias, Reproducibility, Confidence interval, INCENTIVES, PREVALENCE, Meta-analysis, REPLICABILITY, REPLICATION, Developmental Psychopathology, FALSE
Abstract

IntroductionReproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence. Even research of exemplary quality may have irreproducible empirical findings because of random or systematic error.RationaleThere is concern about the rate and predictors of reproducibility, but limited evidence. Potentially problematic practices include selective reporting, selective analysis, and insufficient specification of the conditions necessary or sufficient to obtain the results. Direct replication is the attempt to recreate the conditions believed sufficient for obtaining a previously observed finding and is the means of establishing reproducibility of a finding with new data. We conducted a large-scale, collaborative effort to obtain an initial estimate of the reproducibility of psychological science.ResultsWe conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. There is no single standard for evaluating replication success. Here, we evaluated reproducibility using significance and P values, effect sizes, subjective assessments of replication teams, and meta-analysis of effect sizes. The mean effect size (r) of the replication effects (Mr = 0.197, SD = 0.257) was half the magnitude of the mean effect size of the original effects (Mr = 0.403, SD = 0.188), representing a substantial decline. Ninety-seven percent of original studies had significant results (P < .05). Thirty-six percent of replications had significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.ConclusionNo single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here. Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know.

Published
2015

Catalog

Books, media, physical & digital resources
See catalog results