Search

Your search keyword '"Mark Verschoor"' showing total 7 results
Start Over Author "Mark Verschoor"
7 results on '"Mark Verschoor"'

1. Exploring relationships between climate change beliefs and energy preferences

Author

Mark Verschoor, Linda Steg, Gisela Böhm, Casper J. Albers, Wouter Poortinga, Environmental Psychology, and Psychometrics and Statistics

Subjects
Social Psychology, Cross-country comparison, PUBLIC PERCEPTIONS, Climate change, 050109 social psychology, 050105 experimental psychology, Energy policy, Social group, CENTRALITY, Energy sources, klimaendringer, Policy acceptability, 0501 psychology and cognitive sciences, Energy supply, Applied Psychology, Visualization, Public economics, 05 social sciences, European social survey, Methodology, Energy security, European Social Survey, Geography, CONTEXT, VDP::Samfunnsvitenskap: 200::Psykologi: 260, Norm (social), Energy source
Abstract

© 2020 Elsevier Ltd. Dette er den aksepterte versjonen av en artikkel publisert i Journal of Environmental Psychology. Du finner den publiserte artikkelen her: https://doi.org/10.1016/j.jenvp.2020.101435. // This is the postprint version of the article published in Journal of Environmental Psychology. You can find the published article here: https://doi.org/10.1016/j.jenvp.2020.101435. Understanding public attitudes to climate change and energy preferences is key to a successful transformation to a low-carbon society. While many studies have examined relationships between specific variables, little is known about the breadth of relationships between multiple climate and energy-relevant concepts. In this paper we used network models to explore and visualize relationships between climate change beliefs and energy preferences, using data from Round 8 of the European Social Survey (ESS8). ESS8 was conducted in 22 European countries and Israel. We found positive relationships between climate change salience, climate change beliefs, climate change concern, personal norm, and personal outcome expectancy, in line with prominent theories within the area. Moreover, beliefs on efficacy of actions of different actors (i.e., governments, large groups of people) to reduce climate change were positively related, and participants had consistent preferences for fossil energy sources or renewable energy sources, respectively. Furthermore, two types of energy security concerns could be distinguished, reflecting temporary and long term threats to energy security, respectively. Energy supply source preferences, energy policy support, and energy conservation behaviors were mostly not uniquely related to the other module variables. Furthermore, the relationships between variables, reflected in the network structure, were comparable across countries.

Published
2020

2. Crowdsourcing hypothesis tests:Making transparent how design choices shape research results

Author

Landy, Justin F., Miaolei (Liam) Jia, Isabel, Ding, Domenico, Viganola, Warren, Tierney, Anna, Dreber, Magnus, Johannesson, Thomas, Pfeiffer, Ebersole, Charles R., Gronau, Quentin F., Alexander, Ly, Don van den Bergh, Maarten, Marsman, Eric-Jan, Wagenmakers, Bartels, Daniel M., Bauman, Christopher W., William, Brady, Felix, Cheung, Andrei, Cimpian, Simone, Dohle, Brent Donnellan, M., Adam, Hahn, Michael, Hall, William, Jiménez-Leal, Johnson, David J., Lucas, Richard E., Benoît, Monin, Andres, Montealegre, Elizabeth, Mullen, Jun, Pang, Jennifer, Ray, Reinero, Diego A., Jesse, Reynolds, Walter, Sowden, Daniel, Storage, Runkun, Su, Tworek, Christina M., Van Bavel, Jay J., Daniel, Walco, Julian, Wills, Xiaobing, Xu, Kai Chi Yam, Xiaoyu, Yang, Martin, Schweinsberg, Molly, Urwitz, Matúš, Adamkovič, Ravin, Alaei, Albers, Casper J., Aurélien, Allard, Anderson, Ian A., Andreychik, Michael R., Peter, Babinčák, Baker, Bradley J., Gabriel, Baník, Ernest, Baskin, Jozef, Bavolar, Berkers, Ruud M. W. J., Michał, Białek, Joel, Blanke, Johannes, Breuer, Ambra, Brizi, Brown, Stephanie E. V., Florian, Brühlmann, Hendrik, Bruns, Leigh, Caldwell, Jean-François, Campourcy, Chan, Eugene Y., Yen-Ping, Chang, Cheung, Benjamin Y., Alycia, Chin, Cho, Kit W., Simon, Columbus, Paul, Conway, Corretti, Conrad A., Craig, Adam W., Curran, Paul G., Danvers, Alexander F., Dawson, Ian G. J., Day, Martin V., Erik, Dietl, Doerflinger, Johannes T., Alice, Dominici, Vilius, Dranseika, Edelsbrunner, Peter A., Edlund, John E., Matthew, Fisher, Anna, Fung, Oliver, Genschow, Timo, Gnambs, Goldberg, Matthew H., Lorenz, Graf-Vlachy, Hafenbrack, Andrew C., Sebastian, Hafenbrädl, Andree, Hartanto, Heck, Patrick R., Heffner, Joseph P., Joseph, Hilgard, Felix, Holzmeister, Horchak, Oleksandr V., Huang, Tina S. -T., Joachim, Hüffmeier, Sean, Hughes, Ian, Hussey, Roland, Imhoff, Bastian, Jaeger, Konrad, Jamro, Johnson, Samuel G. B., Andrew, Jones, Lucas, Keller, Olga, Kombeiz, Krueger, Lacy E., Anthony, Lantian, Laplante, Justin P., Lazarevic, Ljiljana B., Jonathan, Leclerc, Nicole, Legate, Leonhardt, James M., Leung, Desmond W., Levitan, Carmel A., Hause, Lin, Qinglan, Liu, Marco Tullio Liuzza, Locke, Kenneth D., Albert L., Ly, Maceacheron, Melanie D., Madan, Christopher R., Harry, Manley, Silvia, Mari, Marcel, Martončik, Mclean, Scott L., Jonathon, Mcphetres, Mercier, Brett G., Corinna, Michels, Mullarkey, Michael C., Musser, Erica D., Ladislas, Nalborczyk, Gustav, Nilsonne, Otis, Nicholas G., Otner, Sarah M. G., Otto, Philipp E., Oscar, Oviedo-Trespalacios, Mariola Paruzel- Czachura, Francesco, Pellegrini, Pereira, Vitor M. D., Hannah, Perfecto, Gerit, Pfuhl, Phillips, Mark H., Ori, Plonsky, Pozzi, Maura, Purić, Danka B., Brett, Raymond-Barker, Redman, David E., Reynolds, Caleb J., Ivan, Ropovik, Lukas, Röseler, Ruessmann, Janna K., Ryan, William H., Nika, Sablaturova, Schuepfer, Kurt J., Astrid, Schütz, Miroslav, Sirota, Matthias, Stefan, Stocks, Eric L., Strosser, Garrett L., Suchow, Jordan W., Anna, Szabelska, Tey, Kian-Siong S., Leonid, Tiokhin, Jais, Troian, Till, Utesch, Alejandro, Vásquez-Echeverría, Leigh Ann Vaughn, Mark, Verschoor, Bettina von Helversen, Pascal, Wallisch, Weissgerber, Sophia C., Wichman, Aaron L., Woike, Jan K., Iris, Žeželj, Zickfeld, Janis H., Yeonsin, Ahn, Blaettchen, Philippe F., Kang, Xi, Yoo Jin Lee, Parker, Philip M., Parker, Paul A., Song, Jamie S., May-Anne, Very, Lynn, Wong, Uhlmann, Eric L., Psychometrics and Statistics, The Crowdsourcing Hypothesis Tests Collaboration [Member of the MPIB: Jan K. Woike], Laboratoire Parisien de Psychologie Sociale (LAPPS), Université Paris Nanterre (UPN)-Université Paris 8 Vincennes-Saint-Denis (UP8), Human Technology Interaction, Psychologische Methodenleer (Psychologie, FMG), Psychology Other Research (FMG), Université Paris 8 Vincennes-Saint-Denis (UP8)-Université Paris Nanterre (UPN), Organizational Psychology, Department of Social Psychology, Landy, J, Jia, M, Ding, I, Viganola, D, Tierney, W, Dreber, A, Johannesson, M, Pfeiffer, T, Ebersole, C, Gronau, Q, Ly, A, van den Bergh, D, Marsman, M, Derks, K, Wagenmakers, E, Proctor, A, Bartels, D, Bauman, C, Brady, W, Cheung, F, Cimpian, A, Dohle, S, Donnellan, M, Hahn, A, Hall, M, Jiménez-Leal, W, Johnson, D, Lucas, R, Monin, B, Montealegre, A, Mullen, E, Pang, J, Ray, J, Reinero, D, Reynolds, J, Sowden, W, Storage, D, Su, R, Tworek, C, Van Bavel, J, Walco, D, Wills, J, Xu, X, Yam, K, Yang, X, Cunningham, W, Schweinsberg, M, Urwitz, M, The Crowdsourcing Hypothesis Tests, C, Uhlmann, E, Mari, S, and Imperial College London

Subjects
Research design, 1ST OFFERS, 1702 Cognitive Sciences, Social Sciences, 050109 social psychology, Ciências Sociais::Psicologia [Domínio/Área Científica], CONCEPTUAL REPLICATIONS, Random Allocation, Empirical research, Crowdsourcing Hypothesis Tests Collaboration, Psychology, research robustness, Psychology(all), General Psychology, ComputingMilieux_MISCELLANEOUS, Marketing, 05 social sciences, SCIENCE, Settore M-PSI/05 - PSICOLOGIA SOCIALE, scientific transparency, Research Design, VDP::Samfunnsvitenskap: 200::Psykologi: 260, [SCCO.PSYC]Cognitive science/Psychology, Crowdsourcing, Cognitive Sciences, crowdsourcing, Cognitive psychology, Adult, replication, Conceptual replications, Social Psychology, Implicit cognition, VDP::Social science: 200::Psychology: 260, Bayesian probability, forecasting, stimulus sampling, INDIVIDUAL-DIFFERENCES, Scientific transparency, Consistency (negotiation), History and Philosophy of Science, Psychology, Multidisciplinary, IMPLICIT, Humans, conceptual replications, crowdsourcing, forecasting, research robustness, scientific transparency, 0501 psychology and cognitive sciences, ATTITUDES, 1505 Marketing, METAANALYSIS, M-PSI/05 - PSICOLOGIA SOCIALE, Statistical hypothesis testing, CONSEQUENCES, business.industry, Crowdsourced testing, SOCIAL-PSYCHOLOGY, M-PSI/03 - PSICOMETRIA, 1701 Psychology, REPLICABILITY, Research robustness, business, Forecasting
Abstract

©American Psychological Association, 2020. This paper is not the copy of record and may not exactly replicate the authoritative document published in the APA journal. Please do not copy or cite without author's permission. The final article is available, upon publication, at: https://doi.apa.org/doi/10.1037/bul0000220 To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = 0.37 to 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.

Published
2020

3. When worry about climate change leads to climate action

Author

Casper J. Albers, Wouter Poortinga, Stephen D. Fisher, Lorraine Whitmarsh, Mark Verschoor, Gisela Böhm, Linda Steg, Thijs Bouman, Environmental Psychology, and Psychometrics and Statistics

Subjects
010504 meteorology & atmospheric sciences, media_common.quotation_subject, Geography, Planning and Development, Climate change, 050109 social psychology, klimabekymring, Management, Monitoring, Policy and Law, Affect (psychology), global warming, 01 natural sciences, klimaendringer, global oppvarming, values, 0501 psychology and cognitive sciences, Moral responsibility, 0105 earth and related environmental sciences, media_common, Global and Planetary Change, European Social Survey, Ecology, Public economics, climate action, 05 social sciences, Global warming, climate worry, personlig ansvar, personal responsibility, climate change, Action (philosophy), Feeling, VDP::Samfunnsvitenskap: 200::Psykologi: 260, Worry, Psychology
Abstract

Dette er den aksepterte versjonen av en artikkel publisert i Global Environmental Change. Du finner den publiserte artikkelen her: https://doi.org/10.1016/j.gloenvcha.2020.102061 // This is the postprint version of the article published in Global Environmental Change. You will find the published article here: https://doi.org/10.1016/j.gloenvcha.2020.102061 The IPCC's report on Global Warming of 1.5°C positioned climate change as one of the most worrying issues mankind has ever faced. Although many people worry about climate change, there is still much unknown about the origins and outcomes of worry about climate change; particularly, whether and how it can motivate specific and personal climate actions. The current paper investigates this critical relationship with data from the European Social Survey Round 8 (44,387 respondents from 23 countries). As expected, the more individuals worried about climate change, the more likely they were to take and support climate action. Yet, the process through which this association occurred differed between actions. Specifically, worry was both directly and indirectly, via feelings of personal responsibility to reduce climate change, associated with climate policy support; whereas worry was mostly indirectly associated with personal climate mitigation behaviours, via personal responsibility. In addition, worry about climate change appears partly rooted in biospheric values (i.e., caring about nature and the environment), and biospheric values were also clearly, directly and positively related to personal climate mitigation behaviours. The relationships were highly consistent across countries but varied somewhat in size. The results show how generic feelings about climate change can directly and indirectly affect both climate policy support and personal climate mitigation behaviours, thereby providing critical insights for science and policy making.

Published
2020

4. Who is more likely to influence others? A value-based approach to pro-environmental social influence behavior (Preprint)

Author

Rebecca J. Sargisson, Mark Verschoor, and Andrea Kis

Subjects
Value (ethics), Negative correlation, Psychology, Positive correlation, Social psychology, Social influence
Abstract

Biospheric values can promote, and egoistic values inhibit, a broad range of pro-environmental behaviors. However, people who strongly endorse egoistic values might undertake pro-environmental behavior involving attempts to influence others. We used a questionnaire to assess the relationship between values and the likelihood that with 193 students will attempt to influence their housemates to engage in pro-environmental behavior. To measure this type of influence behavior, we developed and used the Environmental Social Influence Behavior (E-SIB) questionnaire. Both biospheric and egoistic values promoted influence behaviors. Biospheric values more strongly predicted the likelihood of social-influence actions as egoistic values decreased, except when egoistic values were high. We discuss the connections between values and social-influence behaviors, and current knowledge on the role of egoistic values in environmental actions.

Published
2019

5. The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network

Author

Christopher M. Castille, Zoltan Kekecs, Randy J. McCarthy, Ana María Fernández, Sraddha Pradhan, Sara Álvarez Solas, Hannah Moshontz, Ceylan Okan, Christopher R. Chartier, Armand Chatard, Asil Ali Özdoğru, Babita Pande, Gerit Pfuhl, Bastian Jaeger, Tripat Gill, Martin Voracek, Michelangelo Vianello, Heather L. Urry, Vilius Dranseika, Kathleen Schmidt, Pratibha Kujur, William J. Chopik, Monica A. Koehn, Daniel Storage, S. Mason Garrison, Justin Robert Keene, Gwendolyn Gardiner, Susann Fiedler, Carlota Batres, Jerome Olsen, Sau-Chin Chen, Jon Grahe, Gorka Navarrete, Katherine S. Corker, Diego A. Forero, Kai T. Horstmann, Michael C. Mensink, Cameron Brick, Gwenaël Kaminski, Blair Saunders, David Clarance, Hans IJzerman, Melissa Kline, John Paul Wilson, Patrick S. Forscher, Jack Arnal, Daniel Ansari, Nicholas A. Coles, Noorshama Parveen, Priyanka Chandel, Miguel A. Vadillo, Crystal N. Steltenpohl, Charles R. Ebersole, Lorne Campbell, Stefan Stieger, Burak Aydin, Janis Zickfeld, Erica D. Musser, Miroslav Sirota, Margaret Messiah Singh, Gavin Brent Sullivan, Dana Awlia, Peder M. Isager, Jeremy K. Miller, Evie Vergauwe, Philipp Kanske, Yarrow Dunham, Oscar Oviedo-Trespalacios, Christian K. Tamnes, Martha Lucia Borras-Guevara, Anna Szabelska, Glenn Patrick Williams, John Protzko, Thomas Rhys Evans, Marco Antonio Correa Varella, Steve M. J. Janssen, Nicholas O. Rule, Balazs Aczel, Wolf Vanpaemel, Arti Parganiha, Aycan Kapucu, Jan Antfolk, Nicholas W. Fox, Ivan Ropovik, Barnaby J. W. Dixson, Jessica Kay Flake, Silan Ma, Pavol Kačmár, Lisa M. DeBruine, Hause Lin, Amanda C. Hahn, Benedict C. Jones, Carmel A. Levitan, Ernest Baskin, Vidar Schei, Jaroslava Varella Valentova, Mark Verschoor, Laboratoire Inter-universitaire de Psychologie : Personnalité, Cognition, Changement Social (LIP-PC2S ), Université Savoie Mont Blanc (USMB [Université de Savoie] [Université de Chambéry])-Université Grenoble Alpes [2016-2019] (UGA [2016-2019]), Molecular Genetics, VIB-UA, Department of Medical History and Ethics, Vilnius University [Vilnius], Cognition, Langues, Langage, Ergonomie (CLLE-LTC), École pratique des hautes études (EPHE), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Université Toulouse - Jean Jaurès (UT2J)-Centre National de la Recherche Scientifique (CNRS), Queensland University of Technology [Brisbane] (QUT), Norwegian School of Economics and Business Administration, Universidad de Deusto (DEUSTO), FPSE, Université de Genève (UNIGE), World Bank, Centro de Investigaciones Energéticas Medioambientales y Tecnológicas [Madrid] (CIEMAT), Human Technology Interaction, IJzerman, Hans, Department of Social Psychology, and Organizational Psychology

Subjects
Open science, Collaborative network, VDP::Social science: 200::Psychology: 260, BF, [SHS.PSY]Humanities and Social Sciences/Psychology, 050109 social psychology, Development theory, Crowdsourcing, 050105 experimental psychology, Article, theory development, [SHS.PSY] Humanities and Social Sciences/Psychology, SDG 17 - Partnerships for the Goals, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], PSICOLOGIA BASEADA EM EVIDÊNCIAS, Psychology, 0501 psychology and cognitive sciences, Generalizability theory, generalizability, General Psychology, ComputingMilieux_MISCELLANEOUS, Structure (mathematical logic), [STAT.ME] Statistics [stat]/Methodology [stat.ME], business.industry, Psychological research, 05 social sciences, Neurosciences, Data science, [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Social science: 200::Economics: 210::Economics: 212 [VDP], Transparency (graphic), VDP::Samfunnsvitenskap: 200::Psykologi: 260, large-scale collaboration, Psychological Science Accelerator, crowdsourcing, Social science: 200::Psychology: 260 [VDP], business, [STAT.ME]Statistics [stat]/Methodology [stat.ME]
Abstract

Source at https://doi.org/10.1177/2515245918797607. Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability.

Published
2019

6. Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

Author

Jesse Chandler, Charles R. Ebersole, Marek A. Vranka, Sharon Coen, David Sirlopú, David Torres, Eva E. Chen, Joshua C. Gandi, Anthony J. Nelson, Robbie C. M. van Aert, Eva G. T. Green, Jeroen Stouten, Elsie Ong, Cheryl Alyssa Welch, Anja Eller, Jen-Ho Chang, Catherine Verniers, Liane Young, Canay Doğulu, Joshua M. Tybur, Jon Grahe, Michelangelo Vianello, German Kurapov, Ingrid P. J. Voermans, Maaike J. de Bruijn, Karin C.H.J. Smolders, William E. Davis, Felix D. Schönbrodt, Stephanie Szeto, Katherine S. Corker, Paul G. Curran, Fernando Mena-pacheco, Julie A. Woodzicka, Leander De Schutter, Ángel Gómez, Elisa Maria Galliani, Mike Friedman, Huajian Cai, Agata Sobkow, Joy E. Losee, Kevin Durrheim, Manuela Thomae, Jennifer A. Coleman, Tanuka Ghoshal, Serdar Karabati, Vanessa Smith-Castro, Konrad Bocian, Hans IJzerman, Mark Aveyard, Janko Međedović, Marta Wrońska, Mark Verschoor, Ho Phi Huynh, Félix Neto, Joshua A. Hicks, Alexandra Vázquez, John E. Edlund, Anna van 't Veer, Zubairu K. Dagona, Thierry Devos, Oskar K. Sundfelt, Philipp Spachtholz, Tripat Gill, Maciej Sekerdej, Jennifer A. Joy-Gaba, Masanori Oikawa, Aaron Ocampo, Sean C. Murphy, Haruna Karick, Kathleen Schmidt, Goran Knežević, Ivan Grahek, Nicolas Kervyn, Grant Packard, Fred Hasselman, Byron G. Adams, Satia A. Marotta, Aaron L. Wichman, Olga Bialobrzeska, Manini Srivastava, Mark J. Brandt, Nikolette P. Lipsey, Carmel A. Levitan, Ewa Szumowska, Gábor Orosz, Michael J. Bernstein, Morgan Conway, Timo Gnambs, Koen Ilja Neijenhuijs, John M. Zelenski, Lysandra Podesta, Esther Maassen, Sinan Alper, Ilker Dalgar, Chris N. H. Street, Alexander K. Saeri, Anna Dalla Rosa, Reginald B. Adams, Ljiljana B. Lazarević, Marcel A.L.M. van Assen, Brian Collisson, Mayowa T. Babalola, Daniel Lakens, Cheryl L. Carmichael, Carrie Kovacs, Yarrow Dunham, Yoel Inbar, Patricio Saavedra, Natalia Frankowska, Austin Lee Nichols, Evans Dami Binan, Samuel Lincoln Bezerra Lins, Marije van der Hulst, Jamie Kurtz, Morgan J. Tear, Brad Pinter, Jeffrey R. Huntsinger, Åse Innes-Ker, Steffen R. Giessner, Kakul Hai, Boban Petrović, Elizabeth L. Haines, Francisco Ceric, Andriy Myachykov, Wendy L. Morris, Jordan Axt, Marieke de Vries, Lacy E. Krueger, Erika Salomon, Heather Barry Kappes, Anna Cabak Rédei, Armand Chatard, Matthew Haigh, Haruka Oikawa, Nick Neave, Monique Pollmann, Daniel R. Berry, Abraham M. Rutchick, Fiery Cushman, David C. Cicero, Brian A. Nosek, Ronaldo Pilati, Zeng Zhijia, Jakub Traczyk, Mihály Berkics, Michael P. Hall, Luis Diego Vega, Nerisa Dozo, Roberto González, Paul A. M. Van Lange, William Jiménez-Leal, Leigh Ann Vaughn, Winnee Cheong, Rolando Pérez-Sánchez, Rishtee Batra, Michael A. Smith, Andrew C. W. Tang, Štěpán Bahník, Małgorzata Osowiecka, Gabrielle Pogge, Lisa A. Williams, Alexander S. English, Robert Busching, Melissa-Sue John, Taciano L. Milfont, Fanny Cambier, Alejandro Vásquez Echeverría, Carolyn Finck, Richard A. Klein, Anna Kende, Norbert K. Tanzer, Victor N. Keller, Marie E. Heffernan, Neil A. Lewis, Jordan Theriault, Walter Sowden, Troy G. Steiner, Kristin Nicole Dukes, Adrienn Ujhelyi, Susan L. O'Donnell, Angela T. Maitner, Michael Wood, Jesse Graham, Winfrida Malingumu, Petr Houdek, Katarzyna Cantarero, Miguel-Ángel Freyre, Jeanine L. M. Skorinko, Roza G. Kamiloğlu, Robyn K. Mallett, Sociale Psychologie (Psychologie, FMG), Amsterdam Interdisciplinary Centre for Emotion (AICE, Psychology, FMG), Laboratoire Inter-universitaire de Psychologie : Personnalité, Cognition, Changement Social (LIP-PC2S ), Université Savoie Mont Blanc (USMB [Université de Savoie] [Université de Chambéry])-Université Grenoble Alpes [2016-2019] (UGA [2016-2019]), Florida International University [Miami] (FIU), Massachusetts Institute of Technology (MIT), Universidad Nacional de Educación a Distancia (UNED), EDF (EDF), Biozentrum [Basel, Suisse], University of Basel (Unibas), Tilburg University [Netherlands], Performance Engineering Laboratory [Dublin] (PEL), Dublin City University [Dublin] (DCU), Department of Rheumatology, Ghent University Hospital, Laboratoire de Psychologie Sociale et Cognitive (LAPSCO), Université Clermont Auvergne [2017-2020] (UCA [2017-2020])-Centre National de la Recherche Scientifique (CNRS), School of Information Technology [Kharagpur], Indian Institute of Technology Kharagpur (IIT Kharagpur), Human Technology Interaction, UCL - SSH/LouRIM - Louvain Research Institute in Management and Organizations, Tilburg University, Social Psychology, IBBA, A-LAB, Faculdade de Psicologia e de Ciências da Educação, Department of Social Psychology, Department of Methodology and Statistics, Department Communication and Cognition, Language, Communication and Cognition, Department of Business-Society Management, Department of Organisation and Personnel Management, Erasmus MC other, Research Methods and Techniques, and IJzerman, Hans

Subjects
Sampling effects, H Social Sciences (General), Culture, Open materials, Learning and Plasticity, [SHS.PSY]Humanities and Social Sciences/Psychology, Replication, Situational effects, 050109 social psychology, Context (language use), Social psychology, Preregistered, 050105 experimental psychology, [SHS.PSY] Humanities and Social Sciences/Psychology, social psychology, cognitive psychology, replication, culture, individual differences, sampling effects, situational effects, meta-analysis, Registered Report, open data, open materials, preregistered, [STAT.ML]Statistics [stat]/Machine Learning [stat.ML], Statistical significance, Cognitive psychology, Statistics, Replication (statistics), Medicine and Health Sciences, 0501 psychology and cognitive sciences, General Psychology, ComputingMilieux_MISCELLANEOUS, [STAT.ME] Statistics [stat]/Methodology [stat.ME], Behaviour Change and Well-being, Data Science, 05 social sciences, Open data, [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], C800, Meta-analysis, Registered Report, Individual differences, Developmental Psychopathology, [STAT.ME]Statistics [stat]/Methodology [stat.ME]
Abstract

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely highpowered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied. UCR::Vicerrectoría de Investigación::Unidades de Investigación::Ciencias Sociales::Instituto de Investigaciones Psicológicas (IIP)

Published
2018

7. Estimating the reproducibility of psychological science

Author

Yoram K. Kunkels, Dylan Selterman, Denise J. Humphries, Kristina G. Brown, David G. Dobolyi, David J. Johnson, Mark A. Roebke, Andy T. Woods, John Hodsoll, Marije van der Hulst, Alexander A. Aarts, Kim Kelso, Erin C. Westgate, James A. Grange, Jesse Chandler, Jenelle Feather, Annick Bosch, Olivia Devitt, Benjamin T. Brown, Megan M. Kyc, Štěpán Bahník, Alissa Melinger, Michael Conn, Rebecca S. Frazier, Marc Jekel, Sara Bowman, Michael J. Wood, Erica Baranski, Sining Wu, Milan Valášek, Anna E. Van't Veer, Jeanine L. M. Skorinko, Joeri Wissink, Sara Steegen, Michael C. Pitts, Douglas Gazarian, Steve N.H. Tsang, Matthew W. Kirkhart, Jennifer S. Beer, Nathali Immelman, Elizabeth Chagnon, Robbie C. M. van Aert, Maya B. Mathur, Magnus Johannesson, Joshua D. Foster, Frank J. Farach, Gandalf Nicolas, Ian S. Penton-Voak, Rebecca M. Goldberg, Sarah L. Thomas, Kathleen Schmidt, Stephanie C. Lin, Linda Cillessen, Belén Fernández-Castilla, Taru Flagan, René Schlegelmilch, Joanneke Weerdmeester, Cyril Pernet, Andreas Cordes, Onur Sahin, Jolanda J. Kossakowski, Samuel Shaki, David Santos, Sabine Scholz, Jeremy R. Gray, Frank Renkewitz, Key Jung Lee, Gillian M. Sandstrom, Marie K. Deserno, Melissa Vazquez, Ed Cremata, Rebecca Saxe, Manuela Thomae, Johannes M. Meixner, Emma Heikensten, Sylvia Eboigbe, Carmel A. Levitan, Natalia Vélez, James G. Field, Riet van Bork, Vivien Estel, Michèle B. Nuijten, Lin Lin, Kate M. Johnson, Bobby Den Bezemer, Jennifer A. Joy-Gaba, Francis Tuerlinckx, Frits Traets, Ilse Luteijn, Christopher R. Chartier, Denise C. Marigold, Denny Borsboom, Elizabeth Gilbert, Jeff Galak, Shannon P. Callahan, E. J. Masicampo, Thomas Talhelm, Chris H.J. Hartgerink, Patrick T. Goodbourn, Stephanie M. Müller, Taylor Nervi, Marcus Möschl, Katherine Moore, Wolf Vanpaemel, Seung K. Kim, Elizabeth Bartmess, Heather N. Mainard, Martin Voracek, Gea Hoogendoorn, Sean P. Mackinnon, Ryan Donohue, Kate A. Ratliff, Jin X. Goh, Anastasia E. Rigney, Andreas Glöckner, Marieke Vermue, Angela S. Attwood, Michelle A. DeGaetano, Nick Spencer, Heather Bentley, Nina Strohminger, Geneva T. Dodson, R. Nathan Pipitone, Hayley M. D. Cleary, Matt Motyl, Amanda L. Forest, Marcus R. Munafò, Marcel Zeelenberg, Susann Fiedler, Ann Calhoun-Sauls, Mallorie Miller, Anondah R. Saide, Ljiljana B. Lazarević, Hilmar Brohmer, Mallory C. Kidwell, Pranjal H. Mehta, Jessie Gorges, Russ Clay, Jeffrey R. Spies, Joanna E. Anderson, Johnny van Doorn, Ashley A. Ricker, Elizabeth W. Dunn, Erin L Braswell, Jamie DeCoster, Larissa Seibel, Matthias Lippold, Lutz Ostkamp, William B. Simpson, Cathy On-Ying Hung, Carina Sonnleitner, Emily M. Wright, Laura Dewitte, Koen Ilja Neijenhuijs, Tim Kuhlmann, Job Krijnen, Leah Beyan, Jesse Graham, Andrew M Rivers, Sacha Epskamp, Aamir Laique, Christopher J. Anderson, Peter Raymond Attridge, Eric-Jan Wagenmakers, Agnieszka Slowik, Michael C. Frank, Bryan Gorges, Alejandro Vásquez Echeverría, Gina Vuu, Giulio Costantini, Eskil Forsell, Michelangelo Vianello, Don van den Bergh, Anna Fedor, Courtney K. Soderberg, M. Brent Donnellan, Kayleigh E Easey, Shauna Gordon-McKeon, Raoul Bell, William J. Johnston, Brian A. Nosek, Ashlee Welsh, Melissa Lewis, Anna Dreber, Simon Columbus, Frank A. Bosco, Pia Tio, Joshua K. Hartshorne, Lars Goellner, Elisa Maria Galliani, Etienne P. Le Bel, Kellylynn Zuni, Olivia Perna, Kristi M. Lemm, Marco Perugini, Anniek M. te Dorsthorst, Hedderik van Rijn, Timothy M. Errington, Bennett Kleinberg, Vanessa C. Irsik, Frank Jäkel, Timothy Hayes, Mark Verschoor, Mark D. Cloud, Bethany Lassetter, Justin Goss, Paul J. Turchan, Gavin Brent Sullivan, Darren Loureiro, Jo Embley, Robert S. Ryan, Jovita Brüning, Jan Crusius, Joel S. Snyder, Larissa Gabrielle Johnson, Nicolás Delia Penna, Grace Binion, Calvin K. Lai, Gustav Nilsonne, Heather M. Fuchs, Angela Rachael Dorrough, Michelle Dugas, Johanna Cohoon, Minha Lee, Robert Krause, David Reinhard, Goran Knežević, Jason M. Prenoveau, Kristin A. Lane, Stanka A. Fitneva, Rima-Maria Rahal, Mathijs Van De Ven, Anup Gampa, Marcel A.L.M. van Assen, Jordan Axt, Felix Henninger, Misha Pavel, Daniel Lakens, Jeremy K. Miller, Sara García, Leslie Cramblet Alvarez, Colleen Osborne, Kai J. Jonas, Taylor Holubar, Stefan Stieger, Heather Barry Kappes, Felix Cheung, Daan R. van Renswoude, Catherine Olsson, Roel van Dooren, Tylar Martinez, Megan Tapia, Philip A. Gable, Cody D. Christopherson, Franziska Plessow, Roger Giner-Sorolla, Abraham M. Rutchick, Michael Barnett-Cowan, Mark J. Brandt, Rebecca A. Dore, Michael May, H. Colleen Sinclair, Georg Jahn, Daniel P. Martin, Fred Hasselman, Casey Eggleston, Nicole Mechin, Joshua J. Matacotta, Molly Babel, Franziska Maria Kolorz, Social & Organizational Psychology, IBBA, Clinical Psychology, EMGO+ - Mental Health, Social Networks, Solidarity and Inequality, Department of Social Psychology, Department of Methodology and Statistics, Aarts, A, Anderson, J, Anderson, C, Attridge, P, Attwood, A, Axt, J, Babel, M, Bahník, Š, Baranski, E, Barnett Cowan, M, Bartmess, E, Beer, J, Bell, R, Bentley, H, Beyan, L, Binion, G, Borsboom, D, Bosch, A, Bosco, F, Bowman, S, Brandt, M, Braswell, E, Brohmer, H, Brown, B, Brown, K, Brüning, J, Calhoun Sauls, A, Callahan, S, Chagnon, E, Chandler, J, Chartier, C, Cheung, C, Cd, Cillessen, L, Clay, R, Cleary, H, Cloud, M, Cohn, M, Cohoon, J, Columbus, S, Cordes, A, Costantini, G, Cramblet Alvarez, L, Cremata, E, Crusius, J, Decoster, J, Degaetano, M, Della Penna, N, den Bezemer, B, Deserno, M, Devitt, O, Dewitte, L, Dobolyi, D, Dodson, G, Donnellan, M, Donohue, R, Dore, R, Dorrough, A, Dreber, A, Dugas, M, Dunn, E, Easey, K, Eboigbe, S, Eggleston, C, Embley, J, Epskamp, S, Errington, T, Estel, V, Farach, F, Feather, J, Fedor, A, Fernández Castilla, B, Fiedler, S, Field, J, Fitneva, S, Flagan, T, Forest, A, Forsell, E, Foster, J, Frank, M, Frazier, R, Fuchs, H, Gable, P, Galak, J, Galliani, E, Gampa, A, Garcia, S, Gazarian, D, Gilbert, E, Giner Sorolla, R, Glöckner, A, Goellner, L, Goh, J, Goldberg, R, Goodbourn, P, Gordon McKeon, S, Gorges, B, Gorges, J, Goss, J, Graham, J, Grange, J, Gray, J, Hartgerink, C, Hartshorne, J, Hasselman, F, Hayes, T, Heikensten, E, Henninger, F, Hodsoll, J, Holubar, T, Hoogendoorn, G, Humphries, D, Hung, C, Immelman, N, Irsik, V, Jahn, G, Jäkel, F, Jekel, M, Johannesson, M, Johnson, L, Johnson, D, Johnson, K, Johnston, W, Jonas, K, Joy Gaba, J, Kappes, H, Kelso, K, Kidwell, M, Kim, S, Kirkhart, M, Kleinberg, B, Kneževic, G, Kolorz, F, Kossakowski, J, Krause, R, Krijnen, J, Kuhlmann, T, Kunkels, Y, Kyc, M, Lai, C, Laique, A, Lakens, D, Lane, K, Lassetter, B, Lazarevic, L, Lebel, E, Lee, K, Lee, M, Lemm, K, Levitan, C, Lewis, M, Lin, L, Lin, S, Lippold, M, Loureiro, D, Luteijn, I, Mackinnon, S, Mainard, H, Marigold, D, Martin, D, Martinez, T, Masicampo, E, Matacotta, J, Mathur, M, May, M, Mechin, N, Mehta, P, Meixner, J, Melinger, A, Miller, J, Miller, M, Moore, K, Möschl, M, Motyl, M, Müller, S, Munafo, M, Neijenhuijs, K, Nervi, T, Nicolas, G, Nilsonne, G, Nosek, B, Nuijten, M, Olsson, C, Osborne, C, Ostkamp, L, Pavel, M, Penton Voak, I, Perna, O, Pernet, C, Perugini, M, Pipitone, N, Pitts, M, Plessow, F, Prenoveau, J, Rahal, R, Ratliff, K, Reinhard, D, Renkewitz, F, Ricker, A, Rigney, A, Rivers, A, Roebke, M, Rutchick, A, Ryan, R, Sahin, O, Saide, A, Sandstrom, G, Santos, D, Saxe, R, Schlegelmilch, R, Schmidt, K, Scholz, S, Seibel, L, Selterman, D, Shaki, S, Simpson, E, Sinclair, H, Skorinko, J, Slowik, A, Snyder, J, Soderberg, C, Sonnleitner, C, Spencer, N, Spies, J, Steegen, S, Stieger, S, Strohminger, N, Sullivan, G, Talhelm, T, Tapia, M, te Dorsthorst, A, Thomae, M, Thomas, S, Tio, P, Traets, F, Tsang, S, Tuerlinckx, F, Turchan, P, Valášek, M, van 't Veer, A, Van Aert, R, van Assen, M, van Bork, R, van de Ven, M, van den Bergh, D, van der Hulst, M, van Dooren, R, van Doorn, J, van Renswoude, D, van Rijn, H, Vanpaemel, W, Vásquez Echeverría, A, Vazquez, M, Velez, N, Vermue, M, Verschoor, M, Vianello, M, Voracek, M, Vuu, G, Wagenmakers, E, Weerdmeester, J, Welsh, A, Westgate, E, Wissink, J, Wood, M, Woods, A, Wright, E, Wu, S, Zeelenberg, M, Zuni, K, Sociology/ICS, Experimental Psychology, Human Technology Interaction, Sociale Psychologie (Psychologie, FMG), Ontwikkelingspsychologie (Psychologie, FMG), and Brein en Cognitie (Psychologie, FMG)

Subjects
Research design, Department Psychologie, BF Psychology, media_common.quotation_subject, POWER, Learning and Plasticity, Reproducibility Project, Q1, Experimental Psychopathology and Treatment, Replication (statistics), Statistics, TRUTH, Psychology, General, Mathematics, media_common, Selection bias, Replication crisis, Behaviour Change and Well-being, Multidisciplinary, PUBLICATION, Publication bias, Reproducibility, Confidence interval, INCENTIVES, PREVALENCE, Meta-analysis, REPLICABILITY, REPLICATION, Developmental Psychopathology, FALSE
Abstract

IntroductionReproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence. Even research of exemplary quality may have irreproducible empirical findings because of random or systematic error.RationaleThere is concern about the rate and predictors of reproducibility, but limited evidence. Potentially problematic practices include selective reporting, selective analysis, and insufficient specification of the conditions necessary or sufficient to obtain the results. Direct replication is the attempt to recreate the conditions believed sufficient for obtaining a previously observed finding and is the means of establishing reproducibility of a finding with new data. We conducted a large-scale, collaborative effort to obtain an initial estimate of the reproducibility of psychological science.ResultsWe conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. There is no single standard for evaluating replication success. Here, we evaluated reproducibility using significance and P values, effect sizes, subjective assessments of replication teams, and meta-analysis of effect sizes. The mean effect size (r) of the replication effects (Mr = 0.197, SD = 0.257) was half the magnitude of the mean effect size of the original effects (Mr = 0.403, SD = 0.188), representing a substantial decline. Ninety-seven percent of original studies had significant results (P < .05). Thirty-six percent of replications had significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.ConclusionNo single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here. Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know.

Published
2015

Catalog

Books, media, physical & digital resources
See catalog results