Search

Your search keyword '"Christmann, Pablo"' showing total 40 results
Start Over Author "Christmann, Pablo"

Refine your results

more »

more »

more »

more »
40 results on '"Christmann, Pablo"'

2. Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

Author

Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Nguyen, Hung H. V., Adem, Muna, Adriaans, Jule, Alvarez-Benjumea, Amalia, Andersen, Henrik K., Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Balzer, Dave, Bauer, Gerrit, Bauer, Paul C., Baumann, Markus, Baute, Sharon, Benoit, Verena, Bernauer, Julian, Berning, Carl, Berthold, Anna, Bethke, Felix S., Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes N., Bobzien, Licia, Bohman, Andrea, Bol, Thijs, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin B., Carlos-Castillo, Juan, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian S., Damian, Elena, Ecker, Alejandro, Edelmann, Achim, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Gaasendam, Chris, Gavras, Konstantin, Gayle, Vernon, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Grömping, Max, Groß, Martin, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Hövermann, Andreas, Hunger, Sophia, Hunkler, Christian, Huth, Nora, Ignácz, Zsófia S., Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kauff, Mathias, Kleinert, Manuel, Klinger, Julia, Kolb, Jan-Philipp, Kołczyńska, Marta, Kuk, John, Kunißen, Katharina, Sinatra, Dafina Kurti, Langenkamp, Alexander, Lersch, Philipp M., Löbel, Lea-Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan E., Malancu, Natalia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar J., McManus, Patricia, McWagner, Kyle, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan, Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna O., Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel R., Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gregor, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Regine, Schmidt, Katja M., Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Jürgen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stiers, Dieter, Stojmenovska, Dragana, Storz, Nora, Striessnig, Erich, Stroppe, Anne-Kathrin, Teltemann, Janna, Tibajev, Andrey, Tung, Brian, Vagni, Giacomo, Van Assche, Jasper, van der Linden, Meta, van der Noll, Jolanda, Van Hootegem, Arno, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Yamada, Yuki, Zhang, Nan, Ziller, Conrad, Zins, Stefan, and Żółtak, Tomasz

Published
2022

3. The Impact of Survey Mode Design and Questionnaire Length on Measurement Quality.

Author

Cernat, Alexandru, Sakshaug, Joseph, Christmann, Pablo, and Gummer, Tobias

Subjects
WEB design, LENGTH measurement, DATA quality, QUESTIONNAIRES, NONRESPONSE (Statistics)
Abstract

Mixed-mode surveys are popular as they can save costs and maintain (or improve) response rates relative to single-mode surveys. Nevertheless, it is not yet clear how design decisions like survey mode or questionnaire length impact measurement quality. In this study, we compare measurement quality in an experiment of three distinct survey designs implemented in the German sample of the European Values Study: a single-mode face-to-face design, a mixed-mode mail/web design, and a shorter (matrix) questionnaire in the mixed-mode design. We compare measurement quality in different ways, including differences in distributions across several data quality indicators as well as equivalence testing over 140 items in 25 attitudinal scales. We find similar data quality across the survey designs, although the mixed-mode survey shows more item nonresponse compared to the single-mode survey. Using equivalence testing we find that most scales achieve metric equivalence and, to a lesser extent, scalar equivalence across the designs. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

4. A new data infrastructure for family research and demographic analysis: the German Family Demography Panel Study (FReDA)

Author

Hank, Karsten, primary, Gummer, Tobias, additional, Bujard, Martin, additional, Neyer, Franz J, additional, Pollak, Reinhard, additional, Spieß, C Katharina, additional, Wolf, Christof, additional, Christmann, Pablo, additional, Kunz, Tanja, additional, Lück, Detlev, additional, Naderi, Robert, additional, Nutz, Theresa, additional, Schmid, Lisa, additional, and Thönnissen, Carolin, additional

Published
2024
Full Text
View/download PDF

5. Concurrent, Web-First, or Web-Only? How Different Mode Sequences Perform in Recruiting Participants for a Self-Administered Mixed-mode Panel Study.

Author

Christmann, Pablo, Gummer, Tobias, Häring, Armando, Kunz, Tanja, Oehrlein, Anne-Sophie, Ruland, Michael, and Schmid, Lisa

Subjects
*PANEL analysis, *COVID-19 pandemic, *DEMOGRAPHIC research, *FAMILY research, *DATA quality
Abstract

During the COVID-19 pandemic, many survey programs switched to self-administered modes of data collection, often offering respondents both web-based and paper-based questionnaires. However, there is little guidance as to when to offer which mode, especially when the aim is to recruit participants for a panel survey. This study examines the effectiveness of different mode-sequence designs by analyzing an experiment implemented in the recruitment wave of the German panel study "Family Research and Demographic Analysis." We randomly assigned 108,256 individuals aged 18–49 years to one of three mode-sequence-design conditions: concurrent, web-first including a paper-based questionnaire with the second reminder (web-first-2), and web-first including a paper-based questionnaire with the third reminder (web-first-3). A fourth, simulated group did not receive a paper-based questionnaire (web-only). We analyzed how different mode-sequence designs affected outcome rates, sample composition, response distributions, data quality, share of paper-based questionnaires, postage costs, number of postal mailings in the recruitment wave, and participation in the first regular wave. Our results show no differences in response distributions and small differences in sample composition across the four mode-sequence designs. As the web-first-2 and simulated web-only designs yielded comparatively good response and recruitment rates at reasonable costs, we recommend their use when surveying adults between 18 and 49 years old. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

10. Evaluating an Alternative Frame for Address-Based Sampling in Germany: The Address Database From Deutsche Post Direkt

Author

Stadtmüller, Sven, Silber, Henning, Gummer, Tobias, Sand, Matthias, Zins, Stefan, Beuthner, Christoph, Christmann, Pablo, Stadtmüller, Sven, Silber, Henning, Gummer, Tobias, Sand, Matthias, Zins, Stefan, Beuthner, Christoph, and Christmann, Pablo

Abstract

In Germany, the population registers with addresses of individuals can be used for address-based sampling. However, unlike countries with a centralized register, municipalities in Germany administer their registers themselves. This not only makes sampling for a nation­wide survey more costly and cumbersome but may also result in gaps in the gross sample, as selected municipalities may refuse to allow their registers to be used for sampling pur­poses. If substitute municipalities are not available, other sampling methods are required. The present study tested the feasibility of using the address database from Deutsche Post Direkt (ADB-DPD) as an alternative frame for address-based sampling in Germany. We simultaneously conducted two almost identical surveys in the German city of Mannheim with gross samples of equal size (N = 3,000). One sample was drawn from the city’s popu­lation register, the other from the commercial ADB-DPD. Our findings suggest that the ADB-DPD performs well both in terms of survey response and up-to-dateness. Due to relatively low costs and the fast provision of addresses, the ADB-DPD could be particu­larly attractive for survey projects with limited budgets and tight schedules. However, these benefits come at considerable cost. First, the use of the ADB-DPD is limited to self-admin­istered surveys. More importantly, in the net sample of the DPD survey, women and young persons were considerably underrepresented. This indicates coverage issues about which DPD provided no further information. Based on our analyses, we offer practical insights into the feasibility of using the ADB-DPD for sampling purposes and suggest avenues for future research.

Published
2023

11. Bausteine Forschungsdatenmanagement : 2023, 1

Author

Bechert, Insa, Christmann, Pablo, Franken, Andreas, Franzese, Fabio, Koomen, Maarten, Krell, Kristina, Kunz, Tanja, Pflüger, Senta-Melissa, Scheller, Percy, Trommer, Maximilian, and von Rotz, Christina

Subjects
Plausibilisierung, Sozialwissenschaften, Datenaufbereitung
Abstract

Die vorliegende Handreichung zeigt, dass in quantitativen, sozialwissenschaftlichen Erhebungen Unplausibilitäten entstehen können, die möglicherweise zu Auswertungsproblemen und Fehlanalysen führen. Ziel der Handreichung ist, die möglichen Probleme zu systematisieren und konkrete Hilfestellungen bei der Identifikation, dem Umgang und der Vermeidung von Unplausibilitäten zu geben. Außerdem werden Dokumentationsempfehlungen ausgesprochen. Diese Handreichung richtet sich dabei in erster Linie an Datenmanager:innen oder Datenproduzent:innen von Erhebungen, die bei der Studienplanung und Datenaufbereitung mit Plausibilitätsproblemen in Berührung kommen. Aber auch für Studierende und Forscher:innen könnte sie eine Hilfestellung darstellen. Gegliedert ist die Handreichung wie folgt: Nach einer Einleitung und der inhaltlichen Einbettung des Themas in den Forschungsprozess folgt eine allgemeine Einführung in den Plausibilisierungsprozess. Dabei wird zunächst diskutiert, ob Plausibilisierung überhaupt zwangsläufig stattfinden muss oder sollte. Im Anschluss wird ein Überblick über die Möglichkeiten gegeben, mit denen Plausibilitätsprobleme zunächst identifiziert und dann behandelt werden können. Diese Punkte werden allgemeingültig formuliert und beziehen sich nicht auf spezielle Arten von Plausibilitätsproblemen. Diese werden anhand der spezifischen Use Cases in den Folgekapiteln diskutiert. Die Plausibilitätsprobleme und Beispiele werden weitgehend unabhängig voneinander behandelt, sodass die Leser:innen sich auf die für ihre Studie bzw. Forschungsinteressen relevanten Inhalte konzentrieren oder die Handreichung als eine Art Nachschlagewerk verwenden können.

Published
2023
Full Text
View/download PDF

13. Evaluating an Alternative Frame for Address-Based Sampling in Germany: The Address Database From Deutsche Post Direkt

Author

Stadtmüller, Sven, Silber, Henning, Gummer, Tobias, Sand, Matthias, Zins, Stefan, Beuthner, Christoph, and Christmann, Pablo

Subjects
Erhebungstechniken und Analysetechniken der Sozialwissenschaften, data protection, Sozialwissenschaften, Soziologie, data bank, Stichprobe, personal data, Datenbank, Datenschutz, sample, data capture, Methods and Techniques of Data Collection and Data Analysis, Statistical Methods, Computer Methods, Deutsche Bundespost, ddc:300, Personendaten, address-based sampling, alternative sampling frame, population register, sample evaluation, sample composition, Datengewinnung, Social sciences, sociology, anthropology, address-based sampling, alternative sampling frame, population register, sample evaluation, sample composition, German Federal Postal Administration
Abstract

In Germany, the population registers with addresses of individuals can be used for address-based sampling. However, unlike countries with a centralized register, municipalities in Germany administer their registers themselves. This not only makes sampling for a nation­wide survey more costly and cumbersome but may also result in gaps in the gross sample, as selected municipalities may refuse to allow their registers to be used for sampling pur­poses. If substitute municipalities are not available, other sampling methods are required. The present study tested the feasibility of using the address database from Deutsche Post Direkt (ADB-DPD) as an alternative frame for address-based sampling in Germany. We simultaneously conducted two almost identical surveys in the German city of Mannheim with gross samples of equal size (N = 3,000). One sample was drawn from the city’s popu­lation register, the other from the commercial ADB-DPD. Our findings suggest that the ADB-DPD performs well both in terms of survey response and up-to-dateness. Due to relatively low costs and the fast provision of addresses, the ADB-DPD could be particu­larly attractive for survey projects with limited budgets and tight schedules. However, these benefits come at considerable cost. First, the use of the ADB-DPD is limited to self-admin­istered surveys. More importantly, in the net sample of the DPD survey, women and young persons were considerably underrepresented. This indicates coverage issues about which DPD provided no further information. Based on our analyses, we offer practical insights into the feasibility of using the ADB-DPD for sampling purposes and suggest avenues for future research., methods, data, analyses, Online First

Published
2022
Full Text
View/download PDF

14. Responsiveness, Performance and Corruption: Reasons for the Decline of Political Trust

Author

Torcal, Mariano, Christmann, Pablo, Torcal, Mariano, and Christmann, Pablo

Abstract

We test the importance of responsiveness, performance and corruption to explain the evolution of political trust in Spain between 1997 and 2019. To this end, the study analyses two longitudinal datasets, namely, a repeated cross-sectional dataset from the Spanish samples of Eurobarometer and an individual-level panel survey conducted during a period of economic recovery in 2015. The study finds that perceptions about political corruption and responsiveness matter greatly in shaping political trust and to a lesser extent economic performance. Although the Great Recession is likely responsible for the sharp decline in trust towards political parties and the parliament between 2008 and 2012, the analysis suggests that trust in representative institutions remains low even after the Recession because of a series of devastating corruption incidents and a perceived lack of responsiveness of the political system. On the other hand, the study finds indications that trust in the judicial system might have been mainly affected by perceptions of corruption.

Published
2022

16. Gaining Consent to Survey Respondents' Partners: The Importance of Anchors' Survey Experience in Selfadministered Modes.

Author

Gummer, Tobias, Christmann, Pablo, and Kunz, Tanja

Subjects
*PANEL analysis, *RESEARCH personnel, *RESPONDENTS, *QUESTIONNAIRES
Abstract

Dyadic surveys aim to interview pairs of respondents, such as partners in a relationship. In dyadic surveys, it is often necessary to obtain the anchors' consent to contact their partners and invite them to a survey. If the survey is operated in self-administered modes, no interviewer is present to improve the consent rate, for example, by providing convincing arguments and additional information. To overcome the challenges posed by self-administered modes for dyadic surveys and to improve consent rates, it is important to identify aspects that positively influence the likelihood of anchors giving consent to contact their partners. Ideally, these aspects are in the hands of the researchers, such as the survey design and aspects of the questionnaire. Thus, in this study, we analyzed the relationship between anchors' survey experience and their willingness to consent to surveying their partners in self-administered modes. Based on data from the German Family Demography Panel Study (FReDA), we found that the anchors' perceptions of the questionnaire as "interesting" or "too personal" were related to consent rates. These relationships were consistent across different survey modes and devices. Effects of other aspects of the questionnaire, such as "important for science" and "diverse" varied between modes and devices. We concluded with practical recommendations for survey research and an outlook for future research. [ABSTRACT FROM AUTHOR]

Published
2023
Full Text
View/download PDF

18. Evaluating an Alternative Frame for Address-Based Sampling in Germany: The Address Database of the Deutsche Post Direkt

Author

Matthias, Sand, Gummer, Tobias, Silber, Henning, Zins, Stefan, Stadtmüller, Sven, Christmann, Pablo, and beuthner, Christoph

Subjects
bepress|Social and Behavioral Sciences, SocArXiv|Social and Behavioral Sciences
Abstract

In Germany, the population register with individual addresses can be utilized for address-based sampling. However, unlike in countries with a centralized register, local authorities in Germany administer their own registers. This not only makes sampling for a nation-wide survey more costly and cumbersome but may also result in gaps in the gross sample, as selected municipalities can refuse the use of their register for sampling purposes. If alternative municipalities to substitute refusals are not available, different sampling methods are required. With the present study, we aimed at testing the usability of the Address Database of the Deutsche Post Direkt (ADB-DPD) as an alternative frame for address-based sampling in Germany. We simultaneously carried out two surveys in the German city of Mannheim with gross samples of 3,000 each; one drawn from the city’s population register and the other from the ADB-DPD. Our findings suggest that the ADB-DPD performs well, both in terms of survey response and up-to-dateness. Due to relatively low costs and the fast provision of addresses, the ADB-DPD can be particularly attractive for survey projects with limited budgets and tight schedules. However, these benefits come at considerable cost. First, the usage of the ADB-DPD is limited to self-administered surveys. More importantly, in the net sample of the DPD survey, women and young persons were considerably under-represented. This indicates coverage issues for which DPD provided no further information. Based on our analysis, we provide practical remarks on the usability of the ADB-DPD and suggest avenues for future research.

Published
2021
Full Text
View/download PDF

20. How Many Replicators Does It Take to Achieve Reliability? Investigating Researcher Variability in a Crowdsourced Replication

Author

Breznau, Nate, primary, Rinke, Eike Mark, additional, Wuttke, Alexander, additional, Nguyen, Hung Hoang Viet, additional, Adem, Muna, additional, Adriaans, Jule, additional, Akdeniz, Esra, additional, Alvarez-Benjumea, Amalia, additional, Andersen, Henrik Kenneth, additional, Auer, Daniel, additional, Azevedo, Flavio, additional, Bahnsen, Oke, additional, Bai, Ling, additional, Balzer, Dave, additional, Bauer, Gerrit, additional, Bauer, Paul, additional, Baumann, Markus, additional, Baute, Sharon, additional, Benoit, Verena, additional, Bernauer, Julian, additional, Berning, Carl, additional, Berthold, Anna, additional, Bethke, Felix S., additional, Biegert, Thomas, additional, Blinzler, Katharina, additional, Blumenberg, Johannes, additional, Bobzien, Licia, additional, Bohman, Andrea, additional, Bol, Thijs, additional, Bostic, Amie, additional, Brzozowska, Zuzanna, additional, Burgdorf, Katharina, additional, Burger, Kaspar, additional, Busch, Kathrin, additional, Castillo, Juan Carlos, additional, Chan, Nathan, additional, Christmann, Pablo, additional, Connelly, Roxanne, additional, Czymara, Christian S., additional, Damian, Elena, additional, de Rooij, Eline Adriane, additional, Ecker, Alejandro, additional, Edelmann, Achim, additional, Eder, Christina, additional, Eger, Maureen A., additional, Ellerbrock, Simon, additional, Forke, Anna, additional, Forster, Andrea Gabriele, additional, Freire, Danilo, additional, Gaasendam, Chris, additional, Gavras, Konstantin, additional, Gayle, Vernon, additional, Gessler, Theresa, additional, Gnambs, Timo, additional, Godefroidt, Amélie, additional, Grömping, Max, additional, Groß, Martin, additional, Gruber, Stefan, additional, Gummer, Tobias, additional, Hadjar, Andreas, additional, Halbherr, Verena, additional, Heisig, Jan Paul, additional, Hellmeier, Sebastian, additional, Heyne, Stefanie, additional, Hirsch, Magdalena, additional, Hjerm, Mikael, additional, Hochman, Oshrat, additional, Höffler, Jan H., additional, Hövermann, Andreas, additional, Hunger, Sophia, additional, Hunkler, Christian, additional, Huth, Nora, additional, Ignacz, Zsofia, additional, Israel, Sabine, additional, Jacobs, Laura, additional, Jacobsen, Jannes, additional, Jaeger, Bastian, additional, Jungkunz, Sebastian, additional, Jungmann, Nils, additional, Kanjana, Jennifer, additional, Kauff, Mathias, additional, Khan, Salman, additional, Khatua, Sayak, additional, Kleinert, Manuel, additional, Klinger, Julia, additional, Kolb, Jan-Philipp, additional, Kolczynska, Marta, additional, Kuk, John Seungmin, additional, Kunißen, Katharina, additional, Sinatra, Dafina Kurti, additional, Greinert, Alexander, additional, Lee, Robin C., additional, Lersch, Philipp M., additional, Liu, David, additional, Löbel, Lea-Maria, additional, Lutscher, Philipp, additional, Mader, Matthias, additional, Madia, Joan Eliel, additional, Malancu, Natalia, additional, Maldonado, Luis, additional, Marahrens, Helge, additional, Martin, Nicole, additional, Martinez, Paul, additional, Mayerl, Jochen, additional, MAYORGA, OSCAR Jose, additional, McDonnell, Robert Myles, additional, McManus, Patricia A., additional, Wagner, Kyle, additional, Meeusen, Cecil, additional, Meierrieks, Daniel, additional, Mellon, Jonathan, additional, Merhout, Friedolin, additional, Merk, Samuel, additional, Meyer, Daniel, additional, Micheli, Leticia, additional, Mijs, Jonathan J.B., additional, Moya, Cristóbal, additional, Neunhoeffer, Marcel, additional, Nüst, Daniel, additional, Nygård, Olav, additional, Ochsenfeld, Fabian, additional, Otte, Gunnar, additional, Pechenkina, Anna, additional, Pickup, Mark, additional, Prosser, Christopher, additional, Raes, Louis, additional, Ralston, Kevin, additional, Ramos, Miguel, additional, Reichert, Frank, additional, Roets, Arne, additional, Rogers, Jonathan, additional, Ropers, Guido, additional, Samuel, Robin, additional, Sand, Gergor, additional, Petrarca, Constanza Sanhueza, additional, Schachter, Ariela, additional, Schaeffer, Merlin, additional, Schieferdecker, David, additional, Schlueter, Elmar, additional, Schmidt, Katja, additional, Schmidt, Regine, additional, Schmidt-Catran, Alexander, additional, Schmiedeberg, Claudia, additional, Schneider, Jürgen, additional, Schoonvelde, Martijn, additional, Schulte-Cloos, Julia, additional, Schumann, Sandy, additional, Schunck, Reinhard, additional, Schupp, Juergen, additional, Seuring, Julian, additional, Silber, Henning, additional, Sleegers, Willem W. A., additional, Sonntag, Nico, additional, Staudt, Alexander, additional, Steiber, Nadia, additional, Steiner, Nils, additional, Sternberg, Sebastian, additional, Stiers, Dieter, additional, Stojmenovska, Dragana, additional, Storz, Nora, additional, Striessnig, Erich, additional, Stroppe, Anne-Kathrin, additional, Suchow, Jordan, additional, Teltemann, Janna, additional, Tibajev, Andrey, additional, Tung, Brian B., additional, Vagni, Giacomo, additional, Van Assche, Jasper, additional, van der Linden, Meta, additional, van der Noll, Jolanda, additional, Van Hootegem, Arno, additional, Vogtenhuber, Stefan, additional, Voicu, Bogdan, additional, Wagemans, Fieke, additional, Wehl, Nadja, additional, Werner, Hannah, additional, Wiernik, Brenton M., additional, Winter, Fabian, additional, Wolf, Christof, additional, Wu, Cary, additional, Yamada, Yuki, additional, Zakula, Björn, additional, Zhang, Nan, additional, Ziller, Conrad, additional, Zins, Stefan, additional, and Żółtak, Tomasz, additional

Published
2021
Full Text
View/download PDF

21. The impact of Covid-19 on fieldwork efforts and planning in pairfam and FReDA-GGS

Author

Gummer, Tobias, Schmiedeberg, Claudia, Bujard, Martin, Christmann, Pablo, Hank, Karsten, Kunz, Tanja, Lück, Detlev, and Neyer, Franz J.

Subjects
03 medical and health sciences, 0302 clinical medicine, 030212 general & internal medicine, 010501 environmental sciences, 01 natural sciences, 0105 earth and related environmental sciences
Abstract

The worldwide spread of the COVID-19 pandemic has disrupted the fieldwork of surveys. The data collection efforts via the face-to-face mode have been affected especially, including the ongoing surveys that were in the field during the COVID-19 outbreak and the planned surveys scheduled for fieldwork later in 2020. We provide an account of how COVID-19 has impacted two family studies in Germany: “The German Family Panel” (pairfam) and the “Generations and Gender Survey” (GGS) both of which will be part of the “Family Research and Demographic Analysis” (FReDA) infrastructure. Based on pairfam, we illustrate the effects of the pandemic on ongoing data collection and the measures taken to proceed with fieldwork, and we report on a special COVID-19 survey. Based on FReDA-GGS, we outline how COVID-19 has affected our planned survey schedules, what future challenges are expected when fieldwork becomes possible again, and how we have adapted our plans accordingly., Survey Research Methods, Vol 14 No 2 (2020): Survey Research Methods During the COVID-19 Crisis

Published
2020
Full Text
View/download PDF

22. Fieldwork Monitoring in Practice: Insights from 17 Large-scale Social Science Surveys in Germany

Author

Meitinger, Katharina, Stadtmüller, Sven, Silber, Henning, Auriga, Roman, Bergmann, Michael, Blohm, Michael, Blumenberg, Manuela, Christmann, Pablo, Felderer, Barbara, Frodermann, Corinna, Griese, Florian, Gummer, Tobias, Koch, Achim, Kottwitz, Anita, Krell, Kristina, Krieger, Ulrich, Liebau, Elisabeth, Martin, Silke, Müller-Kuller, Andre, Rammstedt, Beatrice, Schaurer, Ines, Scherpenzeel, Annette, Schmiedeberg, Claudia, Schmidt, Tobias, Schnaudt, Christian, Verhoeven, Sascha, Zabal, Anouk, Leerstoel Schoot, and Methodology and statistics for the behavioural and social sciences

Subjects
Monitoring, fieldwork measures, Umfrageforschung, Fragebogen, Federal Republic of Germany, key performance indicators, fieldwork monitoring, survey research, Messung, field research, Social sciences, sociology, anthropology, Feldforschung, Erhebungstechniken und Analysetechniken der Sozialwissenschaften, Sozialwissenschaften, Soziologie, questionnaire, indicator, indicator research, Indikator, standardization (meth.), Bundesrepublik Deutschland, Methods and Techniques of Data Collection and Data Analysis, Statistical Methods, Computer Methods, Standardisierung, Indikatorenforschung, ddc:300, measurement
Abstract

This study provides a synopsis of the current fieldwork monitoring practices of large-scale surveys in Germany. Based on the results of a standardized questionnaire, the study summarizes fieldwork monitoring indicators used and fieldwork measures carried out by 17 large-scale social sciences surveys in Germany. Our descriptive results reveal that a common set of fieldwork indicators and measures exist on which the studied surveys rely. However, it also uncovers the need for additional design-specific indicators. Finally, it underlines the importance of a close cooperation between survey representatives and fieldwork agencies to optimize processes in fieldwork monitoring in the German survey context. The article concludes with implications for fieldwork practice.

Published
2020

23. Fieldwork Monitoring in Practice: Insights from 17 Large-scale Social Science Surveys in Germany

Author

Meitinger, K.M., Stadtmüller, Sven, Silber, Henning, Auriga, Roman, Bergmann, Michael, Blohm, Michael, Blumenberg, Manuela, Christmann, Pablo, Felderer, Barbara, Frodermann, Corinna, Griese, Florian, Gummer, Tobias, Hähnel, Sascha, Koch, Achim, Kottwitz, Anita, Krell, Kristina, Krieger, Ulrich, Liebau, Elisabeth, Martin, Silke, Müller-Kuller, Andre, Rammstedt, Beatrice, Schaurer, Ines, Scherpenzeel, Annette, Schmiedeberg, Claudia, Schmidt, Tobias, Schnaudt, Christian, Zabal, Anouk, Leerstoel Schoot, and Methodology and statistics for the behavioural and social sciences

Subjects
fieldwork measures, key performance indicators, fieldwork monitoring
Abstract

This study provides a synopsis of the current fieldwork monitoring practices of large-scale surveys in Germany. Based on the results of a standardized questionnaire, the study summarizes fieldwork monitoring indicators used and fieldwork measures carried out by 17 large-scale social sciences surveys in Germany. Our descriptive results reveal that a common set of fieldwork indicators and measures exist on which the studied surveys rely. However, it also uncovers the need for additional design-specific indicators. Finally, it underlines the importance of a close cooperation between survey representatives and fieldwork agencies to optimize processes in fieldwork monitoring in the German survey-context. The article concludes with implications for fieldwork practice.

Published
2020

24. The European Values Study 2017: On the Way to the Future Using Mixed-Modes

Author

Luijkx, Ruud, Jónsdóttir, Guðbjörg Andrea, Gummer, Tobias, Ernst Stähli, Michèle, Frederiksen, Morten, Ketola, Kimmo, Reeskens, Tim, Brislinger, Evelyn, Christmann, Pablo, Gunnarsson, Stefán Þór, Hjaltason, Árni Bragi, Joye, Dominique, Lomazzi, Vera, Maineri, Angelica M., Milbert, Patricia, Ochsner, Michael, Pollien, Alexandre, Sapin, Marlène, Solanes Ros, Ivet, Verhoeven, Sascha, Wolf, Christof, Luijkx, Ruud, Jónsdóttir, Guðbjörg Andrea, Gummer, Tobias, Ernst Stähli, Michèle, Frederiksen, Morten, Ketola, Kimmo, Reeskens, Tim, Brislinger, Evelyn, Christmann, Pablo, Gunnarsson, Stefán Þór, Hjaltason, Árni Bragi, Joye, Dominique, Lomazzi, Vera, Maineri, Angelica M., Milbert, Patricia, Ochsner, Michael, Pollien, Alexandre, Sapin, Marlène, Solanes Ros, Ivet, Verhoeven, Sascha, and Wolf, Christof

Abstract

The European Values Study (EVS) was first conducted in 1981 and then repeated in 1990, 1999, 2008, and 2017, with the aim of providing researchers with data to investigate whether European individual and social values are changing and to what degree. The EVS is traditionally carried out as a probability-based face-to-face survey that takes around 1 hour to complete. In recent years, large-scale population surveys such as the EVS have been challenged by decreasing response rates and increasing survey costs. In the light of these challenges, six countries that participated in the last wave of the EVS tested the application of self-administered mixed-modes (Denmark, Finland, Germany, Iceland, the Netherlands, and Switzerland). With the present data brief, we will introduce researchers to the latest wave of the EVS, the implemented mode experiments, and the EVS data releases. In our view, it is pivotal for data use in substantive research to make the reasoning behind design changes and country-specific implementations transparent as well as to highlight new research opportunities.

Published
2021

25. How Many Replicators Does It Take to Achieve Reliability? Investigating Researcher Variability in a Crowdsourced Replication

Author

Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Nguyen, Hung H. V., Adem, Muna, Adriaans, Jule, Akdeniz, Esra, Alvarez-Benjumea, Amalia, Andersen, Henrik Kenneth, Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Bai, Ling, Balzer, Dave, Bauer, Gerrit, Bauer, Paul C., Baumann, Markus, Baute, Sharon, Benoit, Verena, Bernauer, Julian, Berning, Carl, Berthold, Anna, Bethke, Felix, Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes, Bobzien, Licia, Bohman, Andrea, Bol, Thijs, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin, Castillo, Juan Carlos, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian S., Damian, Elena, Rooij, Eline De, Ecker, Alejandro, Edelmann, Achim, Eder, Christina, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Freire, Danilo, Gaasendam, Chris, Gavras, Konstantin, Gayle, Vernon, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Grömping, Max, Groß, Martin, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Halbherr, Verena, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Höffler, Jan H., Hövermann, Andreas, Hunger, Sophia, Hunkler, Christian, Huth, Nora, Ignacz, Zsofia, Israel, Sabine, Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kanjana, Jennifer, Kauff, Mathias, Khan, Salman, Khatua, Sayak, Kleinert, Manuel, Klinger, Julia, Kolb, Jan-Philipp, Kołczyńska, Marta, Kuk, John Seungmin, Kunißen, Katharina, Sinatra, Dafina Kurti, Greinert, Alexander, C. Lee, Robin, Lersch, Philipp M., Liu, David, Löbel, Lea-Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan, Malancu, Natalia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar Jose, McDonnell, Robert Myles, McManus, Patricia, Wagner, Kyle, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan J. B., Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna, Pickup, Mark, Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel, Reichert, Frank, Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gregor, Petrarca, Constanza Sanhueza, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Katja, Schmidt, Regine, Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Jürgen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stojmenovska, Dragana, Storz, Nora, Striessnig, Erich, Stroppe, Anne-Kathrin, Suchow, Jordan W., Teltemann, Janna, Tibajev, Andrey, Tung, Brian B., Vagni, Giacomo, Assche, Jasper Van, Linden, Meta Van Der, Noll, Jolanda Van Der, Hootegem, Arno Van, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Wu, Cary, Yamada, Yuki, Zakula, Björn, Zhang, Nan, Ziller, Conrad, Zins, Stefan, Żółtak, Tomasz, Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Nguyen, Hung H. V., Adem, Muna, Adriaans, Jule, Akdeniz, Esra, Alvarez-Benjumea, Amalia, Andersen, Henrik Kenneth, Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Bai, Ling, Balzer, Dave, Bauer, Gerrit, Bauer, Paul C., Baumann, Markus, Baute, Sharon, Benoit, Verena, Bernauer, Julian, Berning, Carl, Berthold, Anna, Bethke, Felix, Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes, Bobzien, Licia, Bohman, Andrea, Bol, Thijs, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin, Castillo, Juan Carlos, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian S., Damian, Elena, Rooij, Eline De, Ecker, Alejandro, Edelmann, Achim, Eder, Christina, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Freire, Danilo, Gaasendam, Chris, Gavras, Konstantin, Gayle, Vernon, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Grömping, Max, Groß, Martin, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Halbherr, Verena, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Höffler, Jan H., Hövermann, Andreas, Hunger, Sophia, Hunkler, Christian, Huth, Nora, Ignacz, Zsofia, Israel, Sabine, Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kanjana, Jennifer, Kauff, Mathias, Khan, Salman, Khatua, Sayak, Kleinert, Manuel, Klinger, Julia, Kolb, Jan-Philipp, Kołczyńska, Marta, Kuk, John Seungmin, Kunißen, Katharina, Sinatra, Dafina Kurti, Greinert, Alexander, C. Lee, Robin, Lersch, Philipp M., Liu, David, Löbel, Lea-Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan, Malancu, Natalia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar Jose, McDonnell, Robert Myles, McManus, Patricia, Wagner, Kyle, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan J. B., Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna, Pickup, Mark, Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel, Reichert, Frank, Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gregor, Petrarca, Constanza Sanhueza, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Katja, Schmidt, Regine, Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Jürgen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stojmenovska, Dragana, Storz, Nora, Striessnig, Erich, Stroppe, Anne-Kathrin, Suchow, Jordan W., Teltemann, Janna, Tibajev, Andrey, Tung, Brian B., Vagni, Giacomo, Assche, Jasper Van, Linden, Meta Van Der, Noll, Jolanda Van Der, Hootegem, Arno Van, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Wu, Cary, Yamada, Yuki, Zakula, Björn, Zhang, Nan, Ziller, Conrad, Zins, Stefan, and Żółtak, Tomasz

Abstract

The paper reports findings from a crowdsourced replication. Eighty-four replicator teams attempted to verify results reported in an original study by running the same models with the same data. The replication involved an experimental condition. A “transparent” group received the original study and code, and an “opaque” group received the same underlying study but with only a methods section and description of the regression coefficients without size or significance, and no code. The transparent group mostly verified the original study (95.5%), while the opaque group had less success (89.4%). Qualitative investigation of the replicators’ workflows reveals many causes of non-verification. Two categories of these causes are hypothesized, routine and non-routine. After correcting non-routine errors in the research process to ensure that the results reflect a level of quality that should be present in ‘real-world’ research, the rate of verification was 96.1 in the transparent group and 92.4 in the opaque group. Two conclusions follow: (1) Although high, the verification rate suggests that it would take a minimum of three replicators per study to achieve replication reliability of at least 95 confidence assuming ecological validity in this controlled setting, and (2) like any type of scientific research, replication is prone to errors that derive from routine and undeliberate actions in the research process. The latter suggests that idiosyncratic researcher variability might provide a key to understanding part of the “reliability crisis” in social and behavioral science and is a reminder of the importance of transparent and well documented workflows.

Published
2021

26. Observing Many Researchers Using the Same Data and Hypothesis Reveals a Hidden Universe of Uncertainty

Author

Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Adem, Muna, Adriaans, Jule, Alvarez-Benjumea, Amalia, Andersen, Henrik Kenneth, Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Balzer, Dave, Bauer, Gerrit, Bauer, Paul C., Baumann, Markus, Baute, Sharon, Benoit, Verena, Bernauer, Julian, Berning, Carl, Berthold, Anna, Bethke, Felix, Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes, Bobzien, Licia, Bohman, Andrea, Bol, Thijs, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin, Castillo, Juan Carlos, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian S., Damian, Elena, Ecker, Alejandro, Edelmann, Achim, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Gaasendam, Chris, Gavras, Konstantin, Gayle, Vernon, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Grömping, Max, Groß, Martin, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Hövermann, Andreas, Hunger, Sophia, Hunkler, Christian, Huth, Nora, Ignacz, Zsofia, Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kauff, Mathias, Kleinert, Manuel, Klinger, Julia, Kolb, Jan-Philipp, Kołczyńska, Marta, Kuk, John Seungmin, Kunißen, Katharina, Sinatra, Dafina Kurti, Greinert, Alexander, Lersch, Philipp M., Löbel, Lea-Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan, Malancu, Natalia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar Jose, McManus, Patricia, Wagner, Kyle, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan J. B., Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna, Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel, Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gregor, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Katja M., Schmidt, Regine, Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Jürgen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stiers, Dieter, Stojmenovska, Dragana, Storz, Nora, Striessnig, Erich, Stroppe, Anne-Kathrin, Teltemann, Janna, Tibajev, Andrey, Tung, Brian B., Vagni, Giacomo, Assche, Jasper Van, Linden, Meta Van Der, Noll, Jolanda Van Der, Hootegem, Arno Van, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Yamada, Yuki, Zhang, Nan, Ziller, Conrad, Zins, Stefan, Żółtak, Tomasz, Nguyen, Hung H. V., Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Adem, Muna, Adriaans, Jule, Alvarez-Benjumea, Amalia, Andersen, Henrik Kenneth, Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Balzer, Dave, Bauer, Gerrit, Bauer, Paul C., Baumann, Markus, Baute, Sharon, Benoit, Verena, Bernauer, Julian, Berning, Carl, Berthold, Anna, Bethke, Felix, Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes, Bobzien, Licia, Bohman, Andrea, Bol, Thijs, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin, Castillo, Juan Carlos, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian S., Damian, Elena, Ecker, Alejandro, Edelmann, Achim, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Gaasendam, Chris, Gavras, Konstantin, Gayle, Vernon, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Grömping, Max, Groß, Martin, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Hövermann, Andreas, Hunger, Sophia, Hunkler, Christian, Huth, Nora, Ignacz, Zsofia, Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kauff, Mathias, Kleinert, Manuel, Klinger, Julia, Kolb, Jan-Philipp, Kołczyńska, Marta, Kuk, John Seungmin, Kunißen, Katharina, Sinatra, Dafina Kurti, Greinert, Alexander, Lersch, Philipp M., Löbel, Lea-Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan, Malancu, Natalia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar Jose, McManus, Patricia, Wagner, Kyle, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan J. B., Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna, Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel, Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gregor, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Katja M., Schmidt, Regine, Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Jürgen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stiers, Dieter, Stojmenovska, Dragana, Storz, Nora, Striessnig, Erich, Stroppe, Anne-Kathrin, Teltemann, Janna, Tibajev, Andrey, Tung, Brian B., Vagni, Giacomo, Assche, Jasper Van, Linden, Meta Van Der, Noll, Jolanda Van Der, Hootegem, Arno Van, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Yamada, Yuki, Zhang, Nan, Ziller, Conrad, Zins, Stefan, Żółtak, Tomasz, and Nguyen, Hung H. V.

Abstract

How does noise generated by researcher decisions undermine the credibility of science? We test this by observing all decisions made among 73 research teams as they independently conduct studies on the same hypothesis with identical starting data. We find excessive variation of outcomes. When combined, the 107 observed research decisions taken across teams explained at most 2.6 of the total variance in effect sizes and 10 of the deviance in subjective conclusions. Expertise, prior beliefs and attitudes of the researchers explain even less. Each model deployed to test the hypothesis was unique, which highlights a vast universe of research design variability that is normally hidden from view and suggests humility when presenting and interpreting scientific findings.

Published
2021

28. Observing Many Researchers Using the Same Data and Hypothesis Reveals a Hidden Universe of Uncertainty

Author

Breznau, Nate, primary, Rinke, Eike Mark, additional, Wuttke, Alexander, additional, Adem, Muna, additional, Adriaans, Jule, additional, Alvarez-Benjumea, Amalia, additional, Andersen, Henrik Kenneth, additional, Auer, Daniel, additional, Azevedo, Flavio, additional, Bahnsen, Oke, additional, Balzer, Dave, additional, Bauer, Gerrit, additional, Bauer, Paul, additional, Baumann, Markus, additional, Baute, Sharon, additional, Benoit, Verena, additional, Bernauer, Julian, additional, Berning, Carl, additional, Berthold, Anna, additional, Bethke, Felix S., additional, Biegert, Thomas, additional, Blinzler, Katharina, additional, Blumenberg, Johannes, additional, Bobzien, Licia, additional, Bohman, Andrea, additional, Bol, Thijs, additional, Bostic, Amie, additional, Brzozowska, Zuzanna, additional, Burgdorf, Katharina, additional, Burger, Kaspar, additional, Busch, Kathrin, additional, Castillo, Juan Carlos, additional, Chan, Nathan, additional, Christmann, Pablo, additional, Connelly, Roxanne, additional, Czymara, Christian S., additional, Damian, Elena, additional, Ecker, Alejandro, additional, Edelmann, Achim, additional, Eger, Maureen A., additional, Ellerbrock, Simon, additional, Forke, Anna, additional, Forster, Andrea Gabriele, additional, Gaasendam, Chris, additional, Gavras, Konstantin, additional, Gayle, Vernon, additional, Gessler, Theresa, additional, Gnambs, Timo, additional, Godefroidt, Amélie, additional, Grömping, Max, additional, Groß, Martin, additional, Gruber, Stefan, additional, Gummer, Tobias, additional, Hadjar, Andreas, additional, Heisig, Jan Paul, additional, Hellmeier, Sebastian, additional, Heyne, Stefanie, additional, Hirsch, Magdalena, additional, Hjerm, Mikael, additional, Hochman, Oshrat, additional, Hövermann, Andreas, additional, Hunger, Sophia, additional, Hunkler, Christian, additional, Huth-Stöckle, Nora, additional, Ignacz, Zsofia, additional, Jacobs, Laura, additional, Jacobsen, Jannes, additional, Jaeger, Bastian, additional, Jungkunz, Sebastian, additional, Jungmann, Nils, additional, Kauff, Mathias, additional, Kleinert, Manuel, additional, Klinger, Julia, additional, Kolb, Jan-Philipp, additional, Kołczyńska, Marta, additional, Kuk, John Seungmin, additional, Kunißen, Katharina, additional, Sinatra, Dafina Kurti, additional, Greinert, Alexander, additional, Lersch, Philipp M., additional, Löbel, Lea-Maria, additional, Lutscher, Philipp, additional, Mader, Matthias, additional, Madia, Joan Eliel, additional, Malancu, Natalia, additional, Maldonado, Luis, additional, Marahrens, Helge, additional, Martin, Nicole, additional, Martinez, Paul, additional, Mayerl, Jochen, additional, MAYORGA, OSCAR Jose, additional, McManus, Patricia A., additional, Wagner, Kyle, additional, Meeusen, Cecil, additional, Meierrieks, Daniel, additional, Mellon, Jonathan, additional, Merhout, Friedolin, additional, Merk, Samuel, additional, Meyer, Daniel, additional, Micheli, Leticia, additional, Mijs, Jonathan J.B., additional, Moya, Cristóbal, additional, Neunhoeffer, Marcel, additional, Nüst, Daniel, additional, Nygård, Olav, additional, Ochsenfeld, Fabian, additional, Otte, Gunnar, additional, Pechenkina, Anna, additional, Prosser, Christopher, additional, Raes, Louis, additional, Ralston, Kevin, additional, Ramos, Miguel, additional, Roets, Arne, additional, Rogers, Jonathan, additional, Ropers, Guido, additional, Samuel, Robin, additional, Sand, Gergor, additional, Schachter, Ariela, additional, Schaeffer, Merlin, additional, Schieferdecker, David, additional, Schlueter, Elmar, additional, Schmidt, Katja, additional, Schmidt, Regine, additional, Schmidt-Catran, Alexander, additional, Schmiedeberg, Claudia, additional, Schneider, Jürgen, additional, Schoonvelde, Martijn, additional, Schulte-Cloos, Julia, additional, Schumann, Sandy, additional, Schunck, Reinhard, additional, Schupp, Juergen, additional, Seuring, Julian, additional, Silber, Henning, additional, Sleegers, Willem W. A., additional, Sonntag, Nico, additional, Staudt, Alexander, additional, Steiber, Nadia, additional, Steiner, Nils, additional, Sternberg, Sebastian, additional, Stiers, Dieter, additional, Stojmenovska, Dragana, additional, Storz, Nora, additional, Striessnig, Erich, additional, Stroppe, Anne-Kathrin, additional, Teltemann, Janna, additional, Tibajev, Andrey, additional, Tung, Brian B., additional, Vagni, Giacomo, additional, Van Assche, Jasper, additional, van der Linden, Meta, additional, van der Noll, Jolanda, additional, Van Hootegem, Arno, additional, Vogtenhuber, Stefan, additional, Voicu, Bogdan, additional, Wagemans, Fieke, additional, Wehl, Nadja, additional, Werner, Hannah, additional, Wiernik, Brenton M., additional, Winter, Fabian, additional, Wolf, Christof, additional, Yamada, Yuki, additional, Zhang, Nan, additional, Ziller, Conrad, additional, Zins, Stefan, additional, Żółtak, Tomasz, additional, and Nguyen, Hung Hoang Viet, additional

Published
2021
Full Text
View/download PDF

29. The European Values Study 2017: On the Way to the Future Using Mixed-Modes

Author

Luijkx, Ruud, primary, Jónsdóttir, Guðbjörg Andrea, additional, Gummer, Tobias, additional, Ernst Stähli, Michèle, additional, Frederiksen, Morten, additional, Ketola, Kimmo, additional, Reeskens, Tim, additional, Brislinger, Evelyn, additional, Christmann, Pablo, additional, Gunnarsson, Stefán Þór, additional, Hjaltason, Árni Bragi, additional, Joye, Dominique, additional, Lomazzi, Vera, additional, Maineri, Angelica M, additional, Milbert, Patricia, additional, Ochsner, Michael, additional, Pollien, Alexandre, additional, Sapin, Marlène, additional, Solanes, Ivet, additional, Verhoeven, Sascha, additional, and Wolf, Christof, additional

Published
2020
Full Text
View/download PDF

30. Fieldwork Monitoring in Practice: Insights from 17 Large-scale Social Science Surveys in Germany

Author

Leerstoel Schoot, Methodology and statistics for the behavioural and social sciences, Meitinger, K.M., Stadtmüller, Sven, Silber, Henning, Auriga, Roman, Bergmann, Michael, Blohm, Michael, Blumenberg, Manuela, Christmann, Pablo, Felderer, Barbara, Frodermann, Corinna, Griese, Florian, Gummer, Tobias, Hähnel, Sascha, Koch, Achim, Kottwitz, Anita, Krell, Kristina, Krieger, Ulrich, Liebau, Elisabeth, Martin, Silke, Müller-Kuller, Andre, Rammstedt, Beatrice, Schaurer, Ines, Scherpenzeel, Annette, Schmiedeberg, Claudia, Schmidt, Tobias, Schnaudt, Christian, Zabal, Anouk, Leerstoel Schoot, Methodology and statistics for the behavioural and social sciences, Meitinger, K.M., Stadtmüller, Sven, Silber, Henning, Auriga, Roman, Bergmann, Michael, Blohm, Michael, Blumenberg, Manuela, Christmann, Pablo, Felderer, Barbara, Frodermann, Corinna, Griese, Florian, Gummer, Tobias, Hähnel, Sascha, Koch, Achim, Kottwitz, Anita, Krell, Kristina, Krieger, Ulrich, Liebau, Elisabeth, Martin, Silke, Müller-Kuller, Andre, Rammstedt, Beatrice, Schaurer, Ines, Scherpenzeel, Annette, Schmiedeberg, Claudia, Schmidt, Tobias, Schnaudt, Christian, and Zabal, Anouk

Published
2020

31. The Crowdsourced Replication Initiative: Investigating Immigration and Social Policy Preferences. Executive Report

Author

Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Adem, Muna, Adriaans, Jule, Alvarez-Benjumea, Amalia, Andersen, Henrik, Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Balzer, Dave, Bauer, Gerrit, Bauer, Paul, Baumann, Markus, Baute, Sharon, Benoit, Verena, Berning, Carl, Bernauer, Julian, Berthold, Anna, Bethke, Felix, Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes, Bol, Thijs, Bobzien, Licia, Bohman, Andrea, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin, Castillo, Juan Carlos, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian, Damian, Elena, Edelmann, Achim, Ecker, Alejandro, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Gavras, Konstantin, Gayle, Vernon, Gaasendam, Chris, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Greinert, Alexander, Groß, Martin, Grömping, Max, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Hövermann, Andreas, Huth, Nora, Hunger, Sophia, Hunkler, Christian, Ignacz, Zsofia, Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kauff, Mathias, Kleinert, Manuel, Klinger, Julia, Kolb, Jan-Philipp, Kolczynska, Marta, Kuk, John, Kunißen, Katharina, Kurti, Dafina, Lersch, Philipp M., Löbel, Lea-Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan Eliel, Malancu, Natalia Cornelia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, MAYORGA, Oscar Jose, McManus, Patricia, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan, Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna, Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel, Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gergor, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Katja, Schmidt, Regine, Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Juergen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stiers, Dieter, Striessnig, Erich, Stojmenovska, Dragana, Storz, Nora, Stroppe, Anne-Kathrin, Teltemann, Janna, Tibajev, Andrey, Tung, Brian B., Vagni, Giacomo, Van Assche, Jasper, van der Linden, Meta, van der Noll, Jolanda, Van Hootegem, Arno, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke Maria Antoinet, Wagner, Kyle, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Zakula, Björn, Ziller, Conrad, Zins, Stefan, Zhang, Nan, and Żółtak, Tomasz

Subjects
Sociologie & sciences sociales [H10] [Sciences sociales & comportementales, psychologie], Open Science, Sociology & social sciences [H10] [Social & behavioral sciences, psychology], Data Science, Replication, Crowdsourcing, Immigration, Social Policy
Abstract

In an era of mass migration, social scientists, populist parties and social movements raise concerns over the future of immigration-destination societies. What impacts does this have on policy and social solidarity? Comparative cross-national research, relying mostly on secondary data, has findings in different directions. There is a threat of selective model reporting and lack of replicability. The heterogeneity of countries obscures attempts to clearly define data-generating models. P-hacking and HARKing lurk among standard research practices in this area.This project employs crowdsourcing to address these issues. It draws on replication, deliberation, meta-analysis and harnessing the power of many minds at once. The Crowdsourced Replication Initiative carries two main goals, (a) to better investigate the linkage between immigration and social policy preferences across countries, and (b) to develop crowdsourcing as a social science method. The Executive Report provides short reviews of the area of social policy preferences and immigration, and the methods and impetus behind crowdsourcing plus a description of the entire project. Three main areas of findings will appear in three papers, that are registered as PAPs or in process.

Published
2019

32. The Crowdsourced Replication Initiative: Investigating Immigration and Social Policy Preferences. Executive Report.

Author

Breznau, Nate, primary, Rinke, Eike Mark, additional, Wuttke, Alexander, additional, Adem, Muna, additional, Adriaans, Jule, additional, Alvarez-Benjumea, Amalia, additional, Andersen, Henrik Kenneth, additional, Auer, Daniel, additional, Azevedo, Flavio, additional, Bahnsen, Oke, additional, Balzer, Dave, additional, Bauer, Gerrit, additional, Bauer, Paul, additional, Baumann, Markus, additional, Baute, Sharon, additional, Benoit, Verena, additional, Berning, Carl, additional, Bernauer, Julian, additional, Berthold, Anna, additional, Bethke, Felix, additional, Biegert, Thomas, additional, Blinzler, Katharina, additional, Blumenberg, Johannes, additional, Bol, Thijs, additional, Bobzien, Licia, additional, Bohman, Andrea, additional, Bostic, Amie, additional, Brzozowska, Zuzanna, additional, Burgdorf, Katharina, additional, Burger, Kaspar, additional, Busch, Kathrin, additional, Castillo, Juan Carlos, additional, Chan, Nathan, additional, Christmann, Pablo, additional, Connelly, Roxanne, additional, Czymara, Christian S., additional, Damian, Elena, additional, Edelmann, Achim, additional, Ecker, Alejandro, additional, Eger, Maureen A., additional, Ellerbrock, Simon, additional, Forke, Anna, additional, Forster, Andrea Gabriele, additional, Gavras, Konstantin, additional, Gayle, Vernon, additional, Gaasendam, Chris, additional, Gessler, Theresa, additional, Gnambs, Timo, additional, Godefroidt, Amélie, additional, Greinert, Alexander, additional, Groß, Martin, additional, Grömping, Max, additional, Gruber, Stefan, additional, Gummer, Tobias, additional, Hadjar, Andreas, additional, Heisig, Jan Paul, additional, Hellmeier, Sebastian, additional, Heyne, Stefanie, additional, Hirsch, Magdalena, additional, Hjerm, Mikael, additional, Hochman, Oshrat, additional, Höffler, Jan H., additional, Hövermann, Andreas, additional, Huth, Nora, additional, Hunger, Sophia, additional, Hunkler, Christian, additional, Ignacz, Zsofia, additional, Jacobs, Laura, additional, Jacobsen, Jannes, additional, Jaeger, Bastian, additional, Jungkunz, Sebastian, additional, Jungmann, Nils, additional, Kauff, Mathias, additional, Kleinert, Manuel, additional, Klinger, Julia, additional, Kolb, Jan-Philipp, additional, Kolczynska, Marta, additional, Kuk, John, additional, Kunißen, Katharina, additional, Kurti, Dafina, additional, Lersch, Philipp M., additional, Löbel, Lea-Maria, additional, Lutscher, Philipp, additional, Mader, Matthias, additional, Madia, Joan Eliel, additional, Malancu, Natalia Cornelia, additional, Maldonado, Luis, additional, Marahrens, Helge, additional, Martin, Nicole, additional, Martinez, Paul, additional, Mayerl, Jochen, additional, MAYORGA, OSCAR Jose, additional, McManus, Patricia A., additional, Meeusen, Cecil, additional, Meierrieks, Daniel, additional, Mellon, Jonathan, additional, Merhout, Friedolin, additional, Merk, Samuel, additional, Meyer, Daniel, additional, Micheli, Leticia, additional, Mijs, Jonathan, additional, Moya, Cristóbal, additional, Neunhoeffer, Marcel, additional, Nüst, Daniel, additional, Nygård, Olav, additional, Ochsenfeld, Fabian, additional, Otte, Gunnar, additional, Pechenkina, Anna, additional, Prosser, Christopher, additional, Raes, Louis, additional, Ralston, Kevin, additional, Ramos, Miguel, additional, Roets, Arne, additional, Rogers, Jonathan, additional, Ropers, Guido, additional, Samuel, Robin, additional, Sand, Gergor, additional, Schachter, Ariela, additional, Schaeffer, Merlin, additional, Schieferdecker, David, additional, Schlueter, Elmar, additional, Schmidt, Katja, additional, Schmidt, Regine, additional, Schmidt-Catran, Alexander, additional, Schmiedeberg, Claudia, additional, Schneider, Jürgen, additional, Schoonvelde, Martijn, additional, Schulte-Cloos, Julia, additional, Schumann, Sandy, additional, Schunck, Reinhard, additional, Schupp, Juergen, additional, Seuring, Julian, additional, Silber, Henning, additional, Sleegers, Willem W. A., additional, Sonntag, Nico, additional, Staudt, Alexander, additional, Steiber, Nadia, additional, Steiner, Nils, additional, Sternberg, Sebastian, additional, Stiers, Dieter, additional, Striessnig, Erich, additional, Stojmenovska, Dragana, additional, Storz, Nora, additional, Stroppe, Anne-Kathrin, additional, Teltemann, Janna, additional, Tibajev, Andrey, additional, Tung, Brian B., additional, Vagni, Giacomo, additional, Van Assche, Jasper, additional, van der Linden, Meta, additional, van der Noll, Jolanda, additional, Van Hootegem, Arno, additional, Vogtenhuber, Stefan, additional, Voicu, Bogdan, additional, Wagemans, Fieke Maria Antoinet, additional, Wagner, Kyle, additional, Wehl, Nadja, additional, Werner, Hannah, additional, Wiernik, Brenton M., additional, Winter, Fabian, additional, Wolf, Christof, additional, Yamada, Yuki, additional, Zakula, Björn, additional, Ziller, Conrad, additional, Zins, Stefan, additional, Zhang, Nan, additional, Żółtak, Tomasz, additional, and Nguyen, Hung Hoang Viet, additional

Published
2019
Full Text
View/download PDF

34. Satisfaction with democracy as the evaluation of institutional structure, democratic process and economic performance: a comparative panel study on the sources of political support

Author

Christmann, Pablo, Torcal, Mariano, and Universitat Pompeu Fabra. Departament de Ciències Polítiques i Socials

Subjects
Satisfaction with democracy, Corruption, Party system, Government fractionalization, Economic crisis, Electoral system, Economic performance, Democratic quality
Abstract

This thesis analyses the sources of public evaluations of democratic regimes from a comparative perspective, a longitudinal perspective and with a case study of Spain. The main argument of the dissertation is that people’s satisfaction with democracy (SWD) depends to a great degree on the quality of democratic processes and the institutional context. This conclusion does not deny the relevance of the well-documented relationship between the status of the economy and SWD. The importance of the economy is clearly confirmed by the current dissertation. Instead, this thesis argues that economic factors complement other political explanations. As will be shown, the long-term effects of the economy on SWD even depend on the quality of democratic processes and vice versa. This argument is presented in three related studies. In the first study, based on time-series cross-sectional (TSCS) data from 58 democracies between 1990 and 2012 and pooled surveys from the Comparative Study of Electoral Systems, I show that although electoral proportionality increases SWD, fractionalized party systems and coalition governments decrease SWD. In the second study, based on a twofold longitudinal panel analysis of the Spanish case between 1985 and 2015, I show that despite the negative consequences of prolonged economic crisis, political factors, most notably corruption, are equally relevant to explain the decline of SWD within this context. The third article, based on TSCS panel data from 61 democracies between 1980 and 2014, shows that economic performance and democratic quality are equally important explanations to account for both cross-national and cross-time differences in SWD and that both factors are interrelated in their effects on SWD.

Published
2017

37. Congruence, national context and trust in European institutions.

Author

Torcal, Mariano and Christmann, Pablo

Subjects
*GEOMETRIC congruences, *PANEL analysis, *RECESSIONS, *TRUST, *FINANCIAL crises, *POLITICAL trust (in government)
Abstract

How can we explain the recent decline of trust in representative institutions of the EU in many of its Member States? This article presents evidence supporting the congruence hypothesis, according to which citizens have been extrapolating their increasing distrust in national institutions to the EU institutions. We also find that these contagion effects are produced by citizens' evaluations of national governments. Furthermore, we show that these spillover effects from the national to the EU level tend to be stronger in situations of economic recession and political crisis. The only counterbalance to this contagion comes from citizens' positive evaluation of EU performance. We test these general arguments based on a twofold panel analysis of the Spanish case, a country that has suffered a remarkable deterioration of political trust in a context of profound economic and political crisis, by analysing data from a micro-level panel study and 28 pooled surveys from the Eurobarometer between 1999 and 2015. [ABSTRACT FROM AUTHOR]

Published
2019
Full Text
View/download PDF

38. The Effects of Government System Fractionalization on Satisfaction With Democracy*

Author

Christmann, Pablo and Torcal, Mariano

Abstract

Consensual-pluralistic institutional features of representative democracies have traditionally been associated with satisfaction with democracy (SWD). However, more recent studies report contradictory results on the effects of some of these institutional determinants on SWD. This article confirms these puzzling findings by showing that electoral proportionality increases SWD while other pluralistic factors such as government fractionalization produce the opposite effect. We illustrate this duality of counteracting effects by expanding the number of cases under study to different regions of the world in a comprehensive time-series cross-sectional sample of 58 democracies between 1990 and 2012. In the second part of the paper, we are able to reconfirm these findings at the individual level by employing survey data from the Comparative Study of Electoral Systems.

Published
2018
Full Text
View/download PDF

39. The Crowdsourced Replication Initiative: Investigating Immigration and Social Policy Preferences. Executive Report

Author

Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Adem, Muna, Adriaans, Jule, Alvarez-Benjumea, Amalia, Andersen, Henrik, Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Balzer, Dave, Bauer, Gerrit, Bauer, Paul, Baumann, Markus, Baute, Sharon, Benoit, Verena, Berning, Carl, Bernauer, Julian, Berthold, Anna, Bethke, Felix, Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes, Bol, Thijs, Bobzien, Licia, Bohman, Andrea, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin, Castillo, Juan Carlos, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian, Damian, Elena, Edelmann, Achim, Ecker, Alejandro, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Gavras, Konstantin, Gayle, Vernon, Gaasendam, Chris, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Greinert, Alexander, Groß, Martin, Grömping, Max, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Hövermann, Andreas, Huth, Nora, Hunger, Sophia, Hunkler, Christian, Ignacz, Zsofia, Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kauff, Mathias, Kleinert, Manuel, Klinger, Julia, Kolb, Jan-Philipp, Kolczynska, Marta, Kuk, John, Kunißen, Katharina, Kurti, Dafina, Lersch, Philipp M., Löbel, Lea-Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan Eliel, Malancu, Natalia Cornelia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar Jose, Mcmanus, Patricia, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan, Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna, Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel, Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gergor, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Katja, Schmidt, Regine, Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Juergen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stiers, Dieter, Striessnig, Erich, Stojmenovska, Dragana, Storz, Nora, Stroppe, Anne-Kathrin, Teltemann, Janna, Tibajev, Andrey, Tung, Brian B., Vagni, Giacomo, Assche, Jasper, Linden, Meta, Noll, Jolanda, Hootegem, Arno, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke Maria Antoinet, Wagner, Kyle, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Zakula, Björn, Ziller, Conrad, Zins, Stefan, Zhang, Nan, and Tomasz Żółtak

40. Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

Author

Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Nguyen, Hung H.V., Adem, Muna, Adriaans, Jule, Alvarez-Benjumea, Amalia, Andersen, Henrik K., Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Balzer, Dave, Bauer, Gerrit, Bauer, Paul C., Baumann, Markus, Baute, Sharon, Benoit, Verena, Bernauer, Julian, Berning, Carl, Berthold, Anna, Bethke, Felix S., Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes N., Bobzien, Licia, Bohman, Andrea, Bol, Thijs, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin B., Carlos-Castillo, Juan, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian S., Damian, Elena, Ecker, Alejandro, Edelmann, Achim, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Gaasendam, Chris, Gavras, Konstantin, Gayle, Vernon, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Grömping, Max, Groß, Martin, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Hövermann, Andreas, Hunger, Sophia, Hunkler, Christian, Huth, Nora, Ignácz, Zsófia S., Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kauff, Mathias, Kleinert, Manuel, Klinger, Julia, Kolb, Jan Philipp, Kołczyńska, Marta, Kuk, John, Kunißen, Katharina, Kurti Sinatra, Dafina, Langenkamp, Alexander, Lersch, Philipp M., Löbel, Lea Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan E., Malancu, Natalia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar J., McManus, Patricia, McWagner, Kyle, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan, Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna O., Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel R., Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gregor, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Regine, Schmidt, Katja M., Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Jürgen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stiers, Dieter, Stojmenovska, Dragana, Storz, Nora, Striessnig, Erich, Stroppe, Anne Kathrin, Teltemann, Janna, Tibajev, Andrey, Tung, Brian, Vagni, Giacomo, Van Assche, Jasper, van der Linden, Meta, van der Noll, Jolanda, Van Hootegem, Arno, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Yamada, Yuki, Zhang, Nan, Ziller, Conrad, Zins, Stefan, Żółtak, Tomasz, Breznau, Nate, Rinke, Eike Mark, Wuttke, Alexander, Nguyen, Hung H.V., Adem, Muna, Adriaans, Jule, Alvarez-Benjumea, Amalia, Andersen, Henrik K., Auer, Daniel, Azevedo, Flavio, Bahnsen, Oke, Balzer, Dave, Bauer, Gerrit, Bauer, Paul C., Baumann, Markus, Baute, Sharon, Benoit, Verena, Bernauer, Julian, Berning, Carl, Berthold, Anna, Bethke, Felix S., Biegert, Thomas, Blinzler, Katharina, Blumenberg, Johannes N., Bobzien, Licia, Bohman, Andrea, Bol, Thijs, Bostic, Amie, Brzozowska, Zuzanna, Burgdorf, Katharina, Burger, Kaspar, Busch, Kathrin B., Carlos-Castillo, Juan, Chan, Nathan, Christmann, Pablo, Connelly, Roxanne, Czymara, Christian S., Damian, Elena, Ecker, Alejandro, Edelmann, Achim, Eger, Maureen A., Ellerbrock, Simon, Forke, Anna, Forster, Andrea, Gaasendam, Chris, Gavras, Konstantin, Gayle, Vernon, Gessler, Theresa, Gnambs, Timo, Godefroidt, Amélie, Grömping, Max, Groß, Martin, Gruber, Stefan, Gummer, Tobias, Hadjar, Andreas, Heisig, Jan Paul, Hellmeier, Sebastian, Heyne, Stefanie, Hirsch, Magdalena, Hjerm, Mikael, Hochman, Oshrat, Hövermann, Andreas, Hunger, Sophia, Hunkler, Christian, Huth, Nora, Ignácz, Zsófia S., Jacobs, Laura, Jacobsen, Jannes, Jaeger, Bastian, Jungkunz, Sebastian, Jungmann, Nils, Kauff, Mathias, Kleinert, Manuel, Klinger, Julia, Kolb, Jan Philipp, Kołczyńska, Marta, Kuk, John, Kunißen, Katharina, Kurti Sinatra, Dafina, Langenkamp, Alexander, Lersch, Philipp M., Löbel, Lea Maria, Lutscher, Philipp, Mader, Matthias, Madia, Joan E., Malancu, Natalia, Maldonado, Luis, Marahrens, Helge, Martin, Nicole, Martinez, Paul, Mayerl, Jochen, Mayorga, Oscar J., McManus, Patricia, McWagner, Kyle, Meeusen, Cecil, Meierrieks, Daniel, Mellon, Jonathan, Merhout, Friedolin, Merk, Samuel, Meyer, Daniel, Micheli, Leticia, Mijs, Jonathan, Moya, Cristóbal, Neunhoeffer, Marcel, Nüst, Daniel, Nygård, Olav, Ochsenfeld, Fabian, Otte, Gunnar, Pechenkina, Anna O., Prosser, Christopher, Raes, Louis, Ralston, Kevin, Ramos, Miguel R., Roets, Arne, Rogers, Jonathan, Ropers, Guido, Samuel, Robin, Sand, Gregor, Schachter, Ariela, Schaeffer, Merlin, Schieferdecker, David, Schlueter, Elmar, Schmidt, Regine, Schmidt, Katja M., Schmidt-Catran, Alexander, Schmiedeberg, Claudia, Schneider, Jürgen, Schoonvelde, Martijn, Schulte-Cloos, Julia, Schumann, Sandy, Schunck, Reinhard, Schupp, Jürgen, Seuring, Julian, Silber, Henning, Sleegers, Willem, Sonntag, Nico, Staudt, Alexander, Steiber, Nadia, Steiner, Nils, Sternberg, Sebastian, Stiers, Dieter, Stojmenovska, Dragana, Storz, Nora, Striessnig, Erich, Stroppe, Anne Kathrin, Teltemann, Janna, Tibajev, Andrey, Tung, Brian, Vagni, Giacomo, Van Assche, Jasper, van der Linden, Meta, van der Noll, Jolanda, Van Hootegem, Arno, Vogtenhuber, Stefan, Voicu, Bogdan, Wagemans, Fieke, Wehl, Nadja, Werner, Hannah, Wiernik, Brenton M., Winter, Fabian, Wolf, Christof, Yamada, Yuki, Zhang, Nan, Ziller, Conrad, Zins, Stefan, and Żółtak, Tomasz

Abstract

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

Catalog

Books, media, physical & digital resources
See catalog results