Search

Open research practices seek to enhance the transparency and reproducibility of research. While there is evidence of increased uptake in these practices, such as study preregistration and open data, facilitated by new infrastructure and policies, little research has assessed general uptake of such practices across psychology university researchers. The current study estimates psychologists' level of engagement in open research practices across universities in the United Kingdom and Ireland, while also assessing possible explanatory factors that may impact their engagement. Data were collected from 602 psychology researchers in the United Kingdom and Ireland on the extent to which they have implemented various practices (e.g., use of preprints, preregistration, open data, open materials). Here we present the summarized descriptive results, as well as considering differences between various categories of researcher (e.g., career stage, subdiscipline, methodology), and examining the relationship between researcher's practices and their self‐reported capability, opportunity, and motivation (COM‐B) to engage in open research practices. Results show that while there is considerable variability in engagement of open research practices, differences across career stage and subdiscipline of psychology are small by comparison. We observed consistent differences according to respondent's research methodology and based on the presence of institutional support for open research. COM‐B dimensions were collectively significant predictors of engagement in open research, with automatic motivation emerging as a consistently strong predictor. We discuss these findings, outline some of the challenges experienced in this study, and offer suggestions and recommendations for future research. Estimating the prevalence of responsible research practices is important to assess sustained behaviour change in research reform, tailor educational training initiatives, and to understand potential factors that might impact engagement. [ABSTRACT FROM AUTHOR]

When different researchers study the same research question using the same dataset they may obtain different and potentially even conflicting results. This is because there is often substantial flexibility in researchers' analytical choices, an issue also referred to as "researcher degrees of freedom". Combined with selective reporting of the smallest p-value or largest effect, researcher degrees of freedom may lead to an increased rate of false positive and overoptimistic results. In this paper, we address this issue by formalizing the multiplicity of analysis strategies as a multiple testing problem. As the test statistics of different analysis strategies are usually highly dependent, a naive approach such as the Bonferroni correction is inappropriate because it leads to an unacceptable loss of power. Instead, we propose using the "minP" adjustment method, which takes potential test dependencies into account and approximates the underlying null distribution of the minimal p-value through a permutation-based procedure. This procedure is known to achieve more power than simpler approaches while ensuring a weak control of the family-wise error rate. We illustrate our approach for addressing researcher degrees of freedom by applying it to a study on the impact of perioperative p a O 2 on post-operative complications after neurosurgery. A total of 48 analysis strategies are considered and adjusted using the minP procedure. This approach allows to selectively report the result of the analysis strategy yielding the most convincing evidence, while controlling the type 1 error—and thus the risk of publishing false positive results that may not be replicable. [ABSTRACT FROM AUTHOR]

Abstract When different researchers study the same research question using the same dataset they may obtain different and potentially even conflicting results. This is because there is often substantial flexibility in researchers’ analytical choices, an issue also referred to as “researcher degrees of freedom”. Combined with selective reporting of the smallest p-value or largest effect, researcher degrees of freedom may lead to an increased rate of false positive and overoptimistic results. In this paper, we address this issue by formalizing the multiplicity of analysis strategies as a multiple testing problem. As the test statistics of different analysis strategies are usually highly dependent, a naive approach such as the Bonferroni correction is inappropriate because it leads to an unacceptable loss of power. Instead, we propose using the “minP” adjustment method, which takes potential test dependencies into account and approximates the underlying null distribution of the minimal p-value through a permutation-based procedure. This procedure is known to achieve more power than simpler approaches while ensuring a weak control of the family-wise error rate. We illustrate our approach for addressing researcher degrees of freedom by applying it to a study on the impact of perioperative $$paO_2$$ p a O 2 on post-operative complications after neurosurgery. A total of 48 analysis strategies are considered and adjusted using the minP procedure. This approach allows to selectively report the result of the analysis strategy yielding the most convincing evidence, while controlling the type 1 error—and thus the risk of publishing false positive results that may not be replicable.

The experimental method is designed to secure the reliable attribution of causal relationships by means of controlled comparison across conditions. Doing so, however, depends upon the reduction of uncertainties and inconsistencies in the process of comparison; and this poses particularly significant challenges for the behavioral and social sciences because they work with human subjects, whose malleability and complexity often interact in unexpected ways with experimental manipulations, thus resulting in unpredictable behavior. Drawing on the Science and Technology Studies perspective and one of our authors' experiences in experimental work, this paper examines how experimental social scientists manage to establish objectivity and standardization in the face of vagaries arising from working with human subjects. In identifying experimental researchers' solutions to this challenge, we draw on methodological discussions among applied social scientists as naturally occurring data, through which we show how some seemingly mundane practices play essential roles in extracting patterns out of otherwise unpredictable behaviors in the lab. Closely examining such strategies, we reveal the inherent instabilities in the experimental method when adopted in the social sciences and discuss their methodological implications. In conclusion, we make tentative suggestions for escaping the kinds of methodological impasses which we have identified. [ABSTRACT FROM AUTHOR]

Abstract The .05 boundary within Null Hypothesis Statistical Testing (NHST) “has made a lot of people very angry and been widely regarded as a bad move” (to quote Douglas Adams). Here, we move past meta-scientific arguments and ask an empirical question: What is the psychological standing of the .05 boundary for statistical significance? We find that graduate students in the psychological sciences show a boundary effect when relating p-values across .05. We propose this psychological boundary is learned through statistical training in NHST and reading a scientific literature replete with “statistical significance”. Consistent with this proposal, undergraduates do not show the same sensitivity to the .05 boundary. Additionally, the size of a graduate student’s boundary effect is not associated with their explicit endorsement of questionable research practices. These findings suggest that training creates distortions in initial processing of p-values, but these might be dampened through scientific processes operating over longer timescales.

The.05 boundary within Null Hypothesis Statistical Testing (NHST) "has made a lot of people very angry and been widely regarded as a bad move" (to quote Douglas Adams). Here, we move past meta-scientific arguments and ask an empirical question: What is the psychological standing of the.05 boundary for statistical significance? We find that graduate students in the psychological sciences show a boundary effect when relating p-values across.05. We propose this psychological boundary is learned through statistical training in NHST and reading a scientific literature replete with "statistical significance". Consistent with this proposal, undergraduates do not show the same sensitivity to the.05 boundary. Additionally, the size of a graduate student's boundary effect is not associated with their explicit endorsement of questionable research practices. These findings suggest that training creates distortions in initial processing of p-values, but these might be dampened through scientific processes operating over longer timescales. [ABSTRACT FROM AUTHOR]

We argue that critical areas of memory research rely on problematic measurement practices and provide concrete suggestions to improve the situation. In particular, we highlight the prevalence of memory studies that use tasks (like the "old/new" task: "have you seen this item before? yes/no") where quantifying performance is deeply dependent on counterfactual reasoning that depends on the (unknowable) distribution of underlying memory signals. As a result of this difficulty, different literatures in memory research (e.g., visual working memory, eyewitness identification, picture memory, etc.) have settled on a variety of fundamentally different metrics to get performance measures from such tasks (e.g., A', corrected hit rate, percent correct, d', diagnosticity ratios, K values, etc.), even though these metrics make different, contradictory assumptions about the distribution of latent memory signals, and even though all of their assumptions are frequently incorrect. We suggest that in order for the psychology and neuroscience of memory to become a more cumulative, theory-driven science, more attention must be given to measurement issues. We make a concrete suggestion: The default memory task for those simply interested in performance should change from old/new ("did you see this item'?") to two-alternative forced-choice ("which of these two items did you see?"). In situations where old/new variants are preferred (e.g., eyewitness identification; theoretical investigations of the nature of memory signals), receiver operating characteristic (ROC) analysis should be performed rather than a binary old/new task.

Numerous scientific fields are facing a replication crisis, where the results of a study often cannot be replicated when a new study uses independent data. This issue has been particularly emphasized in psychology, health, and medicine, as incorrect results in these fields could have serious consequences, where lives might be at stake. While other fields have also highlighted significant replication problems, the Earth Sciences seem to be an exception. The paucity of Earth Science research aimed at understanding the replication crisis prompted this study. Specifically, this work aims to fill that gap by seeking to replicate geological results involving various types of time-series. We identify and discuss 11 key variables for replicating U-Pb age distributions: independent data, global sampling, proxy data, data quality, disproportionate non-random sampling, stratigraphic bias, potential filtering bias, accuracy and precision, correlating time-series segments, testing assumptions and divergent analytical methods, and analytical transparency. Even while this work primarily focuses on U-Pb age distributions, most of these factors (or variations of them) also apply to other geoscience disciplines. Thus, some of the discussions involve time-series consisting of εHf, δ18O-zircon, 14C, 10Be, marine δ13C, and marine δ18O. We then provide specific recommendations for minimizing adverse effects related to these factors, and in the process enhancing prospects for replicating geological results.

BackgroundIdentification of widespread biases present in reported research findings in many scientific disciplines, including psychology, such as failures to replicate and the likely extensive application of questionable research practices, has raised serious concerns over the reliability and trustworthiness of scientific research. This has led to the development of, and advocacy for, 'open science' practices, including data, materials, analysis, and output sharing, pre-registration of study predictions and analysis plans, and increased access to published research findings. Implementation of such practices has been enthusiastic in some quarters, but literacy in, and adoption of, these practices has lagged behind among many researchers in the scientific community.AdvancesIn the current article I propose that researchers adopt an open science 'mindset', a comprehensive approach to open science predicated on researchers' operating under the basic assumption that, wherever possible, open science practices will be a central component of all steps of their research projects. The primary, defining feature of the mindset is a commitment to open science principles in all research projects from inception to dissemination. Other features of the mindset include the assumption that all components of research projects (e.g. pre-registered hypotheses, protocols, materials, analysis plans, data, and output) will be accessible broadly; pro-active selection of open fora to disseminate research components and findings; open and transparent dissemination of reports of the research findings in advance of, and after, formal publication; and active promotion of open science practices through education, modeling, and advocacy.ConclusionThe open science mindset is a 'farm to fork' approach to open science aimed at promoting comprehensive quality in application of open science, and widening participation in open science practices so that they become the norm in research in health psychology and behavioral medicine going forward.

Beginning in 2011, public scandals and high‐visibility critiques of research methods in psychology fed a broader "replication crisis": foundational experiments could not be replicated, and statistical methods in social psychology demonstrated vulnerability to fraud and manipulation. Even well‐intended researchers following accepted psychological protocols—zombie methodologies—could unintentionally produce false positives. In response, social psychologists have called for greater sensitivity to cultural diversity, a deeper consideration of social context, and more methodological reflection. The contrast with anthropology is dramatic, highlighting some of the strengths of our field: methodological versatility, appreciation of human variability, theoretical creativity, and a solid foundation for synthetic, interdisciplinary collaboration grounded in our tradition of holism. The human sciences are an important audience for anthropologists, as the example of cognitive science shows. [ABSTRACT FROM AUTHOR]

Failures to replicate the results of scientific studies are often attributed to misinterpretations of the p-value. The p-value may be interpreted as an approximate posterior probability, not that the null hypothesis is true but rather that it explains the data as well as the data-generating distribution. That posterior probability modifies the p-value in the following two broad areas of application, leading to new methods of hypothesis testing and effect size estimation. First, when corrected for multiple comparisons, the posterior probability that the null hypothesis adequately explains the data overcomes both the conservative bias of corrected p-values and the anti-conservative bias of commonly used false discovery rate methods. Second, the posterior probability that the null hypothesis adequately explains the data, conditional on a parameter restriction, transforms the p-value in such a way as to overcome difficulties in restricted parameter spaces. [ABSTRACT FROM AUTHOR]

Copyright of Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Cognitive load theory has been in development since the 1980s. Much of the impetus for that development has come from firstly, replication failures using randomised controlled trials and secondly, from the incorporation of other theories into cognitive load theory. Both have led to theory expansion. The immediate cause of the so-called "replication crisis" in psychology and other disciplines is a failure to replicate previous empirical findings. Using cognitive load theory as an example, I argue that the appearance of contradictory evidence does not necessarily derive from a failure to properly collect data. Rather, it can be caused by initially insufficiently detailed theories, with increasing detail often revealing the reason for a failure to replicate. For cognitive load theory, each failure to replicate, rather than being a negative, contributed to the further development of the theory. In addition, the theory has developed over many years by closely incorporating other theories associated with human cognitive architecture and evolutionary psychology. In this paper, I discuss some of the developmental milestones associated with cognitive load theory and how they were informed by replication failures and theory integration. [ABSTRACT FROM AUTHOR]

Summary: Over the last decades, neuroimaging has become a substantial component of insomnia research. While theoretical underpinnings of different studies vary just like methodological choices and the experimental design, it is suggested that major features of insomnia disorder rely on the impaired function, structure, metabolism and connectivity of brain areas involved in sleep generation, emotion regulation, self‐processing/‐awareness and attentional orientation. However, neuroimaging research on insomnia often suffers from small sample sizes, heterogeneous methodology and a lack of replicability. With respect to these issues, the field needs to address the questions: (1a) how sufficiently large sample sizes can be accumulated within a reasonable economic framework; (1b) how effect sizes in insomnia‐related paradigms can be amplified; (2a) how a higher degree of standardisation and transparency in methodology can be provided; and (2b) how an adequate amount of flexibility/complexity in study design can be maintained. On condition that methodological consistency and a certain degree of adaptability are given, pooled data/large cohort analyses can be considered to be one way to answer these questions. Regarding experimental single‐centre trials, it might be helpful to focus on insomnia‐related transdiagnostic concepts. In doing so, expectable effect sizes (in between‐subjects designs) can be increased by: (a) comparing groups that are truly distinct regarding the variables examined in a concept‐specific paradigm; and (b) facilitated, intensified and precise elicitation of a target symptom. [ABSTRACT FROM AUTHOR]

Journals are the product of intellectual communities; a publication's health mirrors the field it represents. Current developments in the human sciences, especially the replication crisis and growing awareness of problems with cross‐cultural generalization, create opportunities for psychological anthropologists to speak to a broader audience within academia. However, to take advantage of these opportunities, we must write with this potential audience in mind, an academic public that does not share the same theoretical vocabulary, epistemological standards, or methodological principles. Academic publishing has been struggling with the burdens imposed by the pandemic and scandals around misuse of power brought to light in the last few years. At the same time, new models for performance management only increase the pressure falling on editorial staff. This editorial reflects on the necessity of solidarity, innovation, and community investment in a journal to maintain viability in the new publishing landscape. [ABSTRACT FROM AUTHOR]

To mathematize is closely related to the ideal of scientific objectivity, claiming bias-free results. In this respect, mathematization may substantiate the belief that significance testing could prove or disprove a hypothesis, since it is considered bias-free. However, this belief overlooks the fact that scientific objectivity is an epistemic virtue. Additionally, reducing human activity to numbers inevitably involves simplifying the real world, which includes uncertainty. Although statistics is valuable to psychology and science, it may be vulnerable to ontological reductionism.

Objective: Much of psychological research has suffered from small sample sizes and low statistical power, resulting in unstable parameter estimates. The Bayesian approach offers a promising solution by incorporating prior knowledge into statistical models, which may lead to improved stability compared to a frequentist approach. Methods: Simulated data from four populations with known bivariate correlations (ρ = 0.1, 0.2, 0.3, 0.4) was used to estimate the sample correlation as samples were sequentially added from the population, from n = 10 to n = 500. The impact of three different, subjectively defined prior distributions (weakly, moderately, and highly informative) was investigated and compared to a frequentist model. Results: The results show that bivariate correlation estimates are unstable, and that the risk of obtaining an estimate that is exaggerated or in the wrong direction is relatively high, for sample sizes for below 100, and considerably so for sample sizes below 50. However, this instability can be constrained by informative Bayesian priors. Conclusion: Informative Bayesian priors have the potential to significantly reduce sample size requirements and help ensure that obtained estimates are in line with realistic expectations. The combined stabilizing and regularizing effect of a weakly informative prior is particularly useful when conducting research with small samples. The impact of more informative Bayesian priors depends on one's threshold for probability and whether one's goal is to obtain an estimate merely in the correct direction, or to obtain a high precision estimate whose associated interval falls within a narrow range. Implications for sample size requirements and directions for future research are discussed. [ABSTRACT FROM AUTHOR]

While I was a Russell Sage resident located in the basement of Cubberly Hall at Stanford for 2 years, I was invited, "persuaded" to design social psychology experiments on two occasions. I learned a great deal from both experiences. In one, I needed to temporarily raise or lower participants' "self-esteem" just prior to their receiving subtle suggestions. In the other, we attempted to influence participants acceptance and use of the "Johnny Rocco" sad story. Both experiences taught me the importance of generating a compelling scenario for the participants: "good theater." Ever since, in various ways, I have been complaining about the attempts to study the justice motive using "top of the head" methods such as simulations and role playing. Experiments using those methods could not reproduce the compelling experiences and consequences we reported in our early experiments on the justice motive and belief in a just world. More recently, I put together the work of Langer (in: Harvey, Ickes, Kidder (eds) New directions in attribution research, Erlbaum, Potomac, 1978) who compellingly described how people in the normal course of events respond automatically, thoughtlessly, in familiar encounters, and cognitive theory exploring two distinct processes: Kahneman termed them System 1 (fast) intuitive cognitions and System 2 (slow), thoughtful controlled processes. This scholarship provides insight into how context shapes justice judgments. Finally, Bazerman et al. (Curr Dir Psychol Sci 4:39–42, 1995) in a series of experiments revealed how subtle cues can lead participants to shift from Type 1 justice preferences to Type 2, more rational norm-based judgments. [ABSTRACT FROM AUTHOR]

We provide empirical evidence of research practices that can undermine the credibility of empirical research in agricultural economics. We find that, in four popular journals, most studies are underpowered, a characteristic that, when coupled with publication bias, yields unreliable and exaggerated effect sizes. This problem is exacerbated by selective reporting of statistically significant results and multiple hypothesis testing without any adjustments to statistical inferences. Survey respondents self‐report engaging in practices that are consistent with these empirical findings. Addressing these problems in agricultural economics will require that editors, reviewers, and donors change the norms and incentives for authors. [ABSTRACT FROM AUTHOR]

An important aim of research on bilingualism is to understand how the brain adapts to the demands of using more than one language.In this paper, we argue that pursuing such an aim entails valuing our research as a discovery process that acts on variety.Prescriptions about sample size and methodology, rightly aimed at establishing a sound basis for generalization, should be understood as being in the service of science as a discovery process. We propose and illustrate by drawing from previous and contemporary examples within brain and cognitive sciences, that this necessitates exploring the neural bases of bilingual phenotypes:the adaptive variety induced through the interplay of biology and culture. We identify the conceptual and methodological prerequisites for such exploration and briefly allude to the publication practices that afford it as a community practice and to the risk of allowing methodological prescriptions, rather than discovery, to dominate the research endeavor.

Abstract Canadian policymakers are interested in determining whether farmed Atlantic salmon, frequently infected with Piscine orthoreovirus (PRV), may threaten wild salmon populations in the Pacific Northwest. A relevant work has been published in BMC Biology by Polinksi and colleagues, but their conclusion that PRV has a negligible impact on the energy expenditure and respiratory performance of sockeye salmon is disputed by Mordecai and colleagues, whose re-analysis is presented in a correspondence article. So, what is the true effect and what should follow this unresolved dispute? We suggest a ‘registered multi-lab replication with adversaries’.

Failed replications can jeopardize public trust in psychological science and recent findings cast doubt on the idea that self-corrections and reforms can rebuild this trust. These findings are in contrast to trust repair research that proposes changes in transparency, norms, and policies as trust repair mechanisms. This raises the question of whether the used experimental material is one reason behind these unexpected findings. Previous studies used short texts that may give too little information on the replication crisis and initiated reforms in the field. In a pre-registered experiment (N = 390), we, therefore, tested whether comprehensive information about the replication crisis and reforms increases public trust in psychology, compared to a control condition that only informs about the replication crisis. To give comprehensive information, we created an animated video for each experimental condition. After watching the video, participants indicated their trust in researchers, trust in past research findings, and trust in current research findings. As expected and in line with trust repair research, information about reforms increased trust in researchers and in current (vs. past) research, compared with information about the replication crisis and its causes only. We discuss the generalizability of our results and implications for communicating the replication crisis to the public.

‘First impressions’ are a popular topic in social psychology. They are researched because the initial judgments of others are consequential in everyday life (such as job interviews, first dates, justice outcomes). In the context of broader concerns about the credibility of psychological science, first impressions research has developed commendable initiatives for improving reliability (open stimulus databases, international collaborations, replication studies and reanalyses). However, these initiatives can impact the validity of studying how people form first impressions. There is a long history of critiquing the usefulness of passive-observer judgments of controlled, reduced, presentations of people—and these concerns are still relevant today. Here, we highlight the praiseworthy practices improving reliability in first impressions research, before moving on to identify persistent methodological concerns in the field. This includes inadequate stimulus sampling and diversity, constrained participant response options, limited consideration of study context, and limitations of atomised presentations of target people. We identify how these methodological limitations impact theory development, how we might be over/underestimating everyday experience, and even misunderstanding social differences in autism and mental health. Finally, we identify opportunities for methodological reform, focusing on codifying instead of controlling interactions, promoting inductive, participant-led, methodologies, and asking for stronger theory development and clarity on ‘can’ vs. ‘do’ research questions. Overall, we praise reforms for improving the reliability of first impressions research, but improvements to making scientific predictions about first impressions require renewed consideration of validity.

A decade ago, replications were typically not conducted and appreciated in social psychology, although replications play a central role in ensuring trust in scientific fields. Without systematic replication efforts, it is not clear whether findings are trustworthy. As journals can function as gatekeepers for publications, they can influence whether researchers conduct (and publish) replications. Yet, the scholarly culture in social psychology might have changed over the last decade because numerous highly visible studies did not replicate past findings. In light of these insights and the resulting learning opportunities for the field, we predicted an increase in the expressed support for replications in the policies of social psychology journals from 2015 (i.e., the year the replication problem became widely known) to 2022. We coded whether and how replications were mentioned in the author guidelines on the websites of social psychology journals (N = 51). As expected, replications were welcomed more often in 2022 (25%) than they were in 2015 (12%), but they were not mentioned on the websites of most journals (71% in 2022 vs. 82% in 2015). An exploratory analysis suggested that journals that expressed support for replications on their websites were also more likely to publish articles about replication. Further, exploratory analyses of the journals’ TOP factors indicated similar rates of support for replications as for other rigor and transparency promoting policies. In sum, our findings suggest that appreciation for replication has increased, but is not yet part of mainstream culture in social psychology.

The headline findings from the Open Science Collaboration (2015)―namely, that 36% of original experiments replicated at p < .05, with the overall replication effect sizes being half as large as the original effects―cannot be meaningfully interpreted without a formal model. A simple model-based approach might ask: what would the state of original science be and what would replication results show if original experiments tested true effects half the time (prior odds = 1), true effects had a medium effect size (Cohen’s δ = 0.50), and power to detect true effects was 50%? Assuming no questionable research practices, 91% of p < .05 findings in the original literature would be true positives. However, only 58% of original p < .05 findings would be expected to replicate using the Open Science Collaboration approach, and the replication effects overall would be only ~60% as large as the original effects. A minor variant of this model yields an expected replication rate of only 45%, with overall replication effect sizes dropping by half. If the state of original science is as grim as a non-model-based (i.e., intuitive) interpretation of the Open Science Collaboration data suggests, should it be this easy to largely account for those findings using a model in which 91% of statistically significant findings in the original science literature are true positives? Claims that the findings reported by the Open Science Collaboration indicate a replication crisis should not be based solely on intuition but should instead be accompanied by a specific model that supports that interpretation.

One remedy to the misuse of p-values transforms them to bounds on Bayes factors. With a prior probability of the null hypothesis, such a bound gives a lower bound on the posterior probability. Unfortunately, knowing a posterior probability is above some number cannot ensure that the null hypothesis is improbable enough to warrant its rejection. For example, if the lower bound is 0.0001, that implies that the posterior probability is at least 0.0001 but does not imply it is lower than 0.05 or even 0.9. A fiducial argument suggests an alternative estimate of the posterior probability that the null hypothesis is true. In the case that the prior probability of the null hypothesis is 50%, the estimated posterior probability is about p ⁡ (ln ⁡ p) 2 for low p. In other cases, each occurrence of p in the formula is the p-value calibrated by multiplying it by the prior odds of the null hypothesis. In the absence of a prior, p ⁡ (ln ⁡ p) 2 also serves as an asymptotic Bayes factor. Since the fiducial estimate of the posterior probability is greater than the lower bounds, its use in place of a bound leads to more stringent hypothesis testing. Making that replacement in a rationale for 0.005 as the significance level reduces the level to 0.001. [ABSTRACT FROM AUTHOR]

The current condition of (Western) academic psychology can be criticized for various reasons. In the past years, many debates have been centered around the so-called "replication crisis" and the "WEIRD people problem". However, one aspect which has received relatively little attention is the fact that psychological research is typically limited to currently living individuals, while the psychology of the past remains unexplored. We find that more research in the field of historical psychology is required to capture both the similarities and differences between psychological mechanisms both then and now. We begin by outlining the potential benefits of understanding psychology also as a historical science and explore these benefits using the example of stress. Finally, we consider methodological, ideological, and practical pitfalls, which could endanger the attempt to direct more attention toward cross-temporal variation. Nevertheless, we suggest that historical psychology would contribute to making academic psychology a truly universal endeavor that explores the psychology of all humans. [ABSTRACT FROM AUTHOR]

Geometric morphometrics is widely used to quantify morphological variation between biological specimens, but the fundamental influence of operator bias on data reproducibility is rarely considered, particularly in studies using photographs of live animals taken under field conditions. We examined this using four independent operators that applied an identical landmarking scheme to replicate photographs of 291 live Atlantic salmon (Salmo salar L.) from two rivers. Using repeated measures tests, we found significant inter-operator differences in mean body shape, suggesting that the operators introduced a systematic error despite following the same landmarking scheme. No significant differences were detected when the landmarking process was repeated by the same operator on a random subset of photographs. Importantly, in spite of significant operator bias, small but statistically significant morphological differences between fish from the two rivers were found consistently by all operators. Pairwise tests of angles of vectors of shape change showed that these between-river differences in body shape were analogous across operator datasets, suggesting a general reproducibility of findings obtained by geometric morphometric studies. In contrast, merging landmark data when fish from each river are digitised by different operators had a significant impact on downstream analyses, highlighting an intrinsic risk of bias. Overall, we show that, even when significant inter-operator error is introduced during digitisation, following an identical landmarking scheme can identify morphological differences between populations. This study indicates that operators digitising at least a sub-set of all data groups of interest may be an effective way of mitigating inter-operator error and potentially enabling data sharing. [ABSTRACT FROM AUTHOR]

The article presents the most important and almost forgotten theses of Franz Brentano's empirical psychology, which have significance for conceptualization and the method of psychological research. The psychology programme, introduced as early as 1874, remains on the fringes of mainstream empirical psychology, but it was the starting point for Kazimierz Twardowski and his students. The continuation and development of Brentano's thought in the twentieth century can significantly enrich and broaden psychology's theoretical and empirical perspective. This applies primarily to reductionism and the social dimension of mental phenomena. [ABSTRACT FROM AUTHOR]

Despite repeated opportunities to reconsider their natural science ambitions, social psychologists have not done so, and there are no obvious signs of this changing. Why? This paper pursues an answer to this question by defining the field after the fashion of Michael Polanyi's thought. According to Polanyi, interpretative frameworks develop from our primitive bodily encounters with the world and then are shaped by language into the vast conceptual systems of our culture. Concerning frameworks erected on our most fundamental beliefs (e.g., science), he says that we "live in [them] as in the garment of our own skin." Frameworks such as this are not objects of critical evaluation but of commitment, and social psychology, as an outgrowth of positive philosophy, is an interpretative framework in this sense. Professionals' recent responses to the field's political makeup and replication failures demonstrate this. They aim primarily at preserving a natural science understanding of social psychology and point to the influence of belief‐stabilizing mechanisms Polanyi finds operative in folk religious practices. These mechanisms appear at work also in psychology as a whole. They are implied, for instance, in the field's resistance to Sigmund Koch's authoritative judgement against its scientific self‐conception in the latter half of the 20th century. Noting this reveals the broader implications of this paper's definition of social psychology, but it also urges questions about truth and relativity that cannot be ignored. These questions are addressed briefly in the end where it is suggested that what psychology needs most of all is a change of heart, and that this will happen, if at all, not primarily through argument and evidence, but through persons who authentically believe in the veracity of a different framework. [ABSTRACT FROM AUTHOR]

The "reproducibility crisis" has been one of the most significant stories in science in the past 15 years and has led to significant policy changes across the research landscape. Yet, scandals, irreproducible studies, and cries of crisis have occurred for decades in science. This article seeks to explain why the reproducibility crisis has taken root and become a force in science policy in ways previous crises have not. In short, we argue that it was through the scientific, institutional, and cultural efforts of a group of scientific activists we are calling metascientists. Metascience is a scientific social movement that seeks to use quantification and experimentation to diagnose problems in research practice and improve efficiency. It draws together data scientists, experimental and statistical methodologists, and open science activists into a project with both intellectual and policy dimensions. Metascientists have been remarkably successful at winning grants, motivating news coverage, and changing policies at science agencies, journals, and universities. The social movement lens is useful for understanding the popularization and impact of the reproducibility crisis narrative and suggests ways the institutions of science are adapting to meet a changing political and technological landscape. [ABSTRACT FROM AUTHOR]

A common narrative among some behavioural economists and policy makers is that experimental psychology highlights that individuals are more like Homer Simpson than the Mr Spock imagined by neoclassical economics, and that this justifies policies aiming to 'correct' individual behaviours. This narrative is central to nudging policies and suggests that a better understanding of individual cognition will lead to better policy prescriptions. I argue that this Homer economicus narrative is methodologically flawed, and that its emphasis on cognition advances a distorted view of public policies consisting in fixing malfunctioning individuals, while ignoring the characteristics of the socio-economic environment that influence individuals' behaviours. [ABSTRACT FROM AUTHOR]

ObjectiveMuch of psychological research has suffered from small sample sizes and low statistical power, resulting in unstable parameter estimates. The Bayesian approach offers a promising solution by incorporating prior knowledge into statistical models, which may lead to improved stability compared to a frequentist approach.MethodsSimulated data from four populations with known bivariate correlations (ρ = 0.1, 0.2, 0.3, 0.4) was used to estimate the sample correlation as samples were sequentially added from the population, from n = 10 to n = 500. The impact of three different, subjectively defined prior distributions (weakly, moderately, and highly informative) was investigated and compared to a frequentist model.ResultsThe results show that bivariate correlation estimates are unstable, and that the risk of obtaining an estimate that is exaggerated or in the wrong direction is relatively high, for sample sizes for below 100, and considerably so for sample sizes below 50. However, this instability can be constrained by informative Bayesian priors.ConclusionInformative Bayesian priors have the potential to significantly reduce sample size requirements and help ensure that obtained estimates are in line with realistic expectations. The combined stabilizing and regularizing effect of a weakly informative prior is particularly useful when conducting research with small samples. The impact of more informative Bayesian priors depends on one’s threshold for probability and whether one’s goal is to obtain an estimate merely in the correct direction, or to obtain a high precision estimate whose associated interval falls within a narrow range. Implications for sample size requirements and directions for future research are discussed.

Personality is not the most popular subfield of psychology. But, in one way or another, personality psychologists have played an outsized role in the ongoing credibility revolution in psychology. Not only have individual personality psychologists taken on visible roles in the movement, but our fields practices and norms have now become models for other fields to emulate (or, for those who share Baumeisters (2016, https://doi.org/10.1016/j.jesp.2016.02.003) skeptical view of the consequences of increasing rigor, a model for what to avoid). In this article we discuss some unique features of our field that may have placed us in an ideal position to be leaders in this movement. We do so from a subjective perspective, describing our impressions and opinions about possible explanations for personality psychologys disproportionate role in the credibility revolution. We also discuss some ways in which personality psychology remains less-than-optimal, and how we can address these flaws.

Background Identification of widespread biases present in reported research findings in many scientific disciplines, including psychology, such as failures to replicate and the likely extensive application of questionable research practices, has raised serious concerns over the reliability and trustworthiness of scientific research. This has led to the development of, and advocacy for, ‘open science’ practices, including data, materials, analysis, and output sharing, pre-registration of study predictions and analysis plans, and increased access to published research findings. Implementation of such practices has been enthusiastic in some quarters, but literacy in, and adoption of, these practices has lagged behind among many researchers in the scientific community. Advances In the current article I propose that researchers adopt an open science ‘mindset’, a comprehensive approach to open science predicated on researchers’ operating under the basic assumption that, wherever possible, open science practices will be a central component of all steps of their research projects. The primary, defining feature of the mindset is a commitment to open science principles in all research projects from inception to dissemination. Other features of the mindset include the assumption that all components of research projects (e.g. pre-registered hypotheses, protocols, materials, analysis plans, data, and output) will be accessible broadly; pro-active selection of open fora to disseminate research components and findings; open and transparent dissemination of reports of the research findings in advance of, and after, formal publication; and active promotion of open science practices through education, modeling, and advocacy. Conclusion The open science mindset is a ‘farm to fork’ approach to open science aimed at promoting comprehensive quality in application of open science, and widening participation in open science practices so that they become the norm in research in health psychology and behavioral medicine going forward.

Geometric morphometrics is widely used to quantify morphological variation between biological specimens, but the fundamental influence of operator bias on data reproducibility is rarely considered, particularly in studies using photographs of live animals taken under field conditions. We examined this using four independent operators that applied an identical landmarking scheme to replicate photographs of 291 live Atlantic salmon (Salmo salar L.) from two rivers. Using repeated measures tests, we found significant inter-operator differences in mean body shape, suggesting that the operators introduced a systematic error despite following the same landmarking scheme. No significant differences were detected when the landmarking process was repeated by the same operator on a random subset of photographs. Importantly, in spite of significant operator bias, small but statistically significant morphological differences between fish from the two rivers were found consistently by all operators. Pairwise tests of angles of vectors of shape change showed that these between-river differences in body shape were analogous across operator datasets, suggesting a general reproducibility of findings obtained by geometric morphometric studies. In contrast, merging landmark data when fish from each river are digitised by different operators had a significant impact on downstream analyses, highlighting an intrinsic risk of bias. Overall, we show that, even when significant inter-operator error is introduced during digitisation, following an identical landmarking scheme can identify morphological differences between populations. This study indicates that operators digitising at least a sub-set of all data groups of interest may be an effective way of mitigating inter-operator error and potentially enabling data sharing.