Search

Robust scientific knowledge is contingent upon replication of original findings. However, replicating researchers are constrained by resources, and will almost always have to choose one replication effort to focus on from a set of potential candidates. To select a candidate efficiently in these cases, we need methods for deciding which out of all candidates considered would be the most useful to replicate, given some overall goal researchers wish to achieve. In this article we assume that the overall goal researchers wish to achieve is to maximize the utility gained by conducting the replication study. We then propose a general rule for study selection in replication research based on the replication value of the set of claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by conducting a replication of the claim, and is a function of (a) the value of being certain about the claim, and (b) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value. Our goal in this article is not to define concrete guidelines for study selection, but to provide the necessary theoretical foundations on which such concrete guidelines could be built.Translational Abstract Replication-redoing a study using the same procedures-is an important part of checking the robustness of claims in the psychological literature. The practice of replicating original studies has been woefully devalued for many years, but this is now changing. Recent calls for improving the quality of research in psychology has generated a surge of interest in funding, conducting, and publishing replication studies. Because many studies have never been replicated, and researchers have limited time and money to perform replication studies, researchers must decide which studies are the most important to replicate. This way scientists learn the most, given limited resources. In this article, we lay out what it means to think about what is the most important thing to replicate, and we propose a general decision rule for picking a study to replicate. That rule depends on a concept we call replication value. Replication value is a function of the importance of the study, and how uncertain we are about the findings. In this article we explain how researchers can think precisely about the value of replication studies. We then discuss when and how it makes sense to use replication value as a measure of how valuable a replication study would be, and we discuss factors that funders, journals, or scientists could consider when determining how valuable a replication study is.

Using a database of open data policies for 199 journals in ecology and evolution, we found no detectable link between data sharing requirements and article retractions or corrections. Despite the potential for open data to facilitate error detection, poorly archived datasets, the absence of open code and the stigma associated with correcting or retracting articles probably stymie error correction. Requiring code alongside data and destigmatizing error correction among authors and journal editors could increase the effectiveness of open data policies at helping science self-correct.

To foster research integrity (RI), research institutions should develop a continuous RI education approach, addressing various target groups. To support institutions to achieve this, we developed RI education guidelines together with RI experts and research administrators, exploring similarities and differences in recommendations across target groups, as well as recommendations about RI education using approaches other than formal RI training. We used an iterative co-creative process. We conducted four half-day online co-creation workshops with 16 participants in total, which were informed by the RI education evidence-base. In the first two workshops, participants generated ideas for guidelines' content, focusing on different target groups and various approaches to RI education. Based on this content we developed first drafts of the guidelines. Participants in the third and fourth workshop refined those drafts. We next organized a working group which further prioritized, reorganized, and optimized the content of the guidelines. We developed four guidelines on RI education focusing on (a) bachelor, master and PhD students; (b) post-doctorate and senior researchers; (c) other RI stakeholders; as well as (d) continuous RI education. Across guidelines, we recommend mandatory RI training; follow-up refresher training; informal discussions about RI; appropriate rewards and incentives for active participation in RI education; and evaluation of RI educational events. Our work provides experience-based co-created guidance to research institutions on what to consider when developing a successful RI education strategy. Each guideline is offered as a distinct, publicly available tool in our toolbox ( www.sops4ri.eu/toolbox ) which institutions can access, adapt and implement to meet their institution-specific RI education needs.Trial registration https://osf.io/zej5b .

Since its beginnings in the 1940s, automated reasoning by computers has become a tool of ever growing importance in scientific research. So far, the rules underlying automated reasoning have mainly been formulated by humans, in the form of program source code. Rules derived from large amounts of data, via machine learning techniques, are a complementary approach currently under intense development. The question of why we should trust these systems, and the results obtained with their help, has been discussed by philosophers of science but has so far received little attention by practitioners. The present work focuses on independent reviewing, an important source of trust in science, and identifies the characteristics of automated reasoning systems that affect their reviewability. It also discusses possible steps towards increasing reviewability and trustworthiness via a combination of technical and social measures.

The Contributor Roles Taxonomy (CRediT) uses 14 standard roles to specify scholarly contributions to research publications. CRediT has been adopted by hundreds of journals and integrated into journal submission workflows. In 2022, CRediT was formalized as a standard by the American National Standards Institute (ANSI) and the National Information Standards Organization (NISO), providing a sustained support system and custodianship. Subsequently, forming the CRediT Standing Committee aimed to ensure inclusive and transparent evolution and enhance understanding and adoption of CRediT in the research community. This paper outlines the CRediT Standing Committees ongoing strategies to support uptake and reduce ambiguity and confusion when using roles. These strategies include developing user instructions to provide guidance on how to use CRediT, creating persona profiles to improve understanding of existing roles and designing badges for each role to make them more memorable. These resources will address challenges reported by the scholarly community or identified by the CRediT Standing Committee and will be shared openly upon completion.

In recent years, the scientific community has called for improvements in the credibility, robustness, and reproducibility of research, characterized by increased interest and promotion of open and transparent research practices. While progress has been positive, there is a lack of consideration about how this approach can be embedded into undergraduate and postgraduate research training. Specifically, a critical overview of the literature which investigates how integrating open and reproducible science may influence student outcomes is needed. In this paper, we provide the first critical review of literature surrounding the integration of open and reproducible scholarship into teaching and learning and its associated outcomes in students. Our review highlighted how embedding open and reproducible scholarship appears to be associated with (1) students’ scientific literacies (i.e., students’ understanding of open research, consumption of science, and the development of transferable skills); (2) student engagement (i.e., motivation and engagement with learning, collaboration, and engagement in open research), and (3) students’ attitudes towards science (i.e., trust in science and confidence in research findings). However, our review also identified a need for more robust and rigorous methods within pedagogical research, including more interventional and experimental evaluations of teaching practice. We discuss implications for teaching and learning scholarship.

In this post, I focus on Andrea Bazzoli’s (2022) excellent commentary: “Open science and epistemic pluralism: A tale of many perils and some opportunities.”I elaborate on some of his points with my own thoughts inspired by Sabina Leonelli’s (2022)recent work.

El meta-análisis es un método ampliamente utilizado para sintetizar los datos de diferentes estudios. Sin embargo, a menudo estudiantes, profesionales e investigadores carecemos de conocimientos prácticos para hacer e interpretar un meta-análisis. Esta guía presenta una variedad de herramientas para realizar meta-análisis de correlaciones en R, mediante el uso de ejemplos reales. Incluye desde análisis simples y su interpretación, hasta el análisis de moderadores (meta-regresión), usando los paquetes metafor (Viechtbauer, 2010) y metaviz (Kossmeier et al., 2020). También incluye explicaciones para la transformación de coeficientes r de Pearson a z de Fisher (y viceversa), creación de gráficos de bosque (forest plots) y gráficos de embudo (funnel plots), análisis de heterogeneidad y diagnósticos de influencia, así como estrategias para detectar posibles sesgos de publicación utilizando el paquete weightr (Coburn & Vevea, 2019), y para determinar el poder estadístico de un meta-análisis utilizando metameta (Quintana, 2022). Con esta guía, los lectores podrán adquirir las habilidades necesarias para realizar meta-análisis de correlaciones de manera efectiva en R y obtener resultados confiables.

I propose three simple principles to help researchers manage their data and code: (E) set a finite extent (don't integrate) and (L) use libraries (one thing, one place) to (M) be modular. Applying these principles should help to share and reproduce, while reducing perceived workload.

Meta-analysis upweights studies reporting lower standard errors and hence more precision. But in empirical practice, notably in observational research, precision is not given to the researcher. Precision must be estimated, and thus can be p-hacked to achieve statistical significance. Simulations show that a modest dose of spurious precision creates a formidable problem for inverse-variance weighting and bias-correction methods based on the funnel plot. Selection models fail to solve the problem, and the simple mean can beat sophisticated estimators. Cures to publication bias may become worse than the disease. We introduce an approach that surmounts spuriousness: the Meta-Analysis Instrumental Variable Estimator (MAIVE), which employs inverse sample size as an instrument for reported variance.

This study reports an independent replication of the findings presented by Smaldino and McElreath (Smaldino, McElreath 2016 R. Soc. Open Sci. 3 , 160384 ( doi:10.1098/rsos.160384 )). The replication was successful with one exception. We find that selection acting on scientist’s propensity for replication frequency caused a brief period of exuberant replication not observed in the original paper due to a coding error. This difference does not, however, change the authors’ original conclusions. We call for more replication studies for simulations as unique contributions to scientific quality assurance.

Research integrity (RI) is crucial for trustworthy research. Rules are important in setting RI standards and improving research practice, but they can lead to increased bureaucracy; without commensurate commitment amongst researchers toward RI, they are unlikely to improve research practices. In this paper, we explore how to combine rules and commitment in fostering RI. Research institutions can govern RI using markets (using incentives), bureaucracies (using rules), and network processes (through commitment and agreements). Based on Habermas’ Theory of Communicative Action, we argue that network processes, as part of the lifeworld, can legitimize systems – that is, market or bureaucratic governance modes. This can regulate and support RI practices in an efficient way. Systems can also become dominant and repress consensus processes. Fostering RI requires a balance between network, market and bureaucratic governance modes. We analyze the institutional response to a serious RI case to illustrate how network processes can be combined with bureaucratic rules. Specifically, we analyze how the Science Committee established at Tilburg University in 2012 has navigated different governance modes, resulting in a normatively grounded and efficient approach to fostering RI. Based on this case, we formulate recommendations to research institutions on how to combine rules and commitment.

Although systematic reviews are intended to provide trusted and actionable scientific knowledge to meet the needs of policy makers and medical professionals, it is common for systematic reviews on the same topic to yield different conclusions. Often such conflicting systematic reviews display inconsistency in the evidence that they select and synthesize. In this paper, we study evidence selection practices, especially evidence selection inconsistency. To do so, we propose a network construct called the inclusion network. We describe the construct of the inclusion network and its difference to the claim-specific citation network, another influential network construct for studying epistemic phenomena in science, the general data collection procedure, and four metrics designed to study evidence selection practices: the adjusted Jaccard similarity, the inclusion number, the primary study report utilization rate, and the dandelion-ness ratio. We used two inclusion network datasets as two case studies to test the robustness of our inclusion network approach. Using adjusted Jaccard similarity, we were able to quantify the level of consistency in evidence selection. However, adjusted Jaccard similarity is less robust when systematic reviews in an inclusion network exhibit research question heterogeneity. We also demonstrate that time series derived from the inclusion network can detect changes in evidence selection practices and identify potentially influential systematic review reports. We propose several research opportunities opened up by the inclusion network approach and call for a community effort to index included PSR separately and make such data publicly available.

Background:A commonly used metric for evaluating academic success, the number of published articles, focuses on processes that select against female academics. Novel social conditions induced by COVID-19 pandemic likely exacerbated this gender-bias if female academics took on the load of caregiving, domestic, service and teaching roles. We investigate the pandemic effect on the gender gap in research productivity through a systematic review and meta-analysis of published articles across scientific disciplines.Methods:We identified 50 relevant articles with 115 effect sizes that measure the impact of the COVID-19 pandemic on gender-specific research productivity, commonly as the number of submitted/published articles (n=97). We conducted a meta-analysis to 1) investigate the effect of the pandemic on the gender gap in research productivity within academia and 2) test hypotheses on how research field, breadth of gender gap before the pandemic, and authorship position influence this effect.Results:Overall, the gender gap in research productivity within academia has increased during the COVID-19 pandemic compared to before, especially in social sciences and medicine, fields that were previously nearest to being gender equal. We did not detect an influence of authorship position on the effect.Conclusions:We detected that gender biases favouring the productivity and perceived impact of men in academia overall strengthened during the pandemic. We encourage academic and funding institutions to consider a range of different metrics to evaluate academic impact, and to further acknowledge and accommodate individual circumstances.

In recent decades, there has been a move to "open" research, that is, to increase the reach of research products to broader audiences. One component of open access is accessibility. Accessibility generally refers to data and other products being free and open to use by others, but accessibility also refers to considering and meeting the needs of people with disabilities for participation and inclusion. Ensuring that visual content is understandable is a major component of ensuring open access publications are accessible, and alt text is a common way to make inaccessible images and non-text content more accessible. Using image accessibility and alt text as a lens, our objective was to evaluate how open access journals incorporate disability accessibility as part of open access publishing. Using a random sample of 300 English language open access journals, we assessed author guidelines to understand image requirements for submissions and open access statements to understand how journals conceive of openness and accessibility. We found that most open access journals do not include disability accessibility elements in their guidelines to authors when submitting images as part of their scholarship. While over half the journals had required parameters for image submission, none of them required alt text. And while the majority of journals included the word 'access' or 'accessibility' in their open access statements, almost none included disability or inclusion related terms. Our results highlight the importance of guidelines. Our findings speak to the limits of some of the current frameworks of open access. Incorporating disability accessibility into open access has the potential to bridge existing information inequalities for people with disabilities - and to make sure that mandates for open research do not exacerbate those inequalities.

The study addresses the following question: what is the source of the Dark Energy?The study provides sound argumentations that Electromagnetic Waves, from separate sources, can consolidate, even though such consolidations seem to embed paradoxes. The study also proposes a physical experiment which might implement consolidations of Electromagnetic Waves, from separate sources. In addition to the above, the study also addresses paradoxes related to the Mutual Annihilation and the Pair Production processes. The study proposes a resolution for all these paradoxes which also concludes that most of the Dark Energy might be related to Electromagnetism and that the Electric Charge is also a form of Energy.

International large-scale assessments (ILSAs) in education provide researchers with opportunities to study educational phenomena in a large set of contexts, educational systems, domains, and student populations. This project evaluates this potential and develops and illustrates methodological approaches and guidance to incorporating ILSAs in meta-analyses.

Academic research, as a social system, functions as a market between research producers and research consumers. Conventionally, this market introduces information asymmetries between these market participants—research producers know far more about a study than research consumers. This asymmetry can erode trust in the credibility of findings. Transparency, through open science practices, can enable readers to evaluate the credibility of a claim and, by reducing this information asymmetry, promote confidence in our findings. This paper discusses factors that can undermine trust in the credibility of findings, introduces key open science practices that are broadly applicable for IS, considers the current discourses on these practices in IS, and systematically reviews the level of adoption of open science practices in IS as is evident in empirical, quantitative articles published in the Senior Scholars Basket of Eight Journals. This investigation provides a baseline for the adoption of open science practices in our discipline and gives insight into the extent to which claims in IS research can be assessed. The results of the evaluation suggest that, prior to recent reforms, there exists a substantial degree of information asymmetry and a serious neglect of transparency in IS research. Transparency is a necessary component for the continued growth of IS as a credible science and considerable reform is required.

Custom-built VR motion tracking sensors

Increasing evidence suggests that the reproducibility and replicability of scientific findings is threatened by researchers employing questionable research practices (QRPs) in order to achieve statistically significant results. Numerous metrics have been developed to determine replication success but it has not yet been investigated how well those metrics perform in the presence of QRPs. This paper aims to compare the performance of different metrics quantifying replication success in the presence of four types of QRPs: cherry picking of outcomes, questionable interim analyses, questionable inclusion of covariates, and questionable subgroup analyses. Our results show that the metric based on the version of the sceptical p-value that is recalibrated in terms of effect size performs better in maintaining low values of overall type-I error rate, but often requires larger replication sample sizes compared to metrics based on significance, the controlled version of the sceptical p-value, meta-analysis or Bayes factors, especially when severe QRPsare employed.

Most original articles published in the medical literature report the results of multiple statistical tests. In a few simple cases, there is general agreement on whether one should adjust for multiple testing or not. In most cases encountered in practice, however, there are contradictory and confusing recommendations in the literature. In this article, we suggest a unique criterion to decide whether to adjust for multiple testing or not, which has the advantage of being easily understandable for medical researchers and statisticians and of including most existing rules as special cases while providing answers in less straightforward situations.

In this research, we apply network analysis to data gathered through structured interviews and file forensics to evaluate the information flows and research workflows in materials science research projects. We discuss the gaps in reproducibility highlighted by this analysis and suggest practical steps to making these scientific workflows more robust and sharable.

1. Researchers in ecology and evolutionary biology are increasingly dependent on computational code to conduct research. Hence, the use of efficient methods to share, reproduce, and collaborate on code as well as document research is fundamental. GitHub is an online, cloud-based service that can help researchers track, organize, discuss, share, and collaborate on software and other materials related to research production, including data, code for analyses, and protocols. Despite these benefits, the use of GitHub in ecology and evolution is not widespread. 2. To help researchers in ecology and evolution adopt useful features from GitHub to improve their research workflows, we review 12 practical ways to use the platform. 3. We outline features ranging from low to high technical difficulty, including storing code, managing projects, coding collaboratively, conducting peer review, writing a manuscript, and using automated and continuous integration to streamline analyses. Given that members of a research team may have different technical skills and responsibilities, we describe how the optimal use of GitHub features may vary among members of a research collaboration. 4. As more ecologists and evolutionary biologists establish their workflows using GitHub, the field can continue to push the boundaries of collaborative, transparent, and open research. collaboration, data management, ecoinformatics, GitHub, open science, project management, reproducible research, version control

Scientists must choose which among many experiments to perform. We study the epistemic success of experimental choice strategies proposed by philosophers of science or executed by scientists themselves. We develop a multi-agent model of the scientific process that jointly formalizes its core aspects: active experimentation, theorizing, and social learning. We find that agents who choose new experiments at random develop the most accurate theories of the world. The agents aiming to confirm, falsify theories, or resolve theoretical disagreements end up with an illusion of epistemic success: they develop promising accounts for the data they collected, while completely misrepresenting the ground truth that they intended to learn about. Agents experimenting in theory-motivated ways acquire less diverse or less representative samples from the ground truth that are also easier to account for. The novelty-seeking agents suffer from collecting unrepresentative observations which do not allow them to build as successful accounts for reality as enabled by random data collection. Thus, random data collection combines virtues of diverse and representative sampling from a target scientific domain. We suggest that randomization, already a gold standard within experiments, is also beneficial at the level of experiments themselves.

Objectives:A need for Open Research practices exists, yet there remains a lack of validated questionnaires to assess Open Research practices. The study aimed to develop a brief (

Resolving the “replication crisis” is a top priority of the scientific community now. Numerous proposals have been made; still, there lacks not only an established solution but even an agreement on whether there exists a “crisis” or not. Here, by questioning the philosophical foundations of our study designs and analyses, I trace back the “crisis” to reductionist ontologies and methodologies ingrained in the modern statistical methods which have dominated biological, medical, psychological, and social sciences for a century. The crisis is not our inability to “reproduce” results but that, uncritical of our statistical methods and the experimental designs they have inculcated, we expect to be able to “reproduce” results despite neglecting almost all individual-level and contextual variables of complex processes.

SEO stands for search engine optimization. It is a process to improve websites so as to increase its visibility and reach when people search for keywords related to its relevant business or institution in any of the search engines. Reaching a wider audience is vital for all institutions - business or academic. Educational institutions and businesses often find it difficult to showcase their research to the wide world. This is not only because of a low research output. Marketing and awareness of the journals documenting this research is also lacking. This paper explores the importance of off page SEO to helping institutions rank better overall assuming their existing research output remains unchanged both qualitatively and quantitatively. This in turn should help these institutions break out of a vicious cycle where lack of visibility of existing research leads to reduced citations and influence in the academic and corporate world, which in turn reduces the incentive to fund further research. With a concerted effort, it is possible for even relatively newer and lesser known institutions to enhance their reach at a relatively low cost with the help of organic traffic.

Forecasting the reproducibility of research findings is one of the key challenges of Metascience. To date, reliable above-chance predictions have mainly been achieved by pooling the subjective ratings of experts through surveys or via prediction markets. Obtaining such data is laborious, however, and unlikely to increase the theoretical understanding of reproducibility.Here we show that empirical measures of K, a principled metric of knowledge, are correlated with reproducibility forecasts made for the Brazilian Reproducibility Initiative (BRI), an ongoing replication of 60 studies that use three common methodologies in the life sciences. For each study, we calculated the K value by dividing the effect size, measured in bits of Shannon entropy explained, by the descriptive complexity of the study’s methodology, calculated as the optimal Shannon encoding of a conceptual graph representing the replication protocol.K values were statistically associated with subjective predictions about studies’ replication probabilities and relative effect sizes. This relation was robust to controlling for study methodology, and was a stronger predictor than other plausible covariates, including the originally reported effect size, subjective ratings of a study’s methodological difficulty, and raters’ self-reported expertise. The superior fit of K may be due to its particular structure and metrics, which are rooted in information theory and may more accurately reflect the implicit calculations made by raters.This is the first evidence that an objective metric hypothesized to quantify scientific knowledge captures subjective judgments about reproducibility. This finding gives the first independent support of K’s underlying assumption that scientific knowledge is a process of information compression, and it may offer a new tool for metaresearch.

Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This document is then made available with the published research report to allow readers to identify discrepancies between what the researchers originally planned to do and what they actually ended up doing. This historical transparency is supposed to facilitate judgments about the credibility of the research findings. The present article provides a critical review of 17 of the reasons behind this argument. The article covers issues such as HARKing, multiple testing, p-hacking, forking paths, optional stopping, researchers’ biases, selective reporting, test severity, publication bias, and replication rates. It is concluded that preregistration’s historical transparency does not facilitate judgments about the credibility of research findings when researchers provide contemporary transparency in the form of (a) clear rationales for current hypotheses and analytical approaches, (b) public access to research data, materials, and code, and (c) demonstrations of the robustness of research conclusions to alternative interpretations and analytical approaches.

Edward Wilson, in his book Consilience (Wilson, 1999), proposes a unity of knowledge that encompasses all the sciences, from physics and chemistry to the humanities and even art. Energy management is a good example where they can be concatenated, following a fairly intuitive reasoning, from the behavior of large and small physical systems to the natural origin of ethical principles and the need for laws in today's complex human societies. This chapter tries to make this journey very succinctly, where each step is barely outlined, with the ultimate goal of completing the path without losing sight of the entire route. To compensate for this short development, each step includes at least one reference for the curious reader.

BackgroundPrevalence of research misconduct, questionable research practices (QRPs) and their associations with a range of explanatory factors has not been studied sufficiently among academic researchers.Methods The National Survey on Research Integrity was aimed at all disciplinary fields and academic ranks in the Netherlands. The survey enquired about engagement in fabrication, falsification and 11 QRPs over the previous three years, and 12 explanatory factor scales. We ensured strict identity protection and used a randomized response method for questions on research misconduct. Results6,813 respondents completed the survey. Prevalence of fabrication was 4.3% (95% CI: 2.9, 5.7) and falsification 4.2% (95% CI: 2.8, 5.6). Prevalence of QRPs ranged from 0.6% (95% CI: 0.5, 0.9) to 17.5% (95 % CI: 16.4, 18.7) with 51.3% (95% CI: 50.1, 52.5) of respondents engaging frequently in ≥ 1 QRP. Being a PhD candidate or junior researcher increased the odds of frequently engaging in ≥ 1 QRP, as did being male. Scientific norm subscription (odds ratio (OR) 0.79; 95% CI: 0.63, 1.00) and perceived likelihood of detection by reviewers (OR 0.62, 95% CI: 0.44, 0.88) were associated with lower odds of research misconduct. Publication pressure was associated with higher odds of engaging frequently in ≥ 1 QRP (OR 1.22, 95% CI: 1.14, 1.30).ConclusionsWe found higher prevalence of misconduct than earlier surveys. Our results suggest that greater emphasis on scientific norm subscription, strengthening reviewers in their role as gatekeepers of research quality and curbing the “publish or perish” incentive system can promote research integrity.

Quality assessment is a crucial issue within the wine industry. The traditional way of assessing by human experts is time consuming and very expensive. Machine learning techniques help in the process of quality assurance in a wide range of industries. The purpose of this study was to develop and offer a free software, for winemakers and customers in which they can easily provide the physicochemical properties of the wine and receive an accurate prediction of the anticipated quality and type of the wine. We used comprehensive datasets of 6497 examples, which contained physicochemical properties and appropriate quality. We combined these datasets, built and trained several neural networks models. We evaluated their performance and selected the best model. Wine quality estimations were modeled as a regression problem and wine type detection as a classification problem. The best model performed well for prediction of wine quality (root means squared error = 0.54) and type (f-score=0.99). With our free software, winemakers and customers can examine how a fine change in each physicochemical property could affect the quality of the wine. They could easily figure out the importance of each physicochemical property, and which one to ignore for reduction of cost. The process is very fast, accurate and does not require taste experts for sensory tests.

As a subfield of artificial intelligence, machine learning designed to learn the structure of the data. Machine learning has been widely used in many scientific problems. In this study, we used machine learning techniques to figure out the most important physicochemical properties for type classification of red wines. We used a wines' dataset with 13 physicochemical properties. We used a Random Forest classifier to predict wine's type from its features, and permutation feature importance, in order to detect the most important properties of the wine for type classification. The properties: flavanoids, proline, and color intensity were found to be most important for type classification. Additional 4 classifiers: Laso classifier, Ridge classifier, Decision Tree classifier, and Support Vector classifier were used and examined for classification and feature importance. Flavanoids and proline were very important across all classifiers.

When you preregister (or create a pre-analysis plan) you specify your research plan and publicly register it in advance of undertaking your study. The preregistration is a time-stamped, read-only plan. When this plan is extended and undergoes formal peer review at a journal ahead of the research, it is called a Registered Report. This preprint explains why and how to write a preregistration or Registered Report, along with information on the types of work this applies to, whether preregistration and Registered Reports “work”, and a list of resources and templates.

The most common and well-known meta-regression models work under the assumption that there is only one effect size estimate per study and that the estimates are independent. However, meta-analytic reviews of social science research often include multiple effect size estimates per primary study, leading to dependence in the estimates. Some meta-analyses also include multiple studies conducted by the same lab or investigator, creating another potential source of dependence. An increasingly popular method to handle dependence is robust variance estimation (RVE), but this method can result in inflated Type I error rates when the number of studies is small. Small-sample correction methods for RVE have been shown to control Type I error rates adequately but may be overly conservative, especially for tests of multiple-contrast hypotheses. We evaluated an alternative method for handling dependence, cluster wild bootstrapping, which has been examined in the econometrics literature but not in the context of meta-analysis. Results from two simulation studies indicate that cluster wild bootstrapping maintains adequate Type I error rates and provides more power than extant small sample correction methods, particularly for multiple-contrast hypothesis tests. We recommend using cluster wild bootstrapping to conduct hypothesis tests for meta-analyses with a small number of studies. We have also created an R package that implements such tests.

Journals exert considerable control over letters, commentaries and online comments that criticize prior research (post-publication critique). We assessed policies (Study One) and practice (Study Two) related to post-publication critique at 15 top-ranked journals in each of 22 scientific disciplines ( N = 330 journals). Two-hundred and seven (63%) journals accepted post-publication critique and often imposed limits on length (median 1000, interquartile range (IQR) 500–1200 words) and time-to-submit (median 12, IQR 4–26 weeks). The most restrictive limits were 175 words and two weeks; some policies imposed no limits. Of 2066 randomly sampled research articles published in 2018 by journals accepting post-publication critique, 39 (1.9%, 95% confidence interval [1.4, 2.6]) were linked to at least one post-publication critique (there were 58 post-publication critiques in total). Of the 58 post-publication critiques, 44 received an author reply, of which 41 asserted that original conclusions were unchanged. Clinical Medicine had the most active culture of post-publication critique: all journals accepted post-publication critique and published the most post-publication critique overall, but also imposed the strictest limits on length (median 400, IQR 400–550 words) and time-to-submit (median 4, IQR 4–6 weeks). Our findings suggest that top-ranked academic journals often pose serious barriers to the cultivation, documentation and dissemination of post-publication critique.

In this article, we present statacons, an SCons-based build tool for Stata. Because of the integration of Stata and Python in recent versions of Stata, we are able to adapt SCons for Stata workflows without the use of an external shell or extensive configuration. We discuss the usefulness of build tools generally, provide examples of the use of statacons in Stata workflows, present key elements of the syntax of statacons, and discuss extensions, alternatives, and limitations. We provide recommendations for collaborative workflows and, at the end of the article, installation instructions.

Scientists' ability to integrate diverse forms of evidence and evaluate how well they can explain and predict phenomena, in other words, $\textit{to know how much they know}$, struggles to keep pace with technological innovation. Central to the challenge of extracting knowledge from data is the need to develop a metric of knowledge itself. A candidate metric of knowledge, $K$, was recently proposed by the author. This essay further advances and integrates that proposal, by developing a methodology to measure its key variable, symbolized with the Greek letter $\tau$ ("tau"). It will be shown how a $\tau$ can represent the description of any phenomenon, any theory to explain it, and any methodology to study it, allowing the knowledge about that phenomenon to be measured with $K$.To illustrate potential applications, the essay calculates $\tau$ and $K$ values of: logical syllogisms and proofs, mathematical calculations, empirical quantitative knowledge, statistical model selection problems, including how to correct for "forking paths" and "P-hacking" biases, randomised controlled experiments, reproducibility and replicability, qualitative analyses via process tracing, and mixed quantitative and qualitative evidence.Whilst preliminary in many respects, these results suggest that $K$ theory offers a meaningful understanding of knowledge, which makes testable metascientific predictions, and which may be used to analyse and integrate qualitative and quantitative evidence to tackle complex problems.

There is little research on how guidance provided in the European Code of Conduct for Research Integrity (ALLEA code) compares with recommendations developed by European discipline-specific learned societies. We identified, and conducted a content analysis of, 58 guideline documents from 245 societies. Less than 25% of societies in any discipline provide guidance and there are notable disciplinary differences. Recommendations not reflected in the ALLEA code relate primarily to research culture and environment. Medical and Health Sciences societies often focus on regulatory and procedural aspects of research, whereas Natural Sciences societies emphasize the importance of accurate and appropriate dissemination of results. Humanities and Social Sciences societies’ recommendations are more heterogenous and relate to the nature of specific sub-disciplines. Our results reflect differences in epistemological approaches as well as the specific role and responsibilities of societies as membership organizations. We recommend that societies develop, or endorse, appropriate research integrity guidance.

Meta-analytic models for dependent effect sizes have grown increasingly sophisticated over the last few decades, which has created challenges for a priori power calculations. We introduce power approximations for tests of average effect sizes based upon several common approaches for handling dependent effect sizes. In a Monte Carlo simulation, we show that the new power formulas can accurately approximate the true power of meta-analytic models for dependent effect sizes. Lastly, we investigate the Type I error rate and power for several common models, finding that tests using robust variance estimation provide better Type I error calibration than tests with model-based variance estimation. We consider implications for practice with respect to selecting a working model and an inferential approach.

Scientific journals may counter the misuse, misreporting, and misinterpretation of statistics by providing guidance to authors. We described the nature and prevalence of statistical guidance at 15 journals (top-ranked by Impact Factor) in each of 22 scientific disciplines across five high-level domains (N = 330 journals). The frequency of statistical guidance varied across domains (Health & Life Sciences: 122/165 journals, 74%; Multidisciplinary: 9/15 journals, 60%; Social Sciences: 8/30 journals, 27%; Physical Sciences: 21/90 journals, 23%; Formal Sciences: 0/30 journals, 0%). In one discipline (Clinical Medicine), statistical guidance was provided by all examined journals and in two disciplines (Mathematics and Computer Science) no examined journals provided statistical guidance. Of the 160 journals providing statistical guidance, 93 had a dedicated statistics section in their author instructions. The most frequently mentioned topics were confidence intervals (90 journals) and p-values (88 journals). For six “hotly debated” topics (statistical significance, p-values, Bayesian statistics, effect sizes, confidence intervals, and sample size planning/justification) journals typically offered implicit or explicit endorsement and rarely provided opposition. The heterogeneity of statistical guidance provided by top-ranked journals within and between disciplines highlights a need for further research and debate about the role journals can play in improving statistical practice.

We show here that entropic information is capable of unifying all aspects of the universe at all scales in a coherent and global theoretical mathematical framework materialized by entropic information framework, theory and formulas, where dark matter, dark energy and gravity are truly informationals processes and where information is code and code is what creates the process, it is itself the process. Mass, energy and movement of information are respectively dark matter, dark energy, and gravity. Here, we reconcile general relativity and quantum mechanics by introducing quantum gravity for the Planckian scale. The formulas of entropic information are expressed in natural units, physical units of measurement based only on universal constants, constants, which refer to the basic structure of the laws of physics: C and G are part of the structure of space-time in general relativity, and h captures the relationship between energy and frequency that is the basis of quantum mechanics. Here we show that entropic information formulas are able to present entropic information in various unifying aspects and introduce gravity at the Planck scale. We prove that Entropic information theory is thus building the bridge between general relativity and quantum mechanics

The publishers of scholarly journals, particularly the large for-profit publishers, have failed to ensure journal publication is transparent in process and output. They have failed to sustain and invigorate peer review. By relying on the print tradition of publishing, which focuses on the short research report, they have failed to sufficiently support innovations which take advantage of digital platforms for archiving, distributing and critiquing scholarly work. We review the consequences of this for research reliability and call for greater scrutiny of for-profit academic publishing.

Breznau, Rinke, and Wuttke, et al. attempted to test for uniformity in the way hypotheses were tested across multiple research groups. They appeared to find a great deal of discord between conclusions reached using the same hypothesis and data. Yet, what they really found is that scientists are not properly understanding what hypotheses are.

In meta-analyses, it is critical to assess the extent to which publication bias might have compromised the results. Classical methods based on the funnel plot, including Egger’s test and Trim-and-Fill, have become the de facto default methods to do so, with a large majority of recent meta-analyses in top medical journals (85%) assessing for publication bias exclusively using these methods. However, these classical funnel plot methods have important limitations when used as the sole means of assessing publication bias: they essentially assume that the publication process favors large point estimates for small studies and does not affect the largest studies, and they can perform poorly when effects are heterogeneous. In light of these limitations, we recommend that meta-analyses routinely apply other publication bias methods in addition to or instead of classical funnel plot methods. To this end, we describe how to use and interpret selection models. These methods make the often more realistic assumption that publication bias favors ``statistically significant'' results and that also directly accommodate effect heterogeneity. Selection models are well-established in the statistics literature and are supported by user-friendly software, yet remain rarely reported in many disciplines. We use previously published meta-analyses to demonstrate that selection models can yield insights that extend beyond those provided by funnel plot methods, suggesting the importance of establishing more comprehensive reporting practices for publication bias assessment.

Researchers seeking to replicate original research often need to decide which of several relevant candidates to select for replication. Several strategies for study selection have been proposed, utilizing a variety of observed indicators as criteria for selection. However, few strategies clearly specify the goal of study selection and how that goal is related to the indicators that are utilized. We have previously formalized a decision model of replication study selection in which the goal of study selection is to maximize the expected utility gain of the replication effort. We further define the concept of replication value as a proxy for expected utility gain (Isager et al., 2020). In this article, we propose a quantitative operationalization of replication value. We first discuss how value and uncertainty - the two concepts used to determine replication value – could be estimated via information about citation count and sample size. Second, we propose an equation for combining these indicators into an overall estimate of replication value, which we denote RVCn. Third, we suggest how RVCn could be implemented as part of a broader study selection procedure. Finally, we provide preliminary data suggesting that studies that were in fact selected for replication tend to have relatively high RVCn estimates. The goal of this article is to explain how RVCn is intended to work and, in doing so, demonstrate the many assumptions that should be explicit in any replication study selection strategy.

Higher dimensions are impossible to visualize as the size of dimension varies inversely proportional to its level. The more the dimension ranges, the least its size. We are a set of points living in a particular point of space and a particular frame of time. i.e, we live in space-time. The space has more dimensions that meets the human eye. We are living in a world of hyper-space. Our world being a smaller dimension is floating in higher dimensions. The quest for the visually of higher dimensions has been a fantasy to mankind but this aspect of nature is completely locked. We can transform dimensions i.e., from higher to lower dimensions, or from lower to higher dimensions, but only through mathematics. The relative notion of mathematics helps us to do the thing, which is perhaps impossible in the experimental part of physical reality. Humans being an element of 3 Dimensions – length, breath, height can only perceive one higher dimensions, that is space-time. but beyond that the notion of dimension itself changes. The dimensions got curled up in every intersection of the coordinates of space in such a way that the higher dimensions remain stable to us. But in reality it is highly unstable. In the higher dimensions, above 4, the space is tearing apart and joining again spontaneously, but the tearing portion itself covered by 2 dimensional Branes which acts as a stabilizer for the unstable dimensions. Dimensions will get smaller and smaller with the space-time interwoven in it. But at Planks length that is 10^-33 meter, the notion of space-time itself breaks down thereby making impossible for the higher dimensions to coexist along with space. Without space, there will be no identity of any dimension. The space itself is the fabric for the milestone of residing higher dimensions. Imagine our room, which is 3 dimensional. But what is there inside the room. The space and of course the time. Space-time being a totally separate entity is not quite separate when compared with other dimensions because it makes the residing place for the higher dimensions or the hyperspace itself. We all are confined within a lower dimensional world within a randomness of higher dimensions. Time being alike like space is an arrow which has the capability of slicing space into different forms. Thereby taking a snapshot of our every nano-second we vibrate within space-time. As each slice of time represents each slice of space, similarly each slice of space represents each slice of time. The nature of space-time is beyond human consciousness. It is the identity by which we breathe, we play, we survive. It is the whole localization of species that encompasses itself with space thereby making space-time a relative quantity depending upon the reference frame. The only thing that can encompass space-time or even change the relative definition of space-time is the speed, the speed far beyond the speed of light. The more the speed, the less the array of time flows. Space-time being an invisible entity makes the other dimensions visible residing in it only into the level of 3, that is l, b, h. After that there is a infamous structure formed by the curling of higher dimensions called CALABI-YAU manifold. This manifold depicts the usual nature of the dimensional quadrants of the higher order by containing a number of small spherical spheres inside it. The mathematics of string theory is still unable to solve the genus and the containing spheres of the manifold which can be the ultimate quest for the hidden dimensions. Hidden, as, the higher dimensions are hidden from human perspective of macro level but if we probe deeper into the fabric of the space-time of General Relativity then we will find the 5th dimension according to the Kaluza-Klein theory. And if we probe even deeper into it at the perspective of string theory we will be amazed to see the real nature of quantum world. They are so marvelously beautiful, they contain so many forms of higher dimensions ranging from 6 to 10. And even many more of that, but we are still not sure about it where they may exist in a ghost state. After all, the quantum nature is far more beautiful that one can even imagine with a full faze of weirdness.

Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically (mindlessly) assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses., Synthese (2021)

BackgroundWhile substantial attention has been paid to research misbehaviors, responsible research practices (RRPs) and their potential explanatory factors have not been studied extensively.Methods The National Survey on Research Integrity (NSRI) is an online survey targeting all disciplines and academic ranks in The Netherlands. Data was collected on 11 RRPs and 12 explanatory factor scales. Results were controlled for explanatory factor scales, academic rank, disciplinary field, gender, doing empirical research and if respondents belonged to a NSRI supporting institution or not.Results6,813 respondents completed the survey. The RRPs with the highest prevalence were avoiding plagiarism (99%), disclosing conflicts of interest (96.5%) and checking for errors before publication (94.3%). Preregistration of study protocols (42.8%), making accessible underlying data and syntaxes (47.2%), and keeping comprehensive research records (56.3%) had the lowest prevalence. Arts and humanities scholars, PhD candidates and junior researchers were associated with a lower RRP mean (-0.51 and -0.31 respectively) as was publication pressure (-0.05; 95% CI -0.08, -0.02). Mentoring (0.15; 95% CI 0.12, 0.17), scientific norm subscription (0.13; 95% CI 0.1, 0.15) and funding pressure (0.13; 95% CI0.10, 0.17) were significantly associated with a higher RRP mean.ConclusionsWe found publication pressure to affect RRPs negatively. Mentoring, scientific norm subscription and funding pressure may help foster RRPs. Arts and humanities scholars, PhD candidates and junior researchers need more efforts to raise awareness on RRPs. Further research on these groups is warranted in order to understand research integrity challenges that may be unique to them.

The paper interprets the concept “operator in the separable complex Hilbert space” (particalry, “Hermitian operator” as “quantity” is defined in the “classical” quantum mechanics) by that of “quantum information”. As far as wave function is the characteristic function of the probability (density) distribution for all possible values of a certain quantity to be measured, the definition of quantity in quantum mechanics means any unitary change of the probability (density) distribution. It can be represented as a particular case of “unitary” qubits. The converse interpretation of any qubits as referring to a certain physical quantity implies its generalization to non-Hermitian operators, thus neither unitary, nor conserving energy. Their physical sense, speaking loosely, consists in exchanging temporal moments therefore being implemented out of the space-time “screen”. “Dark matter” and “dark energy” can be explained by the same generalization of “quantity” to non-Hermitian operators only secondarily projected on the pseudo-Riemannian space-time “screen” of general relativity according to Einstein's “Mach’s principle” and his field equation.