Search

Papers describing software are an important part of computational fields of scientific research. These “software papers” are unique in a number of ways, and they require special consideration to improve their impact on the scientific community and their efficacy at conveying important information. Here, we discuss 10 specific rules for writing software papers, covering some of the different scenarios and publication types that might be encountered, and important questions from which all computational researchers would benefit by asking along the way., Author summary Computational researchers have a responsibility to ensure that the software they write stands up to the same scientific scrutiny as traditional research studies. These 10 simple rules make doing so easier by enhancing usability, reproducibility, transparency, and other crucial characteristics that aren’t taught in most computer science or research methods curricula.

Peer Review is at the heart of scholarly communications and the cornerstone of scientific publishing. However, academia often criticizes the peer review system as non-transparent, biased, arbitrary, a flawed process at the heart of science, leading to researchers arguing with its reliability and quality. These problems could also be due to the lack of studies with the peer-review texts for various proprietary and confidentiality clauses. Peer review texts could serve as a rich source of Natural Language Processing (NLP) research on understanding the scholarly communication landscape, and thereby build systems towards mitigating those pertinent problems. In this work, we present a first of its kind multi-layered dataset of 1199 open peer review texts manually annotated at the sentence level (∼ 17k sentences) across the four layers, viz. Paper Section Correspondence, Paper Aspect Category, Review Functionality, and Review Significance. Given a text written by the reviewer, we annotate: to which sections (e.g., Methodology, Experiments, etc.), what aspects (e.g., Originality/Novelty, Empirical/Theoretical Soundness, etc.) of the paper does the review text correspond to, what is the role played by the review text (e.g., appreciation, criticism, summary, etc.), and the importance of the review statement (major, minor, general) within the review. We also annotate the sentiment of the reviewer (positive, negative, neutral) for the first two layers to judge the reviewer’s perspective on the different sections and aspects of the paper. We further introduce four novel tasks with this dataset, which could serve as an indicator of the exhaustiveness of a peer review and can be a step towards the automatic judgment of review quality. We also present baseline experiments and results for the different tasks for further investigations. We believe our dataset would provide a benchmark experimental testbed for automated systems to leverage on current NLP state-of-the-art techniques to address different issues with peer review quality, thereby ushering increased transparency and trust on the holy grail of scientific research validation. Our dataset and associated codes are available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html#Peer-Review-Analyze.

Women comprise a minority of the Science, Technology, Engineering, Mathematics, and Medicine (STEMM) workforce. Quantifying the gender gap may identify fields that will not reach parity without intervention, reveal underappreciated biases, and inform benchmarks for gender balance among conference speakers, editors, and hiring committees. Using the PubMed and arXiv databases, we estimated the gender of 36 million authors from >100 countries publishing in >6000 journals, covering most STEMM disciplines over the last 15 years, and made a web app allowing easy access to the data (https://lukeholman.github.io/genderGap/). Despite recent progress, the gender gap appears likely to persist for generations, particularly in surgery, computer science, physics, and maths. The gap is especially large in authorship positions associated with seniority, and prestigious journals have fewer women authors. Additionally, we estimate that men are invited by journals to submit papers at approximately double the rate of women. Wealthy countries, notably Japan, Germany, and Switzerland, had fewer women authors than poorer ones. We conclude that the STEMM gender gap will not close without further reforms in education, mentoring, and academic publishing., Author summary In most fields of science, medicine, and technology research, men comprise more than half of the workforce, particularly at senior levels. Most previous work has concluded that the gender gap is smaller today than it was in the past, giving the impression that there will soon be equal numbers of men and women researchers and that current initiatives to recruit and retain more women are working adequately. Here, we used computational methods to determine the numbers of men and women authors listed on >10 million academic papers published since 2002, allowing us to precisely estimate the gender gap among researchers, as well as its rate of change, for most disciplines of science and medicine. We conclude that many research specialties (e.g., surgery, computer science, physics, and maths) will not reach gender parity this century, given present-day rates of increase in the number of women authors. Additionally, the gender gap varies greatly across countries, with Japan, Germany, and Switzerland having strikingly few women authors. Women were less often commissioned to write ‘invited’ papers, consistent with gender bias by journal editors, and were less often found in authorship positions usually associated with seniority (i.e., the last-listed or sole author). Our results support a need for further reforms to close the gender gap.

The exposure to Non-Ionizing-Electromagnetic Fields (NI-EMFs) is often indicated as a cofactor responsible for the fertility reduction, which has been described in recent years. Despite the great interest in this topic and the research effort in exploring it, to date, there are no reliable data. Therefore, we carried out a scientometric analysis of the scientific literature published in peer reviewed Journals concerning this topic to better understand the reasons of this partial failure. To this aim, we identified and analysed 104 papers, published in last 26 years in peer-reviewed Journals, present in ISI Web of Knowledge Core Collection. Then, we analysed the impact of the Journals in which the papers were published as well as that of the single papers, the paper citation dynamics, the keywords citation busts, the geographical localization of citations and the co-authorship dynamics of the Authors. As a result, we found that different animal models (rodent, rabbit, guinea pig, and swine) and different experimental approaches (epidemiological vs. experimental studies) have the same impact, highlighting the lack of universally adopted standard in research activity. The analysis of the temporal trend in keywords and the high differences in citations between the different countries (also in those belonging to the same geographical and socio-economical area) pointed out the difficulties in approaching this branch of study. Lastly, it was evident that the Authors did not behave as a connected community, but as unconnected clusters of very small size. In conclusion, based on the results of our analysis, we think that important efforts must be undertaken to adopt more standardized models and to improve the research quality and the information exchange within the scientific community, with the aim of improving the reliability and usefulness of the results of research regarding the effect of NI-EMFs on fertility.

Innovation systems are sometimes referred to as complex systems, something that is intuitively understood but poorly defined. A complex system dynamically evolves in non-linear ways giving it unique properties that distinguish it from other systems. In particular, a common signature of complex systems is scale-invariant emergent properties. A scale-invariant property can be identified because it is solely described by a power law function, f(x) = kx^α, where the exponent, α, is a measure of scale-invariance. The focus of this paper is to describe and illustrate that innovation systems have properties of a complex adaptive system. In particular scale-invariant emergent properties indicative of their complex nature that can be quantified and used to inform public policy. The global research system is an example of an innovation system. Peer-reviewed publications containing knowledge are a characteristic output. Citations or references to these articles are an indirect measure of the impact the knowledge has on the research community. Peer-reviewed papers indexed in Scopus and in the Web of Science were used as data sources to produce measures of sizes and impact. These measures are used to illustrate how scale-invariant properties can be identified and quantified. It is demonstrated that the distribution of impact has a reasonable likelihood of being scale-invariant with scaling exponents that tended toward a value of less than 3.0 with the passage of time and decreasing group sizes. Scale-invariant correlations are shown between the evolution of impact and size with time and between field impact and sizes at points in time. The recursive or self-similar nature of scale-invariance suggests that any smaller innovation system within the global research system is likely to be complex with scale-invariant properties too. This record was migrated from the OpenDepot repository service in June, 2017 before shutting down.

Objective To conduct a time-cost analysis of formatting in scientific publishing. Design International, cross-sectional study (one-time survey). Setting Internet-based self-report survey, live between September 2018 and January 2019. Participants Anyone working in research, science, or academia and who submitted at least one peer-reviewed manuscript for consideration for publication in 2017. Completed surveys were available for 372 participants from 41 countries (60% of respondents were from Canada). Main outcome measure Time (hours) and cost (wage per hour x time) associated with formatting a research paper for publication in a peer-reviewed academic journal. Results The median annual income category was US$61,000–80,999, and the median number of publications formatted per year was four. Manuscripts required a median of two attempts before they were accepted for publication. The median formatting time was 14 hours per manuscript, or 52 hours per person, per year. This resulted in a median calculated cost of US$477 per manuscript or US$1,908 per person, per year. Conclusions To our knowledge, this is the first study to analyze the cost of manuscript formatting in scientific publishing. Our results suggest that scientific formatting represents a loss of 52 hours, costing the equivalent of US$1,908 per researcher per year. These results identify the hidden and pernicious price associated with scientific publishing and provide evidence to advocate for the elimination of strict formatting guidelines, at least prior to acceptance.