Search

The multidisciplinary and socially anchored nature of Feminist Studies presents unique challenges for bibliometric analysis, as this research area transcends traditional disciplinary boundaries and reflects discussions from feminist and LGBTQIA+ social movements. This paper proposes a novel approach for identifying gender/sex related publications scattered across diverse scientific disciplines. Using the Dimensions database, we employ bibliometric techniques, natural language processing (NLP) and manual curation to compile a dataset of scientific publications that allows for the analysis of Gender Studies and its influence across different disciplines. This is achieved through a methodology that combines a core of specialized journals with a comprehensive keyword search over titles. These keywords are obtained by applying Topic Modeling (BERTopic) to the corpus of titles and abstracts from the core. This methodological strategy, divided into two stages, reflects the dynamic interaction between Gender Studies and its dialogue with different disciplines. This hybrid system surpasses basic keyword search by mitigating potential biases introduced through manual keyword enumeration. The resulting dataset comprises over 1.9 million scientific documents published between 1668 and 2023, spanning four languages. This dataset enables a characterization of Gender Studies in terms of addressed topics, citation and collaboration dynamics, and institutional and regional participation. By addressing the methodological challenges of studying "more-than-disciplinary" research areas, this approach could also be adapted to delineate other conversations where disciplinary boundaries are difficult to disentangle., Comment: 2 tables, 5 figures

Science is increasingly global, with international collaboration playing a crucial role in advancing scientific development and knowledge exchange across borders. However, the processes that regulate how scientific labor is distributed among countries remain underexplored, leading to challenges in ensuring both effective collaboration and equitable participation across diverse scientific communities. Here, we leverage three million internationally coauthored publications produced by countries worldwide to examine the division of scientific labor in international collaboration, identify the factors that shape this distribution, and assess its broader consequences. Our findings uncover a persistent hierarchical structure in international collaboration, with researchers from scientifically advanced countries tend to occupy leading roles, while those from less-developed countries are often relegated to supportive roles, even after controlling for various influential factors. This hierarchy is also reflected in the research content, as countries with lower scientific capacity tend to participate in international collaborations that deviate from their domestic science. By analyzing the labor division within international collaborations, we demonstrate that researchers from less-developed countries face systematic disadvantages, which not only limit their contributions to the global scientific community but also prevent them from fully benefiting from international collaborations., Comment: 35 pages, 7 figures, 3 tables

Clarivate's Web of Science (WoS) and Elsevier's Scopus have been for decades the main sources of bibliometric information. Although highly curated, these closed, proprietary databases are largely biased towards English-language publications, underestimating the use of other languages in research dissemination. Launched in 2022, OpenAlex promised comprehensive, inclusive, and open-source research information. While already in use by scholars and research institutions, the quality of its metadata is currently being assessed. This paper contributes to this literature by assessing the completeness and accuracy of OpenAlex's metadata related to language, through a comparison with WoS, as well as an in-depth manual validation of a sample of 6,836 articles. Results show that OpenAlex exhibits a far more balanced linguistic coverage than WoS. However, language metadata is not always accurate, which leads OpenAlex to overestimate the place of English while underestimating that of other languages. If used critically, OpenAlex can provide comprehensive and representative analyses of languages used for scholarly publishing. However, more work is needed at infrastructural level to ensure the quality of metadata on language., Comment: Reference list updated and corrected, corresponding author's email contact added, minor typographical errors corrected

The Latin-American scientific community has achieved significant progress towards gender parity, with nearly equal representation of women and men scientists. Nevertheless, women continue to be underrepresented in scholarly communication. Throughout the 20th century, Latin America established its academic circuit, focusing on research topics of regional significance. Through an analysis of scientific publications, this article explores the relationship between gender inequalities in science and the integration of Latin-American researchers into the regional and global academic circuits between 1993 and 2022. We find that women are more likely to engage in the regional circuit, while men are more active within the global circuit. This trend is attributed to a thematic alignment between women's research interests and issues specific to Latin America. Furthermore, our results reveal that the mechanisms contributing to gender differences in symbolic capital accumulation vary between circuits. Women's work achieves equal or greater recognition compared to men's within the regional circuit, but generally garners less attention in the global circuit. Our findings suggest that policies aimed at strengthening the regional academic circuit would encourage scientists to address locally relevant topics while simultaneously fostering gender equality in science.

Global scholarly publishing has been dominated by a small number of publishers for several decades. We aimed to revisit the debate on corporate control of scholarly publishing by analyzing the relative shares of major publishers and smaller, independent publishers. Using the Web of Science, Dimensions and OpenAlex, we managed to retrieve twice as many articles indexed in Dimensions and OpenAlex, compared to the rather selective Web of Science. As a result of excluding smaller publishers, the 'oligopoly' of scholarly publishers persists, at least in appearance, according to the Web of Science. However, both Dimensions' and OpenAlex' inclusive indexing revealed the share of smaller publishers has been growing rapidly, especially since the onset of large-scale online publishing around 2000, resulting in a current cumulative dominance of smaller publishers. While the expansion of small publishers was most pronounced in the social sciences and humanities, the natural and medical sciences showed a similar trend. A major geographical divergence is also revealed, with some countries, mostly Anglo-Saxon and/or located in northwestern Europe, relying heavily on major publishers for the dissemination of their research, while others being relatively independent of the oligopoly, such as those in Latin America, northern Africa, eastern Europe and parts of Asia. The emergence of digital publishing, the reduction of expenses for printing and distribution and open-source journal management tools may have contributed to the emergence of small publishers, while the development of inclusive bibliometric databases has allowed for the effective indexing of journals and articles. We conclude that enhanced visibility to recently created, independent journals may favour their growth and stimulate global scholarly bibliodiversity.

Scopus and the Web of Science have been the foundation for research in the science of science even though these traditional databases systematically underrepresent certain disciplines and world regions. In response, new inclusive databases, notably OpenAlex, have emerged. While many studies have begun using OpenAlex as a data source, few critically assess its limitations. This study, conducted in collaboration with the OpenAlex team, addresses this gap by comparing OpenAlex to Scopus across a number of dimensions. The analysis concludes that OpenAlex is a superset of Scopus and can be a reliable alternative for some analyses, particularly at the country level. Despite this, issues of metadata accuracy and completeness show that additional research is needed to fully comprehend and address OpenAlex's limitations. Doing so will be necessary to confidently use OpenAlex across a wider set of analyses, including those that are not at all possible with more constrained databases.

Diamond open access (OA) journals offer a publishing model that is free for both authors and readers, but their lack of indexing in major bibliographic databases presents challenges in assessing the uptake of these journals. Furthermore, OA characteristics such as publication language and country of publication have often been used to support the argument that OA journals are more diverse and aim to serve a local community, but there is a current lack of empirical evidence related to the geographical and linguistic characteristics of OA journals. Using OpenAlex and the Directory of Open Access Journals as a benchmark, this paper investigates the coverage of diamond and gold through authorship and journal coverage in the Web of Science and Scopus by field, country, and language. Results show their lower coverage in WoS and Scopus, and the local scope of diamond OA. The share of English-only journals is considerably higher among gold journals. High-income countries have the highest share of authorship in every domain and type of journal, except for diamond journals in the social sciences and humanities. Understanding the current landscape of diamond OA indexing can aid the scholarly communications network with advancing policy and practices towards more inclusive OA models.

The US higher education system concentrates the production of science and scientists within a few institutions. This has implications for minoritized scholars and the topics with which they are disproportionately associated. This paper examines topical alignment between institutions and authors of varying intersectional identities, and the relationship with prestige and scientific impact. We observe a Howard-Harvard effect, in which the topical profile of minoritized scholars are amplified in mission-driven institutions and decreased in prestigious institutions. Results demonstrate a consistent pattern of inequality in topics and research impact. Specifically, we observe statistically significant differences between minoritized scholars and White men in citations and journal impact. The aggregate research profile of prestigious US universities is highly correlated with the research profile of White men, and highly negatively correlated with the research profile of minoritized women. Furthermore, authors affiliated with more prestigious institutions are associated with increasing inequalities in both citations and journal impact. Academic institutions and funders are called to create policies to mitigate the systemic barriers that prevent the United States from achieving a fully robust scientific ecosystem.

Contributorship statements have been effective at recording granular author contributions in research articles and have been broadly used to understand how labor is divided across research teams. However, one major limitation in existing empirical studies is that two classification systems have been adopted, especially from its most important data source, journals published by the Public Library of Science (PLoS). This research aims to address this limitation by developing a mapping scheme between the two systems and using it to understand whether there are differences in the assignment of contribution by authors under the two systems. We use all research articles published in PLoS ONE between 2012 to 2020, divided into two five-year publication windows centered by the shift of the classification systems in 2016. Our results show that most tasks (except for writing- and resource-related tasks) are used similarly under the two systems. Moreover, notable differences between how researchers used the two systems are also examined and discussed. This research offers an important foundation for empirical research on division of labor in the future, by enabling a larger dataset that crosses both, and potentially other, classification systems.

Investments in research and development are key to scientific and economic growth and to the well-being of society. Scientific research demands significant resources making national scientific investment a crucial driver of scientific production. As scientific production becomes increasingly multinational, it is critical to study how nations' scientific activities are funded both domestically and internationally. By tracing research grants acknowledged in scholarly publications, our study reveals a shifting duopoly of China and the United States in the global funding landscape, with a contrasting funding pattern; while China has surpassed the United States in publications with acknowledged domestic and international funding, the United States largely maintains its role as the most important global research partner. Our results also highlight the precarity of low- and middle-income countries to global funding disruptions. By revealing the complex interdependence and collaboration between countries in the global scientific enterprise, this work informs future studies investigating the national and global scientific enterprise and how funding leads to both productive cooperation and vulnerable dependencies., Comment: 48 pages, 13 figures

Empirical evidence demonstrates that citations received by scholarly publications follow a pattern of preferential attachment, resulting in a power-law distribution. Such asymmetry has sparked significant debate regarding the use of citations for research evaluation. However, a consensus has yet to be established concerning the historical trends in citation concentration. Are citations becoming more concentrated in a small number of articles? Or have recent geopolitical and technical changes in science led to more decentralized distributions? This ongoing debate stems from a lack of technical clarity in measuring inequality. Given the variations in citation practices across disciplines and over time, it is crucial to account for multiple factors that can influence the findings. This article explores how reference-based and citation-based approaches, uncited articles, citation inflation, the expansion of bibliometric databases, disciplinary differences, and self-citations affect the evolution of citation concentration. Our results indicate a decreasing trend in citation concentration, primarily driven by a decline in uncited articles, which, in turn, can be attributed to the growing significance of Asia and Europe. On the whole, our findings clarify current debates on citation concentration and show that, contrary to a widely-held belief, citations are increasingly scattered., Comment: 17 pages, 8 figures

Science is a cumulative activity, which can manifest itself through the act of citing. Citations are also central to research evaluation, thus creating incentives for researchers to cite their own work. Using a dataset containing more than 63 million articles and 51 million disambiguated authors, this paper examines the relative importance of self-citations and self-references in the scholarly communication landscape, their relationship with the age and gender of authors, as well as their effects on various research evaluation indicators. Results show that self-citations and self-references evolve in different directions throughout researchers' careers, and that men and older researchers are more likely to self-cite. Although self-citations have, on average, a small to moderate effect on author's citation rates, they highly inflate citations for a subset of researchers. Comparison of the abstracts of cited and citing papers to assess the relatedness of different types of citations shows that self-citations are more similar to each other than other types of citations, and therefore more relevant. However, researchers that self-reference more tend to include less relevant citations. The paper concludes with a discussion of the role of self-citations in scholarly communication.

Recent gender policies in the Middle East and North Africa (MENA) region have improved legal equality for women with noticeable effects in some countries. The implications of these policies on science, however, is not well-understood. This study applies a bibliometric lens to describe the landscape of gender disparities in scientific research in MENA. Specifically, we examine 1.7 million papers indexed in the Web of Science published by 1.1 million authors from MENA between 2008 and 2020. We used bibliometric indicators to analyse potential disparities between men and women in the share of authors, research productivity, and seniority in authorship. The results show that gender parity is far from being achieved in MENA. Overall, men authors obtain higher representation, research productivity, and seniority. But some countries standout: Tunisia, Lebanon, Turkey, Algeria and Egypt have higher shares or women researchers compared to the rest of MENA countries. The UAE, Qatar, and Jordan have shown progress in terms of women participation in science, but Saudi Arabia lags behind. We find that women are more likely to stop publishing than men and that men publish on average between 11% and 51% more than women, with this gap increasing over time. Finally, men, on average, achieved senior positions in authorship faster than women. Our longitudinal study contributes to a better understanding of gender disparities in science in MENA which is catching up in terms of policy engagement and women representation. However, the results suggest that the effects of the policy changes have yet to materialize into distinct improvement in women's participation and performance in science., Comment: 29 pages, 11 figures, 4 tables

Diversity in human capital is widely seen as critical to creating holistic and high quality research, especially in areas that engage with diverse cultures, environments, and challenges. Quantifying diverse academic collaborations and its effect on research quality is lacking, especially at international scale and across different domains. Here, we present the first effort to measure the impact of geographic diversity in coauthorships on the citation of their papers across different academic domains. Our results unequivocally show that geographic coauthor diversity improves paper citation, but very long distance collaborations has variable impact. We also discover "well-trodden" collaboration circles that yield much less impact than similar travel distances. These relationships are observed to exist across different subject areas, but with varying strengths. These findings can help academics identify new opportunities from a diversity perspective, as well as inform funders on areas that require additional mobility support., Comment: 26 pages, 16 figures

Disagreement is essential to scientific progress. However, the extent of disagreement in science, its evolution over time, and the fields in which it happens, remains poorly understood. Leveraging a massive collection of English-language scientific texts, we develop a cue-phrase based approach to identify instances of disagreement citations across more than four million scientific articles. Using this method, we construct an indicator of disagreement across scientific fields over the 2000-2015 period. In contrast with black-box text classification methods, our framework is transparent and easily interpretable. We reveal a disciplinary spectrum of disagreement, with higher disagreement in the social sciences and lower disagreement in physics and mathematics. However, detailed disciplinary analysis demonstrates heterogeneity across sub-fields, revealing the importance of local disciplinary cultures and epistemic characteristics of disagreement. Paper-level analysis reveals notable episodes of disagreement in science, and illustrates how methodological artifacts can confound analyses of scientific texts. These findings contribute to a broader understanding of disagreement and establish a foundation for future research to understanding key processes underlying scientific progress., Comment: 49 pages, 10 figures

Science is essential to innovation and economic prosperity. Although studies have shown that national scientific development is affected by geographic, historic, and economic factors, it remains unclear whether there are universal structures and trajectories of national scientific development that can inform forecasting and policymaking. Here, by examining countries' scientific 'exports'-publications that are indexed in international databases-we reveal a three-cluster structure in the relatedness network of disciplines that underpin national scientific development and the organization of global science. Tracing the evolution of national research portfolios reveals that while nations are proceeding to more diverse research profiles individually, scientific production is increasingly specialized in global science over the past decades. By uncovering the underlying structure of scientific development and connecting it with economic development, our results may offer a new perspective on the evolution of global science., Comment: 30 pages(main text), 5 figures(main text), 3 tables(main text)

Racial disparity in academia is a widely acknowledged problem. The quantitative understanding of racial based systemic inequalities is an important step towards a more equitable research system. However, because of the lack of robust information on authors' race, few large scale analyses have been performed on this topic. Algorithmic approaches offer one solution, using known information about authors, such as their names, to infer their perceived race. As with any other algorithm, the process of racial inference can generate biases if it is not carefully considered. The goal of this article is to assess the extent to which algorithmic bias is introduced using different approaches for name based racial inference. We use information from the U.S. Census and mortgage applications to infer the race of U.S. affiliated authors in the Web of Science. We estimate the effects of using given and family names, thresholds or continuous distributions, and imputation. Our results demonstrate that the validity of name based inference varies by race/ethnicity and that threshold approaches underestimate Black authors and overestimate White authors. We conclude with recommendations to avoid potential biases. This article lays the foundation for more systematic and less biased investigations into racial disparities in science.

One of the challenges in machine learning research is to ensure that presented and published results are sound and reliable. Reproducibility, that is obtaining similar results as presented in a paper or talk, using the same code and data (when available), is a necessary step to verify the reliability of research findings. Reproducibility is also an important step to promote open and accessible research, thereby allowing the scientific community to quickly integrate new findings and convert ideas to practice. Reproducibility also promotes the use of robust experimental workflows, which potentially reduce unintentional errors. In 2019, the Neural Information Processing Systems (NeurIPS) conference, the premier international conference for research in machine learning, introduced a reproducibility program, designed to improve the standards across the community for how we conduct, communicate, and evaluate machine learning research. The program contained three components: a code submission policy, a community-wide reproducibility challenge, and the inclusion of the Machine Learning Reproducibility checklist as part of the paper submission process. In this paper, we describe each of these components, how it was deployed, as well as what we were able to learn from this initiative., Comment: To appear at JMLR, 16 pages + Appendix

Tenure provides a permanent position to faculty in higher education institutions. In North America, it is granted to those who have established a record of excellence in research, teaching and services in a limited period. However, in China, research excellence represented by the number of Web of Science publications is highly weighted in the tenure assessment compared to excellence in teaching and services, but this has never been systematically investigated. By analyzing the tenure assessment documents from Chinese universities, this study reveals the role of Web of Science publications in China tenure system and presents the landscape of the tenure assessment process in Chinese higher education institutions., Comment: Accepted by Scientometrics

We analysed a dataset of scientific manuscripts that were submitted to various conferences in artificial intelligence. We performed a combination of semantic, lexical and psycholinguistic analyses of the full text of the manuscripts and compared them with the outcome of the peer review process. We found that accepted manuscripts scored lower than rejected manuscripts on two indicators of readability, and that they also used more scientific and artificial intelligence jargon. We also found that accepted manuscripts were written with words that are less frequent, that are acquired at an older age, and that are more abstract than rejected manuscripts. The analysis of references included in the manuscripts revealed that the subset of accepted submissions were more likely to cite the same publications. This finding was echoed by pairwise comparisons of the word content of the manuscripts (i.e. an indicator or semantic similarity), which were more similar in the subset of accepted manuscripts. Finally, we predicted the peer review outcome of manuscripts with their word content, with words related to machine learning and neural networks positively related with acceptance, whereas words related to logic, symbolic processing and knowledge-based systems negatively related with acceptance.

Contemporary scientific exchanges are international, yet language continues to be a persistent barrier to scientific communication, particularly for non-native English-speaking scholars. Since the ability to absorb knowledge has a strong impact on how researchers create new scientific knowledge, a comprehensive access to and understanding of both domestic and international scientific publications is essential for scientific performance. This study explores the effect of absorbed knowledge on research impact by analyzing the relationship between the language diversity of cited references and the number of citations received by the citing paper. Chinese social sciences are taken as the research object, and the data, 950,302 papers published between 1998 and 2013 with 8,151,327 cited references, were collected from the Chinese Social Sciences Citation Index. Results show that there is a stark increase in the consumption of foreign language material within the Chinese social science community, and English material accounts for the vast majority of this consumption. Papers with foreign language references receive significantly more citations than those without, and the citation advantage of these internationalized work holds when we control for characteristics of the citing papers. However, the citation advantage has decreased from 1998 to 2008, largely as an artifact of the increased number of papers citing foreign language material. After 2008, the decline of the citation advantage subsided and became relatively stable, which suggests that incorporating foreign language literature continues to increase scientific impact, even as the scientific community itself becomes increasingly international. However, internationalization is not without concerns: the work closes with a discussion of the benefits and potential problems of the lack of linguistic diversity in scientific communication., Comment: 24 pages, 9 figures, 10 tables

The number of publications and the number of citations received have become the most common indicators of scholarly success. In this context, scientific writing increasingly plays an important role in scholars' scientific careers. To understand the relationship between scientific writing and scientific impact, this paper selected 12 variables of linguistic complexity as a proxy for depicting scientific writing. We then analyzed these features from 36,400 full-text Biology articles and 1,797 full-text Psychology articles. These features were compared to the scientific impact of articles, grouped into high, medium, and low categories. The results suggested no practical significant relationship between linguistic complexity and citation strata in either discipline. This suggests that textual complexity plays little role in scientific impact in our data sets., Comment: 27pages, 6 figures, accepted by Journal of Informetrics

The purpose of this study is to compare Web of Science (WoS) with a Chinese bibliometric database in terms of authors and their performance, demonstrate the extent of the overlap between the two groups of Chinese most productive authors in both international and Chinese bibliometric databases, and determine how different disciplines may affect this overlap. The results of this study indicate that Chinese bibliometric databases, or a combination of WoS and Chinese bibliometric databases, should be used to evaluate Chinese research performance except in few disciplines in which Chinese research performance could be assessed using WoS only.

This paper presents and describes the methodological opportunities offered by bibliometric data to produce indicators of scientific mobility. Large bibliographic datasets of disambiguated authors and their affiliations allow for the possibility of tracking the affiliation changes of scientists. Using the Web of Science as data source, we analyze the distribution of types of mobile scientists for a selection of countries. We explore the possibility of creating profiles of international mobility at the country level, and discuss potential interpretations and caveats. Five countries (Canada, The Netherlands, South Africa, Spain, and the United States) are used as examples. These profiles enable us to characterize these countries in terms of their strongest links with other countries. This type of analysis reveals circulation among and between countries with strong policy implications.

This paper presents a methodological framework for developing scientific mobility indicators based on bibliometric data. We identify nearly 16 million individual authors from publications covered in the Web of Science for the 2008-2015 period. Based on the information provided across individuals' publication records, we propose a general classification for analyzing scientific mobility using institutional affiliation changes. We distinguish between migrants--authors who have ruptures with their country of origin--and travelers--authors who gain additional affiliations while maintaining affiliation with their country of origin. We find that 3.7 percent of researchers who have published at least one paper over the period are mobile. Travelers represent 72.7 percent of all mobile scholars, but migrants have higher scientific impact. We apply this classification at the country level, expanding the classification to incorporate the directionality of scientists' mobility (i.e., incoming and outgoing). We provide a brief analysis to highlight the utility of the proposed taxonomy to study scholarly mobility and discuss the implications for science policy., Comment: Paper accepted for publication in Journal of Informetrics

The Journal Impact Factor (JIF) is, by far, the most discussed bibliometric indicator. Since its introduction over 40 years ago, it has had enormous effects on the scientific ecosystem: transforming the publishing industry, shaping hiring practices and the allocation of resources, and, as a result, reorienting the research activities and dissemination practices of scholars. Given both the ubiquity and impact of the indicator, the JIF has been widely dissected and debated by scholars of every disciplinary orientation. Drawing on the existing literature as well as on original research, this chapter provides a brief history of the indicator and highlights well-known limitations-such as the asymmetry between the numerator and the denominator, differences across disciplines, the insufficient citation window, and the skewness of the underlying citation distributions. The inflation of the JIF and the weakening predictive power is discussed, as well as the adverse effects on the behaviors of individual actors and the research enterprise. Alternative journal-based indicators are described and the chapter concludes with a call for responsible application and a commentary on future developments in journal indicators., Comment: 33 pages, Forthcoming in Glanzel, W., Moed, H.F., Schmoch U., Thelwall, M. (2018). Springer Handbook of Science and Technology Indicators. Cham (Switzerland): Springer International Publishing

In this study we have investigated the relationship between different document characteristics and the number of Mendeley readership counts, tweets, Facebook posts, mentions in blogs and mainstream media for 1.3 million papers published in journals covered by the Web of Science (WoS). It aims to demonstrate that how factors affecting various social media-based indicators differ from those influencing citations and which document types are more popular across different platforms. Our results highlight the heterogeneous nature of altmetrics, which encompasses different types of uses and user groups engaging with research on social media., Comment: Presented at the 21th International Conference in Science & Technology Indicators (STI), 13-16, September, 2016, Valencia, Spain

Acknowledgments are one of many conventions by which researchers publicly bestow recognition towards individuals, organizations and institutions that contributed in some way to the work that led to publication. Combining data on both co-authors and acknowledged individuals, the present study analyses disciplinary differences in researchers credit attribution practices in collaborative context. Our results show that the important differences traditionally observed between disciplines in terms of team size are greatly reduced when acknowledgees are taken into account. Broadening the measurement of collaboration beyond co-authorship by including individuals credited in the acknowledgements allows for an assessment of collaboration practices and team work that might be closer to the reality of contemporary research, especially in the social sciences and humanities.

Social media has become integrated into the fabric of the scholarly communication system in fundamental ways: principally through scholarly use of social media platforms and the promotion of new indicators on the basis of interactions with these platforms. Research and scholarship in this area has accelerated since the coining and subsequent advocacy for altmetrics -- that is, research indicators based on social media activity. This review provides an extensive account of the state-of-the art in both scholarly use of social media and altmetrics. The review consists of two main parts: the first examines the use of social media in academia, examining the various functions these platforms have in the scholarly communication process and the factors that affect this use. The second part reviews empirical studies of altmetrics, discussing the various interpretations of altmetrics, data collection and methodological limitations, and differences according to platform. The review ends with a critical discussion of the implications of this transformation in the scholarly communication system., Comment: preprint, submitted to JASIST

Scientific abstracts contain what is considered by the author(s) as information that best describe documents' content. They represent a compressed view of the informational content of a document and allow readers to evaluate the relevance of the document to a particular information need. However, little is known on their composition. This paper contributes to the understanding of the structure of abstracts, by comparing similarity between scientific abstracts and the text content of research articles. More specifically, using sentence-based similarity metrics, we quantify the phenomenon of text re-use in abstracts and examine the positions of the sentences that are similar to sentences in abstracts in the IMRaD structure (Introduction, Methods, Results and Discussion), using a corpus of over 85,000 research articles published in the seven PLOS journals. We provide evidence that 84% of abstract have at least one sentence in common with the body of the article. Our results also show that the sections of the paper from which abstract sentence are taken are invariant across the PLOS journals, with sentences mainly coming from the beginning of the introduction and the end of the conclusion., Comment: Preprint: Journal of Documentation, vol. 72, issue 4. Submitted 12-Sep-2015, accepted 10-Feb-2016. 13 pages

In most countries, basic research is supported by research councils that select, after peer review, the individuals or teams that are to receive funding. Unfortunately, the number of grants these research councils can allocate is not infinite and, in most cases, a minority of the researchers receive the majority of the funds. However, evidence as to whether this is an optimal way of distributing available funds is mixed. The purpose of this study is to measure the relation between the amount of funding provided to 12,720 researchers in Quebec over a fifteen year period (1998-2012) and their scientific output and impact from 2000 to 2013. Our results show that both in terms of the quantity of papers produced and of their scientific impact, the concentration of research funding in the hands of a so-called "elite" of researchers generally produces diminishing marginal returns. Also, we find that the most funded researchers do not stand out in terms of output and scientific impact., Comment: Accepted for publication in Research Evaluation

Social media metrics - commonly coined as "altmetrics" - have been heralded as great democratizers of science, providing broader and timelier indicators of impact than citations. These metrics come from a range of sources, including Twitter, blogs, social reference managers, post-publication peer review, and other social media platforms. Social media metrics have begun to be used as indicators of scientific impact, yet the theoretical foundation, empirical validity, and extent of use of platforms underlying these metrics lack thorough treatment in the literature. This editorial provides an overview of terminology and definitions of altmetrics and summarizes current research regarding social media use in academia, social media metrics as well as data reliability and validity. The papers of the special issue are introduced., Comment: Guest Editorial to the special issue "Social Media Metrics in Scholarly Communication: Exploring Tweets, Blogs, Likes and other Altmetrics" in Aslib Journal of Information Management 67(3)

This paper provides the first historical analysis of the relationship between collaboration and scientific impact, using three indicators of collaboration (number of authors, number of addresses, and number of countries) and including articles published between 1900 and 2011. The results demonstrate that an increase in the number of authors leads to an increase in impact--from the beginning of the last century onwards--and that this is not simply due to self-citations. A similar trend is also observed for the number of addresses and number of countries represented in the byline of an article. However, the constant inflation of collaboration since 1900 has resulted in diminishing citation returns: larger and more diverse (in terms of institutional and country affiliation) teams are necessary to realize higher impact. The paper concludes with a discussion of the potential causes of the impact gain in citations of collaborative papers., Comment: 18 pages, 6 figures, Journal of the Association for Information Science and Technology, 2014

This brief communication presents preliminary findings on automated Twitter accounts distributing links to scientific papers deposited on the preprint repository arXiv. It discusses the implication of the presence of such bots from the perspective of social media metrics (altmetrics), where mentions of scholarly documents on Twitter have been suggested as a means of measuring impact that is both broader and timelier than citations. We present preliminary findings that automated Twitter accounts create a considerable amount of tweets to scientific papers and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment. We discuss some definitions of Twitter cyborgs and bots in scholarly communication and propose differentiating between different levels of engagement from tweeting only bibliographic information to discussing or commenting on the content of a paper., Comment: 9 pages, 4 figures, 1 table

MELIBEA is a Spanish database that uses a composite formula with eight weighted conditions to estimate the effectiveness of Open Access mandates (registered in ROARMAP). We analyzed 68 mandated institutions for publication years 2011-2013 to determine how well the MELIBEA score and its individual conditions predict what percentage of published articles indexed by Web of Knowledge is deposited in each institution's OA repository, and when. We found a small but significant positive correlation (0.18) between MELIBEA score and deposit percentage. We also found that for three of the eight MELIBEA conditions (deposit timing, internal use, and opt-outs), one value of each was strongly associated with deposit percentage or deposit latency (immediate deposit required, deposit required for performance evaluation, unconditional opt-out allowed for the OA requirement but no opt-out for deposit requirement). When we updated the initial values and weights of the MELIBEA formula for mandate effectiveness to reflect the empirical association we had found, the score's predictive power doubled (.36). There are not yet enough OA mandates to test further mandate conditions that might contribute to mandate effectiveness, but these findings already suggest that it would be useful for future mandates to adopt these three conditions so as to maximize their effectiveness, and thereby the growth of OA., Comment: 27 pages, 13 figures, 3 tables, 40 references, 7761 words

This paper analyzes the tweeting behavior of 37 astrophysicists on Twitter and compares their tweeting behavior with their publication behavior and citation impact to show whether they tweet research-related topics or not. Astrophysicists on Twitter are selected to compare their tweets with their publications from Web of Science. Different user groups are identified based on tweeting and publication frequency. A moderate negative correlation (p=-0.390*) is found between the number of publications and tweets per day, while retweet and citation rates do not correlate. The similarity between tweets and abstracts is very low (cos=0.081). User groups show different tweeting behavior such as retweeting and including hashtags, usernames and URLs. The study is limited in terms of the small set of astrophysicists. Results are not necessarily representative of the entire astrophysicist community on Twitter and they most certainly do not apply to scientists in general. Future research should apply the methods to a larger set of researchers and other scientific disciplines. To a certain extent, this study helps to understand how researchers use Twitter. The results hint at the fact that impact on Twitter can neither be equated with nor replace traditional research impact metrics. However, tweets and other so-called altmetrics might be able to reflect other impact of scientists such as public outreach and science communication. To the best of our knowledge, this is the first in-depth study comparing researchers' tweeting activity and behavior with scientific publication output in terms of quantity, content and impact., Comment: 14 pages, 5 figures, 7 tables

A set of 1.4 million biomedical papers was analyzed with regards to how often articles are mentioned on Twitter or saved by users on Mendeley. While Twitter is a microblogging platform used by a general audience to distribute information, Mendeley is a reference manager targeted at an academic user group to organize scholarly literature. Both platforms are used as sources for so-called altmetrics to measure a new kind of research impact. This analysis shows in how far they differ and compare to traditional citation impact metrics based on a large set of PubMed papers.

Over the last few years, several major scientific fraud cases have shocked the scientific community. The number of retractions each year has also increased tremendously, especially in the biomedical field, and scientific misconduct accounts for approximately more than half of those retractions. It is assumed that co-authors of retracted papers are affected by their colleagues' misconduct, and the aim of this study is to provide empirical evidence of the effect of retractions in biomedical research on co-authors' research careers. Using data from the Web of Science (WOS), we measured the productivity, impact and collaboration of 1,123 co-authors of 293 retracted articles for a period of five years before and after the retraction. We found clear evidence that collaborators do suffer consequences of their colleagues' misconduct, and that a retraction for fraud has higher consequences than a retraction for error. Our results also suggest that the extent of these consequences is closely linked with the ranking of co-authors on the retracted paper, being felt most strongly by first authors, followed by the last authors, while the impact is less important for middle authors., Comment: Accepted for publication in the Journal of the Association for Information Science and Technology

Genre is considered to be an important element in scholarly communication and in the practice of scientific disciplines. However, scientometric studies have typically focused on a single genre, the journal article. The goal of this study is to understand the role that handbooks play in knowledge creation and diffusion and their relationship with the genre of journal articles, particularly in highly interdisciplinary and emergent social science and humanities disciplines. To shed light on these questions we focused on handbooks and journal articles published over the last four decades belonging to the research area of Science and Technology Studies (STS), broadly defined. To get a detailed picture we used the full-text of five handbooks (500,000 words) and a well-defined set of 11,700 STS articles. We confirmed the methodological split of STS into qualitative and quantitative (scientometric) approaches. Even when the two traditions explore similar topics (e.g., science and gender) they approach them from different starting points. The change in cognitive foci in both handbooks and articles partially reflects the changing trends in STS research, often driven by technology. Using text similarity measures we found that, in the case of STS, handbooks play no special role in either focusing the research efforts or marking their decline. In general, they do not represent the summaries of research directions that have emerged since the previous edition of the handbook., Comment: Accepted for publication in Journal of Informetrics

Data collected by social media platforms have recently been introduced as a new source for indicators to help measure the impact of scholarly research in ways that are complementary to traditional citation-based indicators. Data generated from social media activities related to scholarly content can be used to reflect broad types of impact. This paper aims to provide systematic evidence regarding how often Twitter is used to diffuse journal articles in the biomedical and life sciences. The analysis is based on a set of 1.4 million documents covered by both PubMed and Web of Science (WoS) and published between 2010 and 2012. The number of tweets containing links to these documents was analyzed to evaluate the degree to which certain journals, disciplines, and specialties were represented on Twitter. It is shown that, with less than 10% of PubMed articles mentioned on Twitter, its uptake is low in general. The relationship between tweets and WoS citations was examined for each document at the level of journals and specialties. The results show that tweeting behavior varies between journals and specialties and correlations between tweets and citations are low, implying that impact metrics based on tweets are different from those based on citations. A framework utilizing the coverage of articles and the correlation between Twitter mentions and citations is proposed to facilitate the evaluation of novel social-media based metrics and to shed light on the question in how far the number of tweets is a valid metric to measure research impact., Comment: 22 pages, 4 figures, 5 tables

Since its creation in 1991, arXiv has become central to the diffusion of research in a number of fields. Combining data from the entirety of arXiv and the Web of Science (WoS), this paper investigates (a) the proportion of papers across all disciplines that are on arXiv and the proportion of arXiv papers that are in the WoS, (b) elapsed time between arXiv submission and journal publication, and (c) the aging characteristics and scientific impact of arXiv e-prints and their published version. It shows that the proportion of WoS papers found on arXiv varies across the specialties of physics and mathematics, and that only a few specialties make extensive use of the repository. Elapsed time between arXiv submission and journal publication has shortened but remains longer in mathematics than in physics. In physics, mathematics, as well as in astronomy and astrophysics, arXiv versions are cited more promptly and decay faster than WoS papers. The arXiv versions of papers - both published and unpublished - have lower citation rates than published papers, although there is almost no difference in the impact of the arXiv versions of both published and unpublished papers., Comment: 29 pages, 11 figures

Previous work indicates that over the past 20 years, the highest quality work have been published in an increasingly diverse and larger group of journals. In this paper we examine whether this diversification has also affected the handful of elite journals that are traditionally considered to be the best. We examine citation patterns over the past 40 years of 7 long-standing traditionally elite journals and 6 journals that have been increasing in importance over the past 20 years. To be among the top 5% or 1% cited papers, papers now need about twice as many citations as they did 40 years ago. Since the late 1980s and early 1990s elite journals have been publishing a decreasing proportion of these top cited papers. This also applies to the two journals that are typically considered as the top venues and often used as bibliometric indicators of "excellence", Science and Nature. On the other hand, several new and established journals are publishing an increasing proportion of most cited papers. These changes bring new challenges and opportunities for all parties. Journals can enact policies to increase or maintain their relative position in the journal hierarchy. Researchers now have the option to publish in more diverse venues knowing that their work can still reach the same audiences. Finally, evaluators and administrators need to know that although there will always be a certain prestige associated with publishing in "elite" journals, journal hierarchies are in constant flux so inclusion of journals into this group is not permanent., Comment: 12 pages, 4 figures

We have now tested the Finch Committee's Hypothesis that Green Open Access Mandates are ineffective in generating deposits in institutional repositories. With data from ROARMAP on institutional Green OA mandates and data from ROAR on institutional repositories, we show that deposit number and rate is significantly correlated with mandate strength (classified as 1-12): The stronger the mandate, the more the deposits. The strongest mandates generate deposit rates of 70%+ within 2 years of adoption, compared to the un-mandated deposit rate of 20%. The effect is already detectable at the national level, where the UK, which has the largest proportion of Green OA mandates, has a national OA rate of 35%, compared to the global baseline of 25%. The conclusion is that, contrary to the Finch Hypothesis, Green Open Access Mandates do have a major effect, and the stronger the mandate, the stronger the effect (the Liege ID/OA mandate, linked to research performance evaluation, being the strongest mandate model). RCUK (as well as all universities, research institutions and research funders worldwide) would be well advised to adopt the strongest Green OA mandates and to integrate institutional and funder mandates., Comment: 6 pages, 1 table, 4 figures

Most refereed journal articles today are published in subscription journals, accessible only to subscribing institutions, hence losing considerable research impact. Making articles freely accessible online ("Open Access," OA) maximizes their impact. Articles can be made OA in two ways: by self-archiving them on the web ("Green OA") or by publishing them in OA journals ("Gold OA"). We compared the percent and growth rate of Green and Gold OA for 14 disciplines in two random samples of 1300 articles per discipline out of the 12,500 journals indexed by Thomson-Reuters-ISI using a robot that trawled the web for OA full-texts. We sampled in 2009 and 2011 for publication year ranges 1998-2006 and 2005-2010, respectively. Green OA (21.4%) exceeds Gold OA (2.4%) in proportion and growth rate in all but the biomedical disciplines, probably because it can be provided for all journals articles and does not require paying extra Gold OA publication fees. The spontaneous overall OA growth rate is still very slow (about 1% per year). If institutions make Green OA self-archiving mandatory, however, it triples percent Green OA as well as accelerating its growth rate.

Historically, papers have been physically bound to the journal in which they were published but in the electronic age papers are available individually, no longer tied to their respective journals. Hence, papers now can be read and cited based on their own merits, independently of the journal's physical availability, reputation, or Impact Factor. We compare the strength of the relationship between journals' Impact Factors and the actual citations received by their respective papers from 1902 to 2009. Throughout most of the 20th century, papers' citation rates were increasingly linked to their respective journals' Impact Factors. However, since 1990, the advent of the digital age, the strength of the relation between Impact Factors and paper citations has been decreasing. This decrease began sooner in physics, a field that was quicker to make the transition into the electronic domain. Furthermore, since 1990, the proportion of highly cited papers coming from highly cited journals has been decreasing, and accordingly, the proportion of highly cited papers not coming from highly cited journals has also been increasing. Should this pattern continue, it might bring an end to the use of the Impact Factor as a way to evaluate the quality of journals, papers and researchers., Comment: 14 pages, 5 figures

Using the participation in peer reviewed publications of all doctoral students in Quebec over the 2000-2007 period this paper provides the first large scale analysis of their research effort. It shows that PhD students contribute to about a third of the publication output of the province, with doctoral students in the natural and medical sciences being present in a higher proportion of papers published than their colleagues of the social sciences and humanities. Collaboration is an important component of this socialization: disciplines in which student collaboration is higher are also those in which doctoral students are the most involved in peer-reviewed publications. In terms of scientific impact, papers co-signed by doctorate students obtain significantly lower citation rates than other Quebec papers, except in natural sciences and engineering. Finally, this paper shows that involving doctoral students in publications is positively linked with degree completion and ulterior career in research., Comment: 41 pages, 7 figures, forthcoming in Scientometrics

This paper examines the proximity of authors to those they cite using degrees of separation in a co-author network, essentially using collaboration networks to expand on the notion of self-citations. While the proportion of direct self-citations (including co-authors of both citing and cited papers) is relatively constant in time and across specialties in the natural sciences (10% of citations) and the social sciences (20%), the same cannot be said for citations to authors who are members of the co-author network. Differences between fields and trends over time lie not only in the degree of co-authorship which defines the large-scale topology of the collaboration network, but also in the referencing practices within a given discipline, computed by defining a propensity to cite at a given distance within the collaboration network. Overall, there is little tendency to cite those nearby in the collaboration network, excluding direct self-citations. By analyzing these social references, we characterize the social capital of local collaboration networks in terms of the knowledge production within scientific fields. These results have implications for the long-standing debate over biases common to most types of citation analysis, and for understanding citation practices across scientific disciplines over the past 50 years. In addition, our findings have important practical implications for the availability of 'arm's length' expert reviewers of grant applications and manuscripts.

Articles whose authors make them Open Access (OA) by self-archiving them online are cited significantly more than articles accessible only to subscribers. Some have suggested that this "OA Advantage" may not be causal but just a self-selection bias, because authors preferentially make higher-quality articles OA. To test this we compared self-selective self-archiving with mandatory self-archiving for a sample of 27,197 articles published 2002-2006 in 1,984 journals. The OA Advantage proved just as high for both. Logistic regression showed that the advantage is independent of other correlates of citations (article age; journal impact factor; number of co-authors, references or pages; field; article type; or country) and greatest for the most highly cited articles. The OA Advantage is real, independent and causal, but skewed. Its size is indeed correlated with quality, just as citations themselves are (the top 20% of articles receive about 80% of all citations). The advantage is greater for the more citeable articles, not because of a quality bias from authors self-selecting what to make OA, but because of a quality advantage, from users self-selecting what to use and cite, freed by OA from the constraints of selective accessibility to subscribers only., Comment: 30 pages, 15 figures, 8 tables