Search

The impact factor (IF) is the most commonly used bibliometric method for rating academic journals. However, the practice of journals’ self-citation may artificially elevate the IF. Additional bibliometric methods including Eigenfactor scale, SCImago Journal Ranking (SJR), and corrected IF (cIF) have been created. Comparing general-interest and specialized orthopaedic journals, the aims of this study were to assess: (1) the effect of journal´s self-citation on IF; (2) differences in bibliometric analysis; and (3) to determine thresholds for monitoring self-citation practices by defining the self-citation in orthopaedic research (SCOR) Threshold. The journal citation reports and SCImago Journal and Country Rank databases were queried for orthopaedic journals from 1997 to 2017. The following bibliometrics were compared between general-interest and specialized journals: IF, cIF, Eigenfactor, self-citation rates, and SJR. A novel metric, the cIF ratio, was proposed to represent the relationship between a journal’s IF and cIF. Thresholds for cIF were based on statistical outliers of cIF ratio within general-interest and specialized journals were calculated. Outliers were defined as data points that were greater than the third quartile by 1.5 times the interquartile range using the last 10 years studied (2007–2017). Specialized orthopaedic journals had a higher median self-citation rates compared to general-interest journals (11.85% vs. 6.36%, p

Academic journals play a significant role in the dissemination of new research insights and knowledge among scientists. The number of such journals has recently increased significantly. Scientists prefer to publish their scholarly work at reputed venues. Speed of publication is also an import factor considered by many while selecting a publication venue. To evaluate a journal’s quality, few of the key indicators include impact factor, Source Normalized Impact per Paper (SNIP), and Hirsch index (h-index). Journals’ ranking is an indication of their impact and quality with respect to other venues in a specific discipline. Various measures can be utilized for ranking, like, field specific statistics, intra discipline ranking, or a combination of both. Earlier, the journals’ ranking was done through a manual process by providing an institutional list created by academic leaders. Factors like politicization, biases, and personal interests were the key issues with such categorization. Later, the process evolved to a database system based on impact factor, SNIP (Source Normalized Impact per Paper), h-index, or any combination of these. All this demanded an external source of categorizing academic journals. This work presents a data science-based framework that evaluates journals based on their key bibliometric indicators and presents an automated approach to categorize them. For this, the current proposal is restricted to the journals published in the computer science domain. The journal’s features considered in the proposed framework include: publisher, impact factor, website, CiteScore, SJR (SCImago Journal & Country Rank), SNIP, h-index, country, age, cited half-life, immediacy factor/index, Eigenfactor score, article influence score, open access, percentile, citations, acceptance rate, peer review, and the number of articles published yearly. A dataset is collected for 660 journals consisting of these 19 features. The dataset is preprocessed to fill-in the missing values and perform scaling. Three feature selection techniques, namely, Mutual Information (MI), minimum Redundancy Maximum Relevance (mRMR), and Statistical Dependency (SD) are used to rank the aforementioned features. The dataset is then vertically divided into three sets, all features, top nine features, and bottom ten features. Later, two clustering techniques, namely, k-means and k-medoids are employed to find the optimum number of coherent groups in the dataset. Based on a rigorous evaluation, four groups of journals are identified. It is followed by training two classifiers, i.e., k-NN (Nearest Neighbor) and Artificial Neural Network (ANN) to predict the category of an unknown journal. Where, the ANN shows an average accuracy of 82.85%. A descriptive analysis of the clusters formed is also presented to gain insights about the four journal categories. The proposed framework provides an opportunity to independently categorize academic journals based on data science methods using multiple significant bibliometric indicators.

Meta-syntheses from experts’ judgements and quantitative metrics are two main forms of evaluation. But they both have limitations. This paper constructs a framework for mapping the evaluation results between quantitative metrics and experts’ judgements such that they may be solved. In this way, the weights of metrics in quantitative evaluation are objectively obtained, and the validity of the results can be testified. Weighted average percentile (WAP) is employed to aggregate different experts’ judgements into standard WAP scores. The Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method is used to map quantitative results into experts’ judgements, while WAP scores are equal to the final closeness coefficients generated by the TOPSIS method. However, the closeness coefficients of TOPSIS rely on the weights of quantitative metrics. In this way, the mapping procedure is transformed into an optimization problem, and a genetic algorithm is introduced to search for the best weights. An academic journal ranking in the field of Supply Chain Management and Logistics (SCML) is used to test the validity obtained by mapping results. Four prominent ranking lists from Association of Business Schools, Australian Business Deans Council, German Academic Association for Business Research, and Comite National de la Recherche Scientifique were selected to represent different experts’ judgements. Twelve indices including IF, Eigenfactor Score (ES), H-index, Scimago Journal Ranking, and Source Normalized Impact per Paper (SNIP) were chosen for quantitative evaluation. The results reveal that the mapping results possess high validity for the relative error of experts’ judgements, the quantitative metrics are 43.4%, and the corresponding best weights are determined in the meantime. Thus, some interesting findings are concluded. First, H-index, Impact Per Publication (IPP), and SNIP play dominant roles in the SCML journal’s quality evaluation. Second, all the metrics are positively correlated, although the correlation varies among metrics. For example, ES and NE are perfectly, positively correlated with each other, yet they have the lowest correlation with the other metrics. Metrics such as IF, IFWJ, 5-year IF, and IPP are highly correlated. Third, some highly correlated metrics may perform differently in quality evaluation, such as IPP and 5-year IF. Therefore, when mapping the quantitative metrics and experts’ judgements, academic fields should be treated distinctively.

Understanding the impact of a publication by using bibliometric indices becomes an essential activity not only for universities and research institutes but also for individual academicians. This paper aims to provide a brief review of the current bibliometric tools used by authors and editors and proposes an algorithm to assess the relevance of the most common bibliometric tools to help the researchers select the fittest journal and know the trends of published submissions by using self-evaluation. We present a narrative review answering at least two related consecutive questions triggered by the topics mentioned above. How prestigious is a journal based on its most recent bibliometrics, so authors may choose it to submit their next manuscript? And, how can they self-evaluate/understand the impact of their whole publishing scientific life? We presented the main relevant definitions of each bibliometrics and grouped them in those oriented to evaluated journals or individuals. Also, we share with our readers our algorithm to assess journals before manuscript submission. Since there is a journal performance market and an article performance market, each one with its patterns, an integrative use of these metrics, rather than just the impact factor alone, might represent the fairest and most legitimate approach to assess the influence and importance of an acceptable research issue, and not only a sound journal in their respective disciplines.

We examine the relationships between four citation metrics (impact factor, the numerator of the impact factor, article influence score, and eigenfactor) and the library journal selection decisions made by Manhattan College faculty as part of a large-scale serials review. Our results show that journal selection status (selected or not) is only weakly or moderately related to citation impact. Faculty choosing journals for their universities do consider the citation data provided to them, although they place less emphasis on citation impact than do faculty responding to journal ranking surveys. While previous research suggests that subjective journal ratings are more closely related to size-independent metrics (those that represent the average impact of an article rather than the impact of the journal as a whole) and weighted metrics (those that give more credit for citations in high-impact journals), our current results provide no support for the first assertion and only limited support for the second.

A family of measures of a journal’s impact is considered that takes account of the dispersion, as well as the mean, of the number of citations in a journal. These measures, called the Mean Dispersion Indices (MDI), generalize the stabilized-JIF recently introduced by Lando and Bertoli-Barsotti (J Informetr 11(3):689–703, 2017). The MDI use a geometric weighted average of the number of citations and the Gini coefficient to measure the dispersion of the number of citations. Journal rankings based on these measures are compared with those obtained from the Journal Impact Factor, the Scimago Journal Rank, the h-index measure, and the Eigenfactor rankings, four of the most well-known current impact measures. This comparison suggests that the different rankings may implicitly place different weights on dispersion and the average number of citations and some appear to show little correlation with dispersion.

This study focuses on journals that lead their Web of Science (WoS) subject category ranking when the usual 2-year window for the Journal Impact Factor (JIF2) is used as the ranking variable, and examines evidence that contradicts their top-ranked position in the context of their group. The source data were obtained from all 177 WoS subject categories in the Science Edition 2015 Journal Citation Reports (JCR). I compared journals in each WoS subject category with leaders in terms of JIF2, number of citable items and number of citations that contribute to the JIF2. Rankings were calculated with alternative metrics (for example, the Journal Impact Factor without self-citations and the eigenfactor), and the minimum reduction in the number of citations that would displace the top-ranked journal from its leading position was also calculated. In addition, the stability of rankings over time, the number of WoS subject categories in which journals are leaders, the publishers that own leading journals, and the percentages of research articles (as opposed to review articles) published in different journals were also studied. In general, leading journals are not necessarily the top-ranked in terms of citations received or the number of citable items they publish. In addition, most leaders maintained their position when other metrics were used instead the JIF2, although rankings based on the eigenfactor were at variance with this finding. The distribution of publishers was highly skewed, with a linear relationship between the cumulative number of publishers owning the top-ranked journal and the cumulative number of WoS subject categories. In only 85 subject categories (48%) the percentage of research articles (not reviews) in the number of citable items was greater than the mean percentage for the subject category. In 31 instances, leaders did not publish any research articles, but only reviews.

Both end users and authors commonly evaluate scientific journals based on several popular journal metrics. Such metrics, in particular the "impact factor," carry crucial weight in terms of which journals authors choose for submitting scientific works as well as to what titles an institutional library subscribes. While previous research has focused on the value of journals in terms of "price per page," no study has investigated the relationship between common journal metrics and the price a journal advertises for an annual subscription. In the present study, we took a linear modeling approach using Akaike information criterion to determine which journal metric (impact factor, Eigenfactor score, article influence score, total cites, or proportion reviews) was the "best" predictor of the advertised annual subscription price for scientific journals. Examining three differing scientific fields (aquatic science, sociology, and immunology) and accounting for for-profit versus not-for-profit status, we found results to be field-dependent. Total cites was the best predicting metric for the annual advertised subscription price for aquatic science and immunology, while the Eigenfactor score was the best predictor for sociology. We hypothesize the relationship with price changes with differing magnitudes of citation flows in a field. Clear from our study was that no single measure of journal quality is universally applicable to determine subscription "value."

The literature considers a journal's h-index as a compared measure, however the relation between institutional level h-index (IHI) and journal related indices (JRI) has not been explored in previous studies at meso-level research assessment. This study applies the scientometric approach to meso-level data to examine the association, functional relationship and correlation analysis to evaluate the reliability of IHI with respect to JRI. For this purpose, data from the Web of Science, journal citation report and time cited features were used. The unit of analysis was Malaysian engineering research with a wider time span of 10 year's data (2001---2010) and a larger set of journals (1381). We explored the inter-correlation of IHI with a set of eight JRI and applied principle component analysis, regression analysis, and correlation. At the institutional level, the component analysis and functional relationship of the cumulative impact factor and 5 years IF yielded a more strong association with IHI. Cumulative impact factor is a strong predictor for IHI followed by cumulative 5 years impact factor. Correlation matrix results show that average impact factor (AIF) is correlated with immediacy index and Eigenfactor only. AIF and median impact factor (MIF) have no correlation with each other and IHI is correlated with all indices except AIF and MIF. This study puts forward a better understanding in considering new impact indices at meso level for performance evaluation purpose.

The family of indicators presented in this paper includes indices created by taking into account not only the direct but also the indirect impact of citations and references. Three types of citation graphs are presented, namely, the Paper-Citation graph, the Author-Citation graph and the Journal-Citation graph, along with different methods for constructing them. In addition, the concept of generations of citations is examined in detail, again by presenting various methods for defining them found in the literature. Finally, a number of indirect indicators for papers, authors and journals are discussed, which among others, include PageRank, CiteRank, indirect h-index and the EigenFactor score.

Here we show a novel technique for comparing subject categories, where the prestige of academic journals in each category is represented statistically by an impact-factor histogram. For each subject category we compute the probability of occurrence of scholarly journals with impact factor in different intervals. Here impact factor is measured with Thomson Reuters Impact Factor, Eigenfactor Score, and Immediacy Index. Assuming the probabilities associated with a pair of subject categories our objective is to measure the degree of dissimilarity between them. To do so, we use an axiomatic characterization for predicting dissimilarity between subject categories. The scientific subject categories of Web of Science in 2010 were used to test the proposed approach for benchmarking Cell Biology and Computer Science Information Systems with the rest as two case studies. The former is best-in-class benchmarking that involves studying the leading competitor category; the latter is strategic benchmarking that involves observing how other scientific subject categories compete.

Journals have been ranked on the basis of impact factors for a long time. This is a quality indicator, and often favours review journals with few articles. Integrated impact indicators try to factor in size (quantity) as well, and are correlated with total number of citations. The total number of papers in a portfolio can be considered a zeroth order performance indicator and the total number of citations a first order performance indicator. Indicators like the h-Index and the g-Index are actually performance indicators in that they integrate both quality and quantity assessment into a single number. The p-Index is another variant of this class of performance indicators and is based on the cubic root of a second order performance indicator called the exergy indicator. The Eigenfactor score and article influence are respectively first order quantity and quality indicators. In this paper, we confirm the above relationships.

Here we study the relationship between journal quartile rankings of ISI impact factor (at the 2010) and journal classification in four impact classes, i.e., highest impact, medium highest impact, medium lowest impact, and lowest impact journals in subject category computer science artificial intelligence. To this aim, we use fuzzy maximum likelihood estimation clustering in order to identify groups of journals sharing similar characteristics in a multivariate indicator space. The seven variables used in this analysis are: (1) Scimago Journal Ranking (SJR); (2) H-Index (H); (3) ISI impact factor (IF); (4) 5-Year Impact Factor (5IF); (5) Immediacy Index (II); (6) Eigenfactor Score (ES); and (7) Article Influence Score (AIS). The fuzzy clustering allows impact classes to overlap, thereby accommodating for uncertainty related to the confusion about the impact class attribution for a journal and vagueness in impact classes definition. This paper demonstrates the complex relationship between quartiles of ISI impact factor and journal impact classes in the multivariate indicator space. And that several indicators should be used for a distinct analysis of structural changes at the score distribution of journals in a subject category. Here we propose it can be performed in a multivariate indicator space using a fuzzy classifier.

The paper is concerned with analysing what makes a great journal great in the sciences, based on quantifiable Research Assessment Measures (RAM). Alternative RAM are discussed, with an emphasis on the Thomson Reuters ISI Web of Science database (hereafter ISI). Various ISI RAM that are calculated annually or updated daily are defined and analysed, including the classic 2-year impact factor (2YIF), 5-year impact factor (5YIF), Immediacy (or zero-year impact factor (0YIF)), Eigenfactor, Article Influence, C3PO (Citation Performance Per Paper Online), h-index, Zinfluence, PI-BETA (Papers Ignored - By Even The Authors), Impact Factor Inflation (IFI), and three new RAM, namely Historical Self-citation Threshold Approval Rating (H-STAR), 2 Year Self-citation Threshold Approval Rating (2Y-STAR), and Cited Article Influence (CAI). The RAM data are analysed for the 6 most highly cited journals in 20 highly-varied and well-known ISI categories in the sciences, where the journals are chosen on the basis of 2YIF. The application to these 20 ISI categories could be used as a template for other ISI categories in the sciences and social sciences, and as a benchmark for newer journals in a range of ISI disciplines. In addition to evaluating the 6 most highly cited journals in each of 20 ISI categories, the paper also highlights the similarities and differences in alternative RAM, finds that several RAM capture similar performance characteristics for the most highly cited scientific journals, determines that PI-BETA is not highly correlated with the other RAM, and hence conveys additional information regarding research performance. In order to provide a meta analysis summary of the RAM, which are predominantly ratios, harmonic mean rankings are presented of the 13 RAM for the 6 most highly cited journals in each of the 20 ISI categories. It is shown that emphasizing THE impact factor, specifically the 2-year impact factor, of a journal to the exclusion of other informative RAM can lead to a distorted evaluation of journal performance and influence on different disciplines, especially in view of inflated journal self citations.

Peer-reviewed publications are a major product of presentday research. Beside conference presentations, teaching and industrial project implementation, they are one of the main outputs of research work. Many, mostly statistical, systems exist for expressing the influence of publication, such as Impact Factors (2and 5-years), Total Cites, Immediacy Index, Cited Half-life, Eigenfactor , Score, Article Influence Score from ISIC Web of Knowledge Thompson Reuters (2013, admin-apps.webofknowledge. com/JCR/JCR?wsid=P2bYYMPI6B8gjp1EJx1&ssid=&SID= P2bYYMPI6B8gjp1EJx1) SRJ from SCOPUS, Elsevier (2014, www.scopus.com) and various h-indexes (Hirsch 2005). For instance, h-index is the number of papers an author has published that are cited at least that number of times, h10 index used by Google Scholar (2014, www. scholar.google.com)—the number of papers with minimum ten citations and h1 index—the number of paper with at least one citation. A key issue is that the counted references should be independent, i.e. self citations excluded. This can be sometimes rather tricky as many authors can have the same surname and even same initials—see e.g. Li, Liu, Zhang, Wang, Kim in Asia or Smith, Brown, Johansson in the western hemisphere. Perhaps a kind of a DAI—digital author identifier an equivalent to DOI—The DOI System ISO 26324—Digital Object Identifier System (2014, www. doi.org) is needed in the near future to make all those statistics credible.

To determine the degree of correlation among journal citation indices that reflect the average number of citations per article, the most recent journal ratings were downloaded from the websites publishing four journal citation indices: the Institute of Scientific Information’s journal impact factor index, Eigenfactor’s article influence index, SCImago’s journal rank index and Scopus’ trend line index. Correlations were determined for each pair of indices, using ratings from all journals that could be identified as having been rated on both indices. Correlations between the six possible pairings of the four indices were tested with Spearman’s rho. Within each of the six possible pairings, the prevalence of identifiable errors was examined in a random selection of 10 journals and among the 10 most discordantly ranked journals on the two indices. The number of journals that could be matched within each pair of indices ranged from 1,857 to 6,508. Paired ratings for all journals showed strong to very strong correlations, with Spearman’s rho values ranging from 0.61 to 0.89, all p < 0.001. Identifiable errors were more common among scores for journals that had very discordant ranks on a pair of indices. These four journal citation indices were significantly correlated, providing evidence of convergent validity (i.e. they reflect the same underlying construct of average citability per article in a journal). Discordance in the ranking of a journal on two indices was in some cases due to an error in one index.

Recent research has shown that simple graphical representations of research performance can be obtained using two-dimensional maps based on impact (i) and citations (C). The product of impact and citations leads to an energy term (E). Indeed, using E as the third coordinate, three-dimensional landscape maps can be prepared. In this paper, instead of using the traditional impact factor and total citations received for journal evaluation, Article InfluenceTM and EigenfactorTM are used as substitutes. Article Influence becomes a measure of quality (i.e. a proxy for impact factor) and Eigenfactor is a proxy for size/quantity (like citations) and taken together, the product is an energy-like term. This can be used to measure the influence/prestige of a journal. It is also possible to propose a p-factor (where p = E 1/3) as an alternative measure of the prestige or prominence of a journal which plays the equivalent role of the h-index.

The h-index and Eigenfactor TM values of top and specialized scientific/engineering journals are tabulated and combined to provide a simple graphical representation of the journals. The information may be tailored to specific uses by respective stakeholders to aid decision making processes with regards to scholarly research and scientific journal publications.