Search

Bibliometric indicators are increasingly used to evaluate individual scientists–as is exemplified by the popularity of the many other publication and citation-based indicators used in evaluation. These indicators, however, cover at best some of the quality dimensions relevant for assessing a researcher: productivity and impact. At the same time, research quality has more dimensions than productivity and impact alone. As current bibliometric indicators are not covering various important quality dimensions, we here contribute to developing better indicators for those quality dimensions not yet addressed. One of the quality dimensions lacking valid indicators is an individual researcher’s independence. We propose indicators to measure different aspects of independence: two assessing whether a researcher has developed an own collaboration network and two others assessing the level of thematic independence. Taken together they form an independence indicator. We illustrate how these indicators distinguish between researchers that are equally productive and have a considerable impact. The independence indicator is a step forward in evaluating individual scholarly quality. [ABSTRACT FROM AUTHOR]

Publication pressure does not exist (and if it did, it would be a good thing): Reflection and debate initiates academically inspired discussions on issues that are on the current policy agenda. [ABSTRACT FROM AUTHOR]

It is often argued that female researchers publish on average less than male researchers do, but male and female authored papers have an equal impact. In this paper we try to better understand this phenomenon by (i) comparing the share of male and female researchers within different productivity classes, and (ii) by comparing productivity whereas controlling for a series of relevant covariates. The study is based on a disambiguated Swedish author dataset, consisting of 47,000 researchers and their WoS-publications during the period of 2008-2011 with citations until 2015. As the analysis shows, in order to have impact quantity does make a difference for male and female researchers alike—but women are vastly underrepresented in the group of most productive researchers. We discuss and test several possible explanations of this finding, using a data on personal characteristics from several Swedish universities. Gender differences in age, authorship position, and academic rank do explain quite a part of the productivity differences. [ABSTRACT FROM AUTHOR]

Do highly productive researchers have significantly higher probability to produce top cited papers? Or do high productive researchers mainly produce a sea of irrelevant papers—in other words do we find a diminishing marginal result from productivity? The answer on these questions is important, as it may help to answer the question of whether the increased competition and increased use of indicators for research evaluation and accountability focus has perverse effects or not. We use a Swedish author disambiguated dataset consisting of 48.000 researchers and their WoS-publications during the period of 2008–2011 with citations until 2014 to investigate the relation between productivity and production of highly cited papers. As the analysis shows, quantity does make a difference. [ABSTRACT FROM AUTHOR]

A recent paper in this journal compares the Norwegian model of using publications counts for university funding with a similar intervention in Australia in the mid-1990 s. The authors argue that the Norwegian model (taking into account the quality of publications) performs better than the Australian (which did neglect paper quality other than being peer reviewed). We argue that these conclusions are in contrast to the evidence provided in the article, and therefore should be considered incorrect. [ABSTRACT FROM AUTHOR]

In a recent issue of this journal, we analyzed decision making about grant applications. We argued that there are many uncertainties in selecting the best talents, and we showed that in quite some cases (i) peer review scores hardly influence the decisions, and (ii) differences between success and no success are very small. The council we studied contested our conclusions. In this short article, we present additional evidence supporting our claims. [ABSTRACT FROM AUTHOR]

The research environment of scholars is increasingly web-based. This makes it urgent to study the effects of moving to the Web on research practices, scholarly output and innovation. We propose a theoretical framework and a methodology to study these effects. In a pilot study, we apply theory and method on an online community in biodiversity research, to demonstrate the feasibility of the approach. We also indicate the practical relevance of this kind of analysis for improving the quality of virtual research environments. In the last section, directions for further research are suggested. [ABSTRACT FROM AUTHOR]

By combining concepts from scientometrics and organisation studies, we hypothesise a basic 'life cycle' of organisational research units (institutes, laboratories or groups), if internal and external conditions are stable. Three output indicators enable a comparison of historical patterns with the lifecycle pattern, to reveal basic dynamics and changing conditions. We tested the model for a specific case: the Centre for Science and Technology Studies (CWTS) at Leiden University. The 'standard' life cycle was found from the start of CWTS in the early 1980s until the beginning of this century. Then, a boost of activities indicates the start of a second life cycle, explained by increased demand for performance studies and increased ability of CWTS to deliver standardised products. Recent changes in funding and key membership are expected to start a third cycle with reorientation of CWTS's activity profile. [ABSTRACT FROM AUTHOR]

Does past performance influence success in grant applications? We tested whether the decisions of the Netherlands Research Council for the Economic and Social Sciences correlate with the past performances of applicants in publications and citations, and with the results of the Council's peer reviews. The Council proves successful in distinguishing grant applicants with above-average from below-average performance, but within the former group there was no correlation between past performance and receiving a grant. When comparing the best-performing researchers who were denied funding with those who received it, the rejected researchers significantly outperformed the funded ones. The best rejected proposals score on average as high on the outcomes of the peer-review process as the accepted proposals. The Council successfully corrected for gender effects during the selection process. We explain why these findings may apply beyond this case. However, if research councils are not able to select the 'best' researchers, perhaps they should reconsider their mission. We discuss the role of research councils in the science system in terms of variation, innovation and quality control. [ABSTRACT FROM AUTHOR]

Despite its relevance for research funding, few comparable data are available in official R&D statistics on the amount and composition of project funding. This paper discusses in detail the methodology developed in the European Network of Excellence on Research and Innovation Policies PRIME for systematically producing indicators on public project funding which allow for comparative analysis between different countries and across periods of time. We introduce the design of the methodology, and discuss delimitation problems and how to develop suitable classifications of project funding instruments, as well as data availability and limitations. We present examples of our quantitative results for six European countries and of the questions they raise for comparative policy analysis. [ABSTRACT FROM AUTHOR]

This article presents a comparative analysis of the evolution of national research policies during the past three decades in six European countries (Austria, Italy, France, Netherlands, Norway and Switzerland), with a special focus on the changes of public project funding schemes. It systematically uses indicators on the volume of funding attributed by each instrument and agency, which have been developed in a project of the European network of excellence PRIME. A common model is identified in these countries, where project funding is the second main channel of public funding of research, but also there are considerable variations among them in the share of instruments and agencies, and in beneficiaries. There are three interesting commonalities: a strong increase of project funding volumes; a differentiation of instruments; and a general shift towards instruments oriented to thematic priorities. They also show that individual countries appear to follow quite distinct paths in the organisation setting of funding agencies, and that national differences in funding portfolios persist through time. [ABSTRACT FROM AUTHOR]

A model is described to explain the effect of e-voting technology on voting behavior and the outcome of a ballot. In the model, social identity and trust play a central role. After an initial and partial test, the article discusses directions for further research. [ABSTRACT FROM AUTHOR]

In many places e-voting technologies are under development, and an intensive theoretical and normative debate is taking place about the pros and cons. We investigate the opinions of the users of this type of technologies, as this is crucial for the shaping and acceptance of the technology. We did not use a survey, but held 14 intensive discussion sessions in different countries with voters and organisers of ballots, using the focus groups methodology. We found consensus that e-voting will not influence turnout. The motivation to start with e-voting therefore seems mainly financial, aiming at reducing costs. This suggests that in the future e-voting will replace traditional ways of voting, and therefore the digital divide can be expected to influence the participation in and the outcome of ballots. Finally, although most respondents expect that e-voting may improve (especially local) democracy through a combination of voting technologies with technologies for supporting deliberation and information dissemination, it remains unclear how this should be done. More detailed studies into political participation and the subtle roles of ICT's herein are needed, as this can inform the design of adequate technologies for e-democracy. [ABSTRACT FROM AUTHOR]

In a recent paper in this journal, Loet Leydesdorff and Gaston Heimeriks (2001, Journal of the American Society for Information Science and Technology, 52, 12621294.) argue that biotechnology develops in a self-organizational mode, through interaction between the intellectual structure and the institutional network of the research field. This claim is empirically supported by a multivariate analysis of documents from core biotechnology journals. One unexpected finding in this paper is the relationship between the title words of documents and the region of their origin. This claim requires examination because, as will be shown, it seems to be an artifact of the method used. if this is so, it undermines the authors' theoretical claim that the production of knowledge is a self-organizing process. [ABSTRACT FROM AUTHOR]

Schumpeter's (1939) distinction between changes in the form of the production function corresponding to innovation, and shifts along the production function corresponding to factor substitution, does not preclude that the underlying dynamics interact. In an evolutionarily complex system, such interactions are expected: they lead to non-linear terms in the model, and therefore to stabilization and self-organization in addition to selection and variation. Relatively simple simulations enable us to specify various concepts used in "evolutionary economics" in terms of non-linear dynamics. While a technological trajectory can be considered as a stabilization in a (distributed) environment, a technological regime can be defined as a next-higher-order globalization in a hyper-space. A regime is able to restore its order despite local disturbances, for example by the political system. Technology policies may be effective at the level of the (sub-)systems if they provide the relevant agents with room for "creative destruction" of the globalized hyper-systems. Implications for firm behavior and innovation policies are elaborated. Introduction [ABSTRACT FROM AUTHOR]

Analyzes the relationship between information technology and employment. How the rise of the information societies impacted on level of unemployment; Comparison of various levels of aggregation; Discussion of the effects of technological change.

Can trade unions act upon the development of science and technology (S&T) in our types of society? Some major political programmes aiming at the development of a labour-oriented technology are analysed and assessed in terms of their impact on S&T. The results of a survey of grass-roots initiatives ('workers' plans') and two questionnaires among experts suggest that no progress has yet been made in the attempts to develop an interface with S&T. This conclusion leads to some reflection on the conditions of a policy which would allow the workforce to take part in the processes of technological innovation. [ABSTRACT FROM AUTHOR]

Discusses the lack of consensus among specialists, about the nature of Artificial Intelligence (AI) as an emerging specialty. Use of aggregated journal-journal citations for describing AI as sets of journals; Use of factor analytic techniques to analyze the development of AI in terms of stability and coherency of the journal-sets; Identification of specialties, such as pattern analysis, and computer science, which are related to AI.

A letter to the editor is presented in response to the article related to predictive validity of research funding decisions with a multiple imputation for missing data approach by R. Mutz and colleagues published is a previous issue of the periodical.

Comments on an article about descriptive statistics, inferential statistics and rhetorical statistics by Loet Leydesdorff. Data of Leydesdorff which are attributes of the articles in a selected set of biotechnology journals; Illustration of descriptive statistics; Attempt at the development of a methodology for second-order theorizing.

In this article, we study the evaluation of talented early career researchers, as done in grant allocation processes. To better understand funding decisions, we studied the grant allocation process in more detail, and compared the notion of talent in grant allocation with more general notions of talent existing in the academic work environment. The comparison is based on interviews with 29 scholars who have experience with identifying talent both in their daily academic work and in the process of grant allocation. Overall, there is large agreement on the notion of talent. However, the characteristics ascribed to top talent vary depending on the evaluation context. In grant allocation, a narrower talent definition prevails compared with more general evaluation. Furthermore, difficulties arise in the process of panel decision-making, when selection criteria need to be concrete and explicit to enable comparison. Having to choose between many applicants of similar quality makes the selection process liable to subjectivity, arbitrariness, and randomness. Despite these uncertainties, grants are ascribed a very high symbolic value. Small quality differences are enlarged into considerable differences in recognition, consequently affecting career opportunities, as they provide academics with both financial and symbolic resources. [ABSTRACT FROM PUBLISHER]

Talent selection within science is increasingly performed by panels, e.g. by reviewing grant or fellowship applications. Many studies from fields of sociology of science and science policy studies have been conducted to identify biases and predict outcomes of these processes, mainly focusing on characteristics of applicants, applications, and reviewers. However, as panel reviewing entails social interaction, group dynamics influence these processes. By adding insights from social psychology to current knowledge on panel reviews, we are better able to identify factors affecting talent selection and funding decisions in grant panels. By opening up this so-called black box, we aim to contribute to a better understanding of the dynamics of panel decision making. This knowledge is also relevant for various stakeholders involved in grant allocation, for applicants, reviewers, and policymakers, as it can be used to improve transparency, fairness, and legitimation of talent selection processes. [ABSTRACT FROM PUBLISHER]

This paper answers five questions about the societal impact of research. Firstly, we examine the opinions of research group leaders about the increased emphasis on societal impact, i.e. does it influence their research agenda, communication with stakeholders, and knowledge dissemination to stakeholders? Furthermore, we investigate the quality of their societal output. We also study whether the societal and scholarly productivity of academic groups are positively or negatively related. In addition, we investigate which managerial and organisational factors (e.g. experience of the principal investigator, group size and funding) influence societal output. Finally, we show for one case (virology) that societal impact is also visible through indirect links. Our study shows that research group leaders have a slightly positive attitude towards the increased emphasis on the societal impact of research. The study also indicates a wide variety of societal-oriented output. Furthermore, the societal and scientific productivity of academic groups are unrelated, suggesting that stimulating social relevance requires specific organisational and contextual interventions. [ABSTRACT FROM AUTHOR]

The idea of constructing science maps based on bibliographic data has intrigued researchers for decades, and various techniques have been developed to map the structure of research disciplines. Most science mapping studies use a single method. However, as research fields have various properties, a valid map of a field should actually be composed of a set of maps derived from a series of investigations using different methods. That leads to the question of what can be learned from a combination-triangulation-of these different science maps. In this paper we propose a method for triangulation, using the example of water science. We combine three different mapping approaches: journal-journal citation relations (JJCR), shared author keywords (SAK), and title word-cited reference co-occurrence (TWRC). Our results demonstrate that triangulation of JJCR, SAK, and TWRC produces a more comprehensive picture than each method applied individually. The outcomes from the three different approaches can be associated with each other and systematically interpreted to provide insights into the complex multidisciplinary structure of the field of water research. [ABSTRACT FROM AUTHOR]

Universities are increasingly expected to fulfill a third mission in addition to those of research and education. Universities must demonstrate engagement with society through the application and exploitation of knowledge. As societal impact of research is uncertain, long term and always dependent on other factors, we argue here that evaluation should focus on the conditions under which societal impact is generated rather than on the impact itself. Here we focus on a specific set of those conditions: the interactions between academic researchers and societal actors. Instead of speculating about potential impacts of research, we argue that current productive interactions of researchers with societal stakeholders improve the probability that future societal impact will occur. This article supports this idea by examining in detail several, mainly retrospective examples. As productive interactions are field specific, we restrict ourselves to ‘professional adhocracy fields’, especially to information and communication technologies (ICT) research. We address the patterns of productive interactions that result in societal impact within this field and we discuss whether differences are observed in contrast to other fields, such as social sciences and humanities (fragmented adhocracies). We end by discussing the implications that these patterns have for societal impact assessment. Shifting the focus to interactions allows assessment of short-term knowledge transfer and other collaborative efforts with stakeholders that contribute to long-term societal impact. [ABSTRACT FROM PUBLISHER]

The article investigates the participatory co-development of suitable indicators for the assessment of internationalization of funding agencies (FAs). Our research questions are: How can we measure different functions and institutional characteristics using a participatory process? What and who do we measure (and assess), against what do we assess? Which of the stakeholders should/could use the indicators (ministerial principals to assess the agencies, agency leadership, agency staff, external evaluators) and for what purposes? In this way we intend to contribute to the literature on social construction of S&T indicators, and to explore the representation of FAs' functions and activities for evaluation purposes. The results show how the process was designed and evolved, its strengths and weaknesses, and the effects generated as to the conceptualization for the indicators production. The work confirms the importance of indicators assessment forums for designing indicators, and how these arenas allow constituting a common language, enable the actors to discuss their representations, and favour collective learning. [ABSTRACT FROM AUTHOR]