Search

The Openness Position Paper published by the Arqus European University Alliance emphasises that Arqus institutions, in line with the policies, roadmaps and strategies of the EU and a wide range of stakeholders, are striving jointly to make further progress towards realising Open Science. The Position Paper identifies and acknowledges aims and values of Open Science and relates them to values, principles, and standards shared by the Arqus Alliance, followed by a vision for a future with Open Science. In the interest of a nuanced picture, the Position Paper discusses not only desired effects, but also possible areas of tension related to Open Science. It presents a wide range of specific aims and recommendations for each of the eleven elements of Open Science defined by the Arqus Openness Task Force: Governance Publications (including Open Access) Data (including research data management, FAIR and Open Data) Infrastructures (including support staff, Open Science software and tools, repositories, Open Labs) Methods (including source code, preregistration, materials, workflows, protocols, lab notes) Awareness and training (including education of early-stage researchers) Evaluation (including Open Metrics, research assessment, Open Peer Review, rewards and incentives) Communication (including multilingualism) Citizen Science Open Education Open Innovation The Position Paper concludes with an annex that highlights the progress already made in the implementation and support of Open Science practices at Arqus institutions., Cofunded by the Erasmus+Programme of the European Union

Since the 17th century, learned societies - voluntary non-profit organizations involving academics - have existed for the advancement of scholarship, research, disciplines, publishing and public understanding of science. Publishing scientific journals and books has traditionally been an important part of the activities of learned societies but they also take other various activities including arranging conferences, supporting research, and popularizing knowledge. The most well-known learned societies are the national science academies, in which membership is typically based on invitation and merit. However, there also exists a much broader archipelago of local, national and international societies, whose membership is open to all academics, and often also to interested professionals and citizens. In English-language literature, a learned society can also be referred to as a learned/scholarly/scientific association or scholarly society. Occasionally, the term professional society is also used. We have estimated in a working-paper that there are around 9000 learned societies in Europe (https://doi.org/10.5281/zenodo.5513560). In historical perspective, learned societies constitute, along with universities and other kinds of research performing organizations, the foundation of contemporary academia. International Survey on Research Integrity (IRIS: https://doi.org/10.17605/OSF.IO/XB9RK) provides the broadest available indication of the continued relevance of learned societies. Over 82% of 60,885 active researchers from Europe, United States and other countries, who in 2021 answered the IRIS survey question concerning professional societies, identified at least “a little bit” with the societies, and 62% identified “moderately”, “a lot” or “a great deal”. Yet, surprisingly little information and research exists on their current number, forms of organizations and operation, or contributions to scientific and societal impact of research in specific countries, regions or globally. While more attention has been paid to the obstacles and challenges for promoting open access in the landscape of learned society journals, next to nothing is known or said about the (potential) role of learned societies in the area of responsible research, including research integrity and research assessment. However, we argue that these societies create networks of scholars and professionals and have discipline specific expertise on scholarly activities that should be exploited more. In this paper we address the following questions: 1. What role does open science and responsible research play in societies’ activities? 2. Do the members expect open science and responsible research to be part of the societies’ activities? To answer these questions, we would need a global review of learned societies, their activities and membership. The scope of our present study is mainly focused on Finland, from which we have a comprehensive report in 2019 (https://doi.org/10.23847/isbn.9789525995190). Besides Finland, one survey study has covered learned societies in Portugal, and another one social science societies in the UK. Methods and data We provide data from Finland and from Finnish societies. Currently, almost 300 societies are members of the Federation of Finnish Learned Societies (TSV), and they have more than 250,000 individual members (Finland has a population of 5,5 million). Recently TSV collected two broad datasets from its member societies and the individual members of these societies.The first dataset (N=114 unique societies) concerns the actions related with responsible research of learned societies. The second dataset (N=1540 individual members of learned societies) concern the membership of the societies. By integrating these two datasets we are able to provide insight about the open science and responsible research activities from the viewpoint of the societies and their membership. Results Over 94% of 114 learned societies in Finland, who answered the survey in 2021, indicated that the promotion of scientific activities is a major part of their activities, followed by the promotion of general understanding of science and societal impact. Generally speaking, the promotion of responsible research and open science has not yet become a key part of learned societies’ operations. Yet around 70 per cent of the societies participate in developing open science, 60 per cent publish immediately openly available publications. However, activities linked to research integrity, open data, open education and citizen science are rarer. Nevertheless, the majority of societies were interested in developing their activities in one or more areas of responsible research: identifying and proposing experts to working groups, committees or as evaluators (65 % of societies), promoting open science and organizing events (more than 50 %), evaluating the quality of research (more than 40 %), assessment of social impact, science education and research ethics (approx. 30 %). The membership of the learned societies regards traditional forms of operation, such as networking, publications, events and the popularization of science as the most important parts of the societies’ operations. Over 90 per cent of the respondents agreed at least partially with the statement that the promotion of the openness of research-based knowledge is important, and more than 80 per cent agreed that this should be free of charge to the readers. Almost 90 percent of the respondents agreed at least partially with the statement that the promotion of research integrity should be an important task of societies, and almost 80% agreed also on the importance of responsible assessment of researchers and research quality. Almost 60% agreed on the promotion of citizen science, while 45% agreed that producing open learning materials is an important task of societies. Significance Our case study in Finland shows that societies are active in the most traditional forms of open science, such as open access publishing. Publication activities are indeed one of the learned societies’ key forms of operation, and a significant proportion of learned societies’ publications already complies with the principles of open science. The societies have been important enablers and promoters of open, non-profit and scientific publication activities without author fees (so called Diamond OA) in Finland. Other forms of open science and responsible research are not yet established but highly supported by the members of the societies. The perceptions from the membership should guide the future development of the societies. Data on society memberships shows that learned societies represent the science community on a broad scale in terms of age, field of education, work organization and place of residence. The networks created by the societies are cross-organizational and often international and provide a unique resource for identifying experts of different domains and creating collaboration between the domains. Thus, societies have a great potential in implementing and developing field-specific open science practices through engagement of researchers at the grass-root level towards a more responsible direction. Policy makers, research performing and funding organizations could develop more strategic collaboration with learned societies as a valued intermediary between science and society.

A discussion of related work is part of every research paper and grant proposal submission, and as such is evaluated in the reviewing process. If a reviewer is an expert in all covered areas, a qualified judgement of the section can be expected. However, in many cases, especially when the discussion touches on different research areas and application domains, most reviewers will only have a partial understanding of the related research. Hence, they face a difficult challenge in assessing the quality and completeness of the discussion of related work. Starting an own full-fledged literature search with traditional academic search tools poses an unrealistically high effort for the reviewers. Facilitated by an increased availability of open citation data, a new group of tools has recently been published that support users in tracking the citation links of multiple researcher papers. For instance, “Citation Gecko” and “PURE suggest” visualize the surrounding citation network and suggest related papers that are linked through citations to the selected ones. Other tools, like “ResearchRabbit” and “Inciteful”, embed similar features with a broader range of analysis and recommendation options. Loading the list of references of a paper under review, these tools hence allow inspecting the citation network and suggest related, non-listed publications. This can be the starting point for an efficient check of the character and completeness of a related work discussion. In this talk, I specifically explore the capabilities of our tool “PURE suggest” (https://fabian-beck.github.io/pure-suggest/) for assessing a list of references. The top-ranked suggestions can be quickly checked for relevancy. If the scores of these suggestions are comparable to the scores of already included papers, this might already hint at a gapped discussion. Boost keywords help the reviewer to investigate specific directions more closely. Filtering by year allows checking whether the newest publications have already been considered. Similarly, filtering for underrepresented works might point out relevant, but somewhat unknown works that have been overlooked. The cluster-oriented visualization of the citation network reveals whether expected clusters of works show in the network, are missing, or disconnected. This demonstrates that these tools, although targeted primarily on literature search, can be also leveraged for assessing lists of related work in the sense of a sanity check. It does not replace the expertise of the reviewers but complements it with a data-driven approach. The reviewers work with the tool and use it as an aid to reflect on various aspects of the related work. Of course, a similar approach can be used by the authors of the submission already to prevent gaps in their discussion of related work. Although it might sound like a threat to the suggested assessment, on the contrary, it would be a desirable effect as it will, already in the first place, likely improve the quality of the submitted work.

Slides used for the Recognition & Rewards Festival 2023, Workshop 1.8 - Recognising supporting roles on the 13th of April 2023. Session contributors listed in alphabetical order. Workshop abstract: The position paper ‘Room for everyone's talent’ focusses, amongst other things, on diversifying career paths and how to recognise team performances. This should be a key opportunity to involve staff at research institutions that have more specialised or supporting tasks in these teams. This workshop aims to increase awareness regarding supporting roles, what role they play in Team Science and how to recognise their expertise. During this interactive workshop we will exchange practices and start with setting up a survey and roadmap to ensure that supporting roles are involved in current and future Recognition and Rewards efforts. Please note that animations were used in the slides, which are properly displayed in the .pptx format.

Slides used for the Recognition & Rewards Festival 2023, Workshop 1.8 - Recognising supporting roles on the 13th of April 2023. Session contributors listed in alphabetical order. Workshop abstract: The position paper ‘Room for everyone's talent’ focusses, amongst other things, on diversifying career paths and how to recognise team performances. This should be a key opportunity to involve staff at research institutions that have more specialised or supporting tasks in these teams. This workshop aims to increase awareness regarding supporting roles, what role they play in Team Science and how to recognise their expertise. During this interactive workshop we will exchange practices and start with setting up a survey and roadmap to ensure that supporting roles are involved in current and future Recognition and Rewards efforts. Please note that animations were used in the slides, which are properly displayed in the .pptx format.

Little attention has been devoted to whether the Impact Factor (IF) can be considered a responsible metric in light of bibliodiversity. This paper critically engages with this question in measuring the following variables of IF journals included in the 2021 Journal CItation Reports and examining their distribution: publishing models (hybrid, Open Access with or without fees, subscription), world regions, language(s) of publication, subject categories, publishers, and the prices of article processing charges (APC) if any. Our results show that the quest for prestige or perceived quality through the IF brand poses serious threats to bibliodiversity. The IF brand can indeed hardly be considered a responsible metric insofar as it perpetuates publishing concentration, maintains a domination of the Global North and its attendant artificial image of mega producer of scholarly content, does not promote linguistic diversity, and de-incentivizes fair and equitable open access by entrenching fee-based OA delivery options with rather high APCs. Dataset The accompanying data to this study can be found here: 10.5281/zenodo.7683743

Data-intensive science needs indicators to understand and assess scientific collaboration. This paper reports the use of a novel data source to develop indicators for an invisible layer of scientific collaborations: the metadata about datasets from large open research data repositories. Using social network analysis, we analyze how dataset collaborations contribute to network connectivity, an indicator of knowledge diffusion. Findings show that dataset collaborations significantly increase the network connectivity properties, and that the overlap of the publication and submission networks (1992-2021) increases over time. We find low assortativity in the dataset network yet high assortativity in the publication network, revealing the distinct structural role that dataset authors play. Findings indicate dataset producers play a middle-management role, which has implications for knowledge diffusion, by performing a bridging function between the data production and publication networks. We argue science of science research should use dataset collaboration networks to develop novel indicators of network connectivity because datasets provide critical insights into the core functions of science – knowledge diffusion, mentorship networks, and the division of labor in data-intensive scientific collaboration.

Research assessmentreform is an urgent priority of European research policy. While initially closely tied to the advancement of Open Science practices, this objective has grown into a larger movement that addresses many different shortcomings of dominant evaluation practices.European University Alliances are directly concerned by this policy objective and are seen as possible testbeds for reform. EUTOPIA has also committed to implement a framework policy for research assessment and Open Science through the EUTOPIA-TRAIN project. With this paper, we aim to present practical methods and indicators addressing Open Science that are being discussed and piloted at research performing organisations across Europe. We hope to stimulate the institutional transformation of our own alliance and otter universities through the provision of the practical ideas covered in this report. The report builds on an earlier background noteof EUTOPIA in which we have explored the political momentum in the area of research assessment and its implications for European Universities. This report includes sections covering an overview of initiatives and declarations which seek to reform research assessment, including DORA, the Leiden Manifesto and others. Our summary focuses on the role of quantitative and qualitative assessment in these initiatives and declarations. quantitative indicators and metrics that could be used to measure Open Science activities and outputs. Our overview is based on existing source documents from various national and international frameworks. We also provide a summary of the commonalities of these frameworks- as well as challenges when implementing quantitative approaches. qualitative methods for assessment. Our overview is largely based on the emerging experience with narrative CVs, but other measures taken, for instance contextualising achievements and outputs, as well as contravening biases in qualitative assessment are addressed, too. the underlying infrastructure and (open) information systems for assessment that have been discussed and proposed elsewhere to provide an comprehensive, up-to-date, and accessible evidence base for assessments. We also summarise briefly the obstacles one encounters when assessing the FAIRness of research data.

Academic publishing represents one of the core functions of scientific activity and thus of the research system. The Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) is committed to the adequate communication of research results and the maintenance of a functioning publication system as a basic requirement of the research system. Ensuring that academic publishing serves foremost the purposes of publicising, quality review and documenting scientific findings should be an essential goal of responsible science administration and promotion. The position paper therefore analyses the central background of the current situation in scientific publishing, identifies challenges and points out possible fields of action. With this positioning, the DFG contributes to the further development of the publication system in a way that is appropriate for science and research., This publication is available in German and English.

In recent years, there has been increased attention given to how research assessment is conducted at different levels – research proposals, individual researchers, research organizations. In this context, the current paper explores existing literature regarding current research assessment approaches, and especially recommendations provided through four reference documents on the topic: the San Francisco Declaration on Research Assessment (2013), the Leiden Manifesto (Hickset al.2015), Science Europe’s Position Statement and Recommendations on Research Assessment Processes (2020), and the Hong Kong Principles for assessing researchers (Moheret al.2020). The aim is to provide a basis for analysing the legislative and normative framework regulating how researchers are being evaluated in Romania for hiring and career development, and then apply it to identify recommendations regarding how the latter can be improved. Some of the recommendations identified refer to aspects such as: the need to explore evaluation arrangements which incorporate research activity related practices, and that measure performance against research units’ research goals; increased focus on qualitative approaches, and on scientific content, as opposed to performance in publication metrics; developing a broader list with indicators, including Open Science and societal relevance related; introducing the principle of transparency; developing a monitoring and evaluation framework.