Search

Recent developments in causal inference allow us to transport a causal effect of a time-fixed treatment from a randomized trial to a target population across space but within the same time frame. In contrast to transportability across space, transporting causal effects across time or forecasting causal effects of future interventions is more challenging due to time-varying confounders and time-varying effect modifiers. In this article, we seek to formally clarify the causal estimands for forecasting causal effects over time and the structural assumptions required to identify these estimands. Specifically, we develop a set of novel nonparametric identification formulas--g-computation formulas--for these causal estimands, and lay out the conditions required to accurately forecast causal effects from a past observed sample to a future population in a future time window. Our overaching objective is to leverage the modern causal inference theory to provide a theoretical framework for investigating whether the effects seen in a past sample would carry over to a new future population. Throughout the article, a working example addressing the effect of public policies or social events on COVID-related deaths is considered to contextualize the developments of analytical results.

This paper describes the 2nd edition of the ICML Topological Deep Learning Challenge that was hosted within the ICML 2024 ELLIS Workshop on Geometry-grounded Representation Learning and Generative Modeling (GRaM). The challenge focused on the problem of representing data in different discrete topological domains in order to bridge the gap between Topological Deep Learning (TDL) and other types of structured datasets (e.g. point clouds, graphs). Specifically, participants were asked to design and implement topological liftings, i.e. mappings between different data structures and topological domains --like hypergraphs, or simplicial/cell/combinatorial complexes. The challenge received 52 submissions satisfying all the requirements. This paper introduces the main scope of the challenge, and summarizes the main results and findings., Comment: Proceedings of the Geometry-grounded Representation Learning and Generative Modeling Workshop (GRaM) at ICML 2024

Could the generalized adoption of the COARA principles make administrative research evaluation socially desirable and solve all its problems? The answer to this question is no. To reach this conclusion, two main arguments are discussed. The first characterises COARA as a form of 'technocracy' perfectly consistent with a neo-liberal view of research. The second consists in the adoption of Philip Kitcher's idea of well-ordered science. It is argued that administrative evaluation of research, even if correct on the basis of COARA principles, is at odds with the principles of well-ordered science since it cannot escape neither the tyranny of expertise nor the tyranny of ignorance. These two arguments allow to suggest limiting administrative evaluation to the bare minimum (recruitment of researchers and funding of projects), and to focus attention mainly to the fairness of evaluation procedures., Comment: 11 pages

In research evaluation, the fair representation of panels is usually defined in terms of observable characteristics of scholars such as gender or affiliations. An empirical strategy is proposed for exploring hidden connections between panellists such that, despite the respect of formal requirements, the panel could be considered alike as unfair with respect to the representation of diversity of research approaches and methodologies. The case study regards the three panels selected to evaluate research in economics, statistics and business during the Italian research assessment exercises. The first two panels were appointed directly by the governmental agency responsible for the evaluation, while the third was randomly selected. Hence the third panel can be considered as a control for evaluating about the fairness of the others. The fair representation is explored by comparing the networks of panellists based on their co-authorship relations, the networks based on journals in which they published and the networks based on their affiliated institutions (universities, research centres and newspapers). The results show that the members of the first two panels had connections much higher than the members of the control group. Hence the composition of the first two panels should be considered as unfair, as the results of the research assessments., Comment: 40 pages, 6 figures

Fake news detection models are critical to countering disinformation but can be manipulated through adversarial attacks. In this position paper, we analyze how an attacker can compromise the performance of an online learning detector on specific news content without being able to manipulate the original target news. In some contexts, such as social networks, where the attacker cannot exert complete control over all the information, this scenario can indeed be quite plausible. Therefore, we show how an attacker could potentially introduce poisoning data into the training data to manipulate the behavior of an online learning method. Our initial findings reveal varying susceptibility of logistic regression models based on complexity and attack type.

The synthesis of information deriving from complex networks is a topic receiving increasing relevance in ecology and environmental sciences. In particular, the aggregation of multilayer networks, i.e. network structures formed by multiple interacting networks (the layers), constitutes a fast-growing field. In several environmental applications, the layers of a multilayer network are modelled as a collection of similarity matrices describing how similar pairs of biological entities are, based on different types of features (e.g. biological traits). The present paper first discusses two main techniques for combining the multi-layered information into a single network (the so-called monoplex), i.e. Similarity Network Fusion (SNF) and Similarity Matrix Average (SMA). Then, the effectiveness of the two methods is tested on a real-world dataset of the relative abundance of microbial species in the ecosystems of nine glaciers (four glaciers in the Alps and five in the Andes). A preliminary clustering analysis on the monoplexes obtained with different methods shows the emergence of a tightly connected community formed by species that are typical of cryoconite holes worldwide. Moreover, the weights assigned to different layers by the SMA algorithm suggest that two large South American glaciers (Exploradores and Perito Moreno) are structurally different from the smaller glaciers in both Europe and South America. Overall, these results highlight the importance of integration methods in the discovery of the underlying organizational structure of biological entities in multilayer ecological networks., Comment: 19 pages, 4 figures

In order to explore the suitability of a fine-grained classification of journal articles by exploiting multiple sources of information, articles are organized in a two-layer multiplex. The first layer conveys similarities based on the full-text of articles, and the second similarities based on cited references. The information of the two layers are only weakly associated. The Similarity Network Fusion process is adopted to combine the two layers into a new single-layer network. A clustering algorithm is applied to the fused network and the classification of articles is obtained. In order to evaluate its coherence, this classification is compared with the ones obtained by applying the same algorithm to each of two layers. Moreover, the classification obtained for the fused network is also compared with the classifications obtained when the layers of information are integrated using different methods available in literature. In the case of the Cambridge Journal of Economics, Similarity Network Fusion appears to be the best option. Moreover, the achieved classification appears to be fine-grained enough to represent the extreme heterogeneity characterizing the contributions published in the journal., Comment: 35 pages, 4 figures, 10 tables

Numerous logics have been developed to reason either about threshold-induced opinion diffusion in a network, or about similarity-driven network structure evolution, or about both. In this paper, we first introduce a logic containing different dynamic operators to capture changes that are 'asynchronous' (opinion change only, network-link change only) and changes that are 'synchronous' (both at the same time). Second, we show that synchronous operators cannot, in general, be replaced by asynchronous operators and vice versa. Third, we characterise the class of models on which the synchronous operator can be reduced to sequences of asynchronous operators., Comment: In Proceedings TARK 2023, arXiv:2307.04005

This study investigates the role of editorial board members as gatekeepers in science, creating and utilizing a database of 1,516 active economics journals in 2019, which includes more than 44,000 scholars from over 6,000 institutions and 142 countries. The composition of these editorial boards is explored in terms of geographic affiliation, institutional affiliation, and gender. Results highlight that the academic publishing environment is primarily governed by men affiliated with elite universities in the United States. The study further explores social similarities among journals using a network analysis perspective based on interlocking editorship. Comparison of networks generated by all scholars, editorial leaders, and non-editorial leaders reveals significant structural similarities and associations among clusters of journals. These results indicate that links between pairs of journals tend to be redundant, and this can be interpreted in terms of social and intellectual homophily within each board, and between boards of journals belonging to the same cluster. Finally, the analysis of the most central journals and scholars in the networks suggests that journals probably adopt 'strategic decisions' in the selection of the editorial board members. The documented high concentration of editorial power poses a serious risk to innovative research in economics., Comment: 23 pages, 17 tables, 6 figures

Self-citations are a key topic in evaluative bibliometrics because they can artificially inflate citation-related performance indicators. Recently, self-citations defined at the largest scale, i.e., country self-citations, have started to attract the attention of researchers and policymakers. According to a recent research, in fact, the anomalous trends in the country self-citation rates of some countries, such as Italy, have been induced by the distorting effect of citation metrics-centered science policies. In the present study, we investigate the trends of country self-citations in 50 countries over the world in the period 1996-2019 using Scopus data. Results show that for most countries country self-citations have decreased over time. 12 countries (Colombia, Egypt, Indonesia, Iran, Italy, Malaysia, Pakistan, Romania, Russian Federation, Saudi Arabia, Thailand, and Ukraine), however, exhibit different behavior, with anomalous trends of self-citations. We argue that these anomalies should be attributed to the aggressive science policies adopted by these countries in recent years, which are all characterized by direct or indirect incentives for citations. Our analysis confirms that when bibliometric indicators are integrated into systems of incentives, they are capable of affecting rapidly and visibly the citation behavior of entire countries., Comment: 44 pages, 15 figures

We introduce a methodology based on averaging similarity matrices with the aim of integrating the layers of a multiplex network into a single monoplex network. Multiplex networks are adopted for modelling a wide variety of real-world frameworks, such as multi-type relations in social, economic and biological structures. More specifically, multiplex networks are used when relations of different nature (layers) arise between a set of elements from a given population (nodes). A possible approach for investigating multiplex networks consists in aggregating the different layers in a single network (monoplex) which is a valid representation -- in some sense -- of all the layers. In order to obtain such an aggregated network, we propose a theoretical approach -- along with its practical implementation -- which stems on the concept of similarity matrix average. This methodology is finally applied to a multiplex similarity network of statistical journals, where the three considered layers express the similarity of the journals based on co-citations, common authors and common editors, respectively., Comment: 23 pages, 8 figures

Hypergraphs and simplical complexes both capture the higher-order interactions of complex systems, ranging from higher-order collaboration networks to brain networks. One open problem in the field is what should drive the choice of the adopted mathematical framework to describe higher-order networks starting from data of higher-order interactions. Unweighted simplicial complexes typically involve a loss of information of the data, though having the benefit to capture the higher-order topology of the data. In this work we show that weighted simplicial complexes allow to circumvent all the limitations of unweighted simplicial complexes to represent higher-order interactions. In particular, weighted simplicial complexes can represent higher-order networks without loss of information, allowing at the same time to capture the weighted topology of the data. The higher-order topology is probed by studying the spectral properties of suitably defined weighted Hodge Laplacians displaying a normalized spectrum. The higher-order spectrum of (weighted) normalized Hodge Laplacians is here studied combining cohomology theory with information theory. In the proposed framework, we quantify and compare the information content of higher-order spectra of different dimension using higher-order spectral entropies and spectral relative entropies. The proposed methodology is tested on real higher-order collaboration networks and on the weighted version of the simplicial complex model "Network Geometry with Flavor"., Comment: 20 pages, 11 figures

This paper explores intellectual and social proximity among scholarly journals by using network fusion techniques. Similarities among journals are initially represented by means of a three-layer network based on co-citations, common authors and common editors. The information contained in the three layers is then combined by building a fused similarity network. The fusion consists in an unsupervised process that exploits the structural properties of the layers. Subsequently, partial distance correlations are adopted for measuring the contribution of each layer to the structure of the fused network. Finally, the community morphology of the fused network is explored by using modularity. In the three fields considered (i.e. economics, information and library sciences and statistics) the major contribution to the structure of the fused network arises from editors. This result suggests that the role of editors as gatekeepers of journals is the most relevant in defining the boundaries of scholarly communities. In information and library sciences and statistics, the clusters of journals reflect sub-field specializations. In economics, clusters of journals appear to be better interpreted in terms of alternative methodological approaches. Thus, the graphs representing the clusters of journals in the fused network are powerful instruments for exploring research fields., Comment: 21 pages, 4 figures

Two alternative accounts can be given of the information contained in the acknowledgments of academic publications. According to the mainstream normative account, the acknowledgments serve to repay debts towards formal or informal collaborators. According to the strategic account, by contrast, the acknowledgments serve to increase the perceived quality of papers by associating the authors with influential scholars. The two accounts are assessed by analyzing the acknowledgments indexed in Web of Science of 1218 articles published in the "top-five journals" of economics for the years 2015-2019. The analysis is focused on six dimensions: (i) the style of acknowledging texts, (ii) the distribution of mentions, (iii) the identity of the most mentioned acknowledgees, (iv) the shares of highly and lowly mentioned acknowledgees, (v) the hierarchy of the acknowledgment network, and (vi) the correlation at a paper level between intellectual similarity, measured by common references, and social similarity, measured by common acknowledges. Results show that the normative and the strategic account should be considered as valid but partial explanations of acknowledging behavior. Hence, acknowledgments should be used with extreme caution for investigating collaboration practices and they should not be used to produce acknowledgments-based metrics of scholars for evaluative purposes., Comment: 46 pages, 5 figures, 5 tables

This paper explores, by using suitable quantitative techniques, to what extent the intellectual proximity among scholarly journals is also a proximity in terms of social communities gathered around the journals. Three fields are considered: statistics, economics and information and library sciences. Co-citation networks (CC) represent the intellectual proximity among journals. The academic communities around the journals are represented by considering the networks of journals generated by authors writing in more than one journal (interlocking authorship: IA), and the networks generated by scholars sitting in the editorial board of more than one journal (interlocking editorship: IE). For comparing the whole structure of the networks, the dissimilarity matrices are considered. The CC, IE and IA networks appear to be correlated for the three fields. The strongest correlations is between CC and IA for the three fields. Lower and similar correlations are obtained for CC and IE, and for IE and IA. The CC, IE and IA networks are then partitioned in communities. Information and library sciences is the field where communities are more easily detectable, while the most difficult field is economics. The degrees of association among the detected communities show that they are not independent. For all the fields, the strongest association is between CC and IA networks; the minimum level of association is between IE and CC. Overall, these results indicate that the intellectual proximity is also a proximity among authors and among editors of the journals. Thus, the three maps of editorial power, intellectual proximity and authors communities tell similar stories., Comment: 21 pages, 9 Figures. Accepted for publication in Quantitative Science Studies. Fixed the spelling of the name of one of the authors

It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. We attempt to shed light on this topic by investigating how Italian researchers reacted to the introduction in 2011 of national regulations in which key passages of professional careers are governed by bibliometric indicators. A new inwardness measure, able to gauge the degree of scientific self-referentiality of a country, is defined as the proportion of citations coming from the country itself compared to the total number of citations gathered by the country. Compared to the trends of the other G10 countries in the period 2000-2016, Italy's inwardness shows a net increase after the introduction of the new evaluation rules. Indeed, globally and also for a large majority of the research fields, Italy became the European country with the highest inwardness. Possible explanations are proposed and discussed, concluding that the observed trends are strongly suggestive of a generalized strategic use of citations, both in the form of author self-citations and of citation clubs. We argue that the Italian case offers crucial insights on the constitutive effects of evaluation systems. As such, it could become a paradigmatic case in the debate about the use of indicators in science-policy contexts.

This paper appraises the concordance between bibliometrics and peer review, by drawing evidence from the data of two experiments realized by the Italian governmental agency for research evaluation. The experiments were performed for validating the dual system of evaluation, consisting in the interchangeable use of bibliometyrics and peer review, adopted by the agency in the research assessment exercises. The two experiments were based on stratified random samples of journal articles. Each article was scored by bibliometrics and by peer review. The degree of concordance between the two evaluations is then computed. The correct setting of the experiments is defined by developing the design-based estimation of the Cohen's kappa coefficient and some testing procedures for assessing the homogeneity of missing proportions between strata. The results of both experiments show that for each research areas of hard sciences, engineering and life sciences, the degree of agreement between bibliometrics and peer review is -- at most -- weak at an individual article level. Thus, the outcome of the experiments does not validate the use of the dual system of evaluation in the Italian research assessments. More in general, the very weak concordance indicates that metrics should not replace peer review at the level of individual article. Hence, the use of the dual system of evaluation for reducing costs might introduce unknown biases in a research assessment exercise., Comment: 28 pages, 6 tables, 4 Figures. This version contains identical results and maths. It adds an extended literature review, a deeper discussion of findings, and 4 new figures illustrating results

Italy adopted a performance-based system for funding universities that is centered on the results of a national research assessment exercise, realized by a governmental agency (ANVUR). ANVUR evaluated papers by using 'a dual system of evaluation', that is by informed peer review or by bibliometrics. In view of validating that system, ANVUR performed an experiment for estimating the agreement between informed review and bibliometrics. Ancaiani et al. (2015) presents the main results of the experiment. Baccini and De Nicolao (2017) documented in a letter, among other critical issues, that the statistical analysis was not realized on a random sample of articles. A reply to the letter has been published by Research Evaluation (Benedetto et al. 2017). This note highlights that in the reply there are (1) errors in data, (2) problems with 'representativeness' of the sample, (3) unverifiable claims about weights used for calculating kappas, (4) undisclosed averaging procedures; (5) a statement about 'same protocol in all areas' contradicted by official reports. Last but not least: the data used by the authors continue to be undisclosed. A general warning concludes: many recently published papers use data originating from Italian research assessment exercise. These data are not accessible to the scientific community and consequently these papers are not reproducible. They can be hardly considered as containing sound evidence at least until authors or ANVUR disclose the data necessary for replication., Comment: 9 pages, 5 tables

The opportunity to assess short term impact of air pollution relies on the causal interpretation of the exposure-outcome association, but up to now few studies explicitly faced this issue within a causal inference framework. In this paper, we reformulated the problem of assessing the short term impact of air pollution on health using the potential outcome approach to causal inference. We focused on the impact of high daily levels of PM10 on mortality within two days from the exposure in the metropolitan area of Milan (Italy), during the period 2003-2006. After defining the number of attributable deaths in terms of difference between potential outcomes, we used the estimated propensity score to match each high exposure-day with a day with similar background characteristics but lower PM10 level. Then, we estimated the impact by comparing mortality between matched days. We found that during the study period daily exposures larger than 40 microgram per cubic meter were responsible of 1079 deaths (116; 2042). The impact was more evident among the elderly than in the younger classes of age. The propensity score matching turned out to be an appealing method to assess historical impacts in this field.

During the Italian research assessment exercise, the national agency ANVUR performed an experiment to assess agreement between grades attributed to journal articles by informed peer review (IR) and by bibliometrics. A sample of articles was evaluated by using both methods and agreement was analyzed by weighted Cohen's kappas. ANVUR presented results as indicating an overall 'good' or 'more than adequate' agreement. This paper re-examines the experiment results according to the available statistical guidelines for interpreting kappa values, by showing that the degree of agreement, always in the range 0.09-0.42 has to be interpreted, for all research fields, as unacceptable, poor or, in a few cases, as, at most, fair. The only notable exception, confirmed also by a statistical meta-analysis, was a moderate agreement for economics and statistics (Area 13) and its sub-fields. We show that the experiment protocol adopted in Area 13 was substantially modified with respect to all the other research fields, to the point that results for economics and statistics have to be considered as fatally flawed. The evidence of a poor agreement supports the conclusion that IR and bibliometrics do not produce similar results, and that the adoption of both methods in the Italian research assessment possibly introduced systematic and unknown biases in its final results. The conclusion reached by ANVUR must be reversed: the available evidence does not justify at all the joint use of IR and bibliometrics within the same research assessment exercise., Comment: in Scientometrics, 2016