Search

Motivated by recent work on the use of topological methods to study collections of rings between an integral domain and its quotient field, we examine spaces of subrings of a commutative ring, where these spaces are endowed with the Zariski or patch topologies. We introduce three notions to study such a space $X$: patch bundles, patch presheaves and patch algebras. When $X$ is compact and Hausdorff, patch bundles give a way to approximate $X$ with topologically more tractable spaces, namely Stone spaces. Patch presheaves encode the space $X$ into stalks of a presheaf of rings over a Boolean algebra, thus giving a more geometrical setting for studying $X$. To both objects, a patch bundle and a patch presheaf, we associate what we call a patch algebra, a commutative ring that efficiently realizes the rings in $X$ as factor rings, or even localizations, and whose structure reflects various properties of the rings in $X$.

Co-based honeycomb magnets have been actively studied recently for the potential realization of emergent quantum magnetism therein such as the Kitaev spin liquid. Here we employ density functional and dynamical mean-field theory methods to examine a family of the Kitaev magnet candidates BaCo$_2$($X$O$_4$)$_2$ ($X$ = P, As, Sb), where the compound with $X$ = Sb being not synthesized yet. Our study confirms the formation of Mott insulating phase and the $J_{\rm eff}$ = 1/2 spin moments at Co$^{2+}$ sites despite the presence of a sizable amount of trigonal crystal field in all three compounds. The pnictogen substitution from phosphorus to antimony significantly changes the in-plane lattice parameters and direct overlap integral between the neighboring Co ions, leading to the suppression of the Heisenberg interaction. More interestingly, the marginal antiferromagnetic nearest-neighbor Kitaev term changes sign into a ferromagnetic one and becomes sizable at the $X$ = Sb limit. Our study suggests that the pnictogen substitution can be a viable route to continuously tune magnetic exchange interactions and to promote magnetic frustration for the realization of potential spin liquid phases in BaCo$_2$($X$O$_4$)$_2$., Comment: 8 pages, 4 figures

We develop MultiSTOP, a Reinforcement Learning framework for solving functional equations in physics. This new methodology produces actual numerical solutions instead of bounds on them. We extend the original BootSTOP algorithm by adding multiple constraints derived from domain-specific knowledge, even in integral form, to improve the accuracy of the solution. We investigate a particular equation in a one-dimensional Conformal Field Theory., Comment: ICLR 2024 Workshop on AI4DifferentialEquations In Science

Continual Learning (CL) focuses on maximizing the predictive performance of a model across a non-stationary stream of data. Unfortunately, CL models tend to forget previous knowledge, thus often underperforming when compared with an offline model trained jointly on the entire data stream. Given that any CL model will eventually make mistakes, it is of crucial importance to build calibrated CL models: models that can reliably tell their confidence when making a prediction. Model calibration is an active research topic in machine learning, yet to be properly investigated in CL. We provide the first empirical study of the behavior of calibration approaches in CL, showing that CL strategies do not inherently learn calibrated models. To mitigate this issue, we design a continual calibration approach that improves the performance of post-processing calibration methods over a wide range of different benchmarks and CL strategies. CL does not necessarily need perfect predictive models, but rather it can benefit from reliable predictive models. We believe our study on continual calibration represents a first step towards this direction., Comment: Accepted at CLVISION workshop, CVPR 2024

We employ ab-initio electronic structure calculations to investigate the charge-density waves and periodic lattice distortions in bilayer 2H-NbSe$_2$. We demonstrate that the vertical stacking can give rise to a variety of patterns that may lower the symmetry of the charge-density waves exhibited separately by the two composing 1H-NbSe$_2$ monolayers. The general tendency to a spontaneous symmetry breaking observed in the ground state and the first excited states is shown to originate from a non-negligible inter-layer coupling. Simulated images for scanning tunnelling microscopy (STM) as well as diffraction/scattering patterns show signatures of the different stacking orders. This may not only be useful to reinterpret past experiments on surfaces and thin films, but may also be exploited to devise ad-hoc experiments for the investigation of the stacking order in 2H-NbSe$_2$. We anticipate that our analysis does not only apply to the 2H-NbSe$_2$ bilayer, but is also relevant for thin films and bulk, whose smallest centro-symmetric component is indeed the bilayer. Finally, our results illustrate clearly that the vertical stacking is not only important for 1T structures, as exemplified by the metal-to-insulator transition observed in 1T-TaS$_2$, but seems to be a general feature of metallic layered transition metal dichalcogenides as well., Comment: 15 pages, 9 figures

We explore the inelastic spectra of electrons impinging in a magnetic system. The methodology here presented is intended to highlight the charge-dependent interaction of the electron beam in a STEM-EELS experiment, and the local vector potential generated by the magnetic lattice. This interaction shows an intensity $10^{-2}$ smaller than the purely spin interaction, which is taken to be functionally the same as in the inelastic neutron experiment. On the other hand, it shows a strong scattering vector dependence ($\kappa^{-4}$) and a dependence with the relative orientation between the probe wavevector and the local magnetic moments of the solid. We present YIG as a case study due to its high interest by the community.

As highlighted in a series of recent papers by Tringali and the author, fundamental aspects of the classical theory of factorization can be significantly generalized by blending the languages of monoids and preorders. Specifically, the definition of a suitable preorder on a monoid allows for the exploration of decompositions of its elements into (more or less) arbitrary factors. We provide an overview of the principal existence theorems in this new theoretical framework. Furthermore, we showcase additional applications beyond classical factorization, emphasizing its generality. In particular, we recover and refine a classical result by Howie on idempotent factorizations in the full transformation monoid of a finite set.

Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by examining recent continual learning papers published at four major machine learning conferences, and show that memory-constrained settings dominate the field. Then, we discuss five open problems in machine learning, and even though they might seem unrelated to continual learning at first sight, we show that continual learning will inevitably be part of their solution. These problems are model editing, personalization and specialization, on-device learning, faster (re-)training and reinforcement learning. Finally, by comparing the desiderata from these unsolved problems and the current assumptions in continual learning, we highlight and discuss four future directions for continual learning research. We hope that this work offers an interesting perspective on the future of continual learning, while displaying its potential value and the paths we have to pursue in order to make it successful. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.

Using first-principles techniques, we study the structural, magnetic and electronic properties of (111)-oriented (LaMnO$_3$)$_{2n}$$\vert$(SrMnO$_3$)$_{n}$ superlattices of varying thickness ($n=2,4,6$). We find that the properties of the thinnest superlattice ($n=2$) are similar to the celebrated half-metallic ferromagnetic alloy La$_{2/3}$Sr$_{1/3}$MnO$_3$, with quenched Jahn-Teller distortions. At intermediate thickness ($n=4$), the $a^{-}a^{-}a^{-}$ tilting pattern transitions to the $a^{-}a^{-}c^{+}$ tilting pattern, driven by the lattice degrees of freedom in the LaMnO$_3$ region. The emergence of the Jahn-Teller modes and the spatial extent needed for their development play a key role in this structural transition. For the largest thickness considered ($n=6$), we unveil an emergent separation of Jahn-Teller and volume-breathing orders in the ground-state structure with the $a^{-}a^{-}c^{+}$ tilting pattern, whereas it vanishes in the antiferromagnetic configurations. The ground state of all superlattices is half-metallic ferromagnetic, not affected by the underlying series of structural transitions. Overall, these results outline a thickness-induced crossover between the physical properties of bulk La$_{2/3}$Sr$_{1/3}$MnO$_3$ and bulk LaMnO$_3$., Comment: 7 pages excluding references, 8 figures

Online continual learning aims to get closer to a live learning experience by learning directly on a stream of data with temporally shifting distribution and by storing a minimum amount of data from that stream. In this empirical evaluation, we evaluate various methods from the literature that tackle online continual learning. More specifically, we focus on the class-incremental setting in the context of image classification, where the learner must learn new classes incrementally from a stream of data. We compare these methods on the Split-CIFAR100 and Split-TinyImagenet benchmarks, and measure their average accuracy, forgetting, stability, and quality of the representations, to evaluate various aspects of the algorithm at the end but also during the whole training period. We find that most methods suffer from stability and underfitting issues. However, the learned representations are comparable to i.i.d. training under the same computational budget. No clear winner emerges from the results and basic experience replay, when properly tuned and implemented, is a very strong baseline. We release our modular and extensible codebase at https://github.com/AlbinSou/ocl_survey based on the avalanche framework to reproduce our results and encourage future research., Comment: ICCV Visual Continual Learning Workshop 2023 accepted paper

NEWFOCUS is an EU COST Action targeted at exploring radical solutions that could influence the design of future wireless networks. The project aims to address some of the challenges associated with optical wireless communication (OWC) and to establish it as a complementary technology to the radio frequency (RF)-based wireless systems in order to meet the demanding requirements of the fifth generation (5G) and the future sixth generation (6G) backhaul and access networks. Only 6G will be able to widely serve the exponential growth in connected devices (i.e., more than 500 billion) in 2030, real-time holographic communication, future virtual reality, etc. Space is emerging as the new frontier in 5 and 6G and beyond communication networks, where it offers high-speed wireless coverage to remote areas both in lands and sees. This activity is supported by the recent development of low-altitude Earth orbit satellite mega-constellations. The focus of this 2nd White Paper is on the use of OWC as an enabling technology for medium- and long-range links for deployment in (i) smart-cities and intelligent transportation systems; (ii) first- and last-mile access and backhaul/fronthaul wireless networks; (iii) hybrid free-space optics/RF adaptive wireless connections; (iv) space-to-ground, inter-satellite, ground-to-air, and air-to-air communications; and (v) underwater communications.

Continual Learning trains models on a stream of data, with the aim of learning new information without forgetting previous knowledge. Given the dynamic nature of such environments, explaining the predictions of these models can be challenging. We study the behavior of SHAP values explanations in Continual Learning and propose an evaluation protocol to robustly assess the change of explanations in Class-Incremental scenarios. We observed that, while Replay strategies enforce the stability of SHAP values in feedforward/convolutional models, they are not able to do the same with fully-trained recurrent models. We show that alternative recurrent approaches, like randomized recurrent models, are more effective in keeping the explanations stable over time., Comment: ESANN 2023, 6 pages, added link to code

The influence of stable and unstable stratification on the amplification of coherent structures in turbulent channel flows is investigated by computing the linear response to stochastic forcing. The responses to momentum and thermal forcing are considered separately. Consistently with results of previous direct numerical simulations, the influence of the mean flow stratification on stochastic forcing amplifications is found to be non-negligible only for streamwise-elongated large-scale structures. Unstable stratification is found to enhance the peak variance of the response, except for the velocity response to thermal forcing, and to increase the spanwise wavelength of the most amplified structures. Stable stratification induces opposite effects. The spanwise wavelengths maximizing the different types of variance amplifications, all converge to approximately 6h when approaching the linear instability threshold where large-scale coherent rolls become linearly unstable. In the presence of even moderately unstable stratification, the profiles of turbulent buoyancy and momentum fluxes and of rms vertical velocity of all types of most amplified stochastic responses are nearly indistinguishable from those of the critical mode becoming unstable at the critical Richardson number. The two most energetic POD modes always contribute to more than 90% of the variance of the response, except for the thermal response to thermal forcing. We conclude that the same mechanism underlies the onset of the instability of coherent large-scale rolls at the critical Richardson number and the amplification of coherent large-scale structures at subcritical Richardson numbers., Comment: accepted for publication in Phys. Rev. Fluids

Distributed learning on the edge often comprises self-centered devices (SCD) which learn local tasks independently and are unwilling to contribute to the performance of other SDCs. How do we achieve forward transfer at zero cost for the single SCDs? We formalize this problem as a Distributed Continual Learning scenario, where SCD adapt to local tasks and a CL model consolidates the knowledge from the resulting stream of models without looking at the SCD's private data. Unfortunately, current CL methods are not directly applicable to this scenario. We propose Data-Agnostic Consolidation (DAC), a novel double knowledge distillation method that consolidates the stream of SC models without using the original data. DAC performs distillation in the latent space via a novel Projected Latent Distillation loss. Experimental results show that DAC enables forward transfer between SCDs and reaches state-of-the-art accuracy on Split CIFAR100, CORe50 and Split TinyImageNet, both in reharsal-free and distributed CL scenarios. Somewhat surprisingly, even a single out-of-distribution image is sufficient as the only source of data during consolidation.

Continual learning is the problem of learning from a nonstationary stream of data, a fundamental issue for sustainable and efficient training of deep neural networks over time. Unfortunately, deep learning libraries only provide primitives for offline training, assuming that model's architecture and data are fixed. Avalanche is an open source library maintained by the ContinualAI non-profit organization that extends PyTorch by providing first-class support for dynamic architectures, streams of datasets, and incremental training and evaluation methods. Avalanche provides a large set of predefined benchmarks and training algorithms and it is easy to extend and modular while supporting a wide range of continual learning scenarios. Documentation is available at \url{https://avalanche.continualai.org}.

Real-world data streams naturally include the repetition of previous concepts. From a Continual Learning (CL) perspective, repetition is a property of the environment and, unlike replay, cannot be controlled by the agent. Nowadays, the Class-Incremental (CI) scenario represents the leading test-bed for assessing and comparing CL strategies. This scenario type is very easy to use, but it never allows revisiting previously seen classes, thus completely neglecting the role of repetition. We focus on the family of Class-Incremental with Repetition (CIR) scenario, where repetition is embedded in the definition of the stream. We propose two stochastic stream generators that produce a wide range of CIR streams starting from a single dataset and a few interpretable control parameters. We conduct the first comprehensive evaluation of repetition in CL by studying the behavior of existing CL strategies under different CIR streams. We then present a novel replay strategy that exploits repetition and counteracts the natural imbalance present in the stream. On both CIFAR100 and TinyImageNet, our strategy outperforms other replay approaches, which are not designed for environments with repetition., Comment: Accepted to the 2nd Conference on Lifelong Learning Agents (CoLLAs), 2023 19 pages

This paper describes the fundamental timing properties of a single-pixel sensor for charged particle detection based on the 3D-trench silicon structure. We derive the results both analytically and numerically by considering a simple ideal sensor and the corresponding fast front-end electronics in two different case scenarios: ideal integrator and real fast electronics (trans-impedance amplifier). The particular shape of the Time of Arrival (TOA) distribution is examined and the relation between the time resolution and the spread of intrinsic charge collection time is discussed, by varying electronics parameters and discrimination thresholds. The results are obtained with and without simulated electronics noise. We show that the 3D-trench sensors are characterized by a $synchronous~region$, i.e. a portion of the active volume which leads to the same TOA values when charged particles cross it. The synchronous region size is dependent on the front-end electronics and discrimination threshold, and the phenomenon represents an intrinsic physical effect that leads to the excellent time resolution of these sensors. Moreover, we show that the TOA distribution is characterized by an intrinsic asymmetry, due to the 3D geometry only, that becomes negligible in case of significant electronics jitter.

Detectors based on pixels with timing capabilities are gaining increasing importance in the last years. Next-to-come high-energy physics experiments at colliders require the use of time information in tracking, due to the expected levels of track densities in the foreseen experimental conditions. A promising solution to gain high-resolution performance at the sensor level is given by so-called 3D silicon sensors. The excellent intrinsic time resolution of a special case of 3D sensors, the trench type, is limited by residual non-uniformities in the duration of the induced currents. The intrinsic contribution of the sensor to the total time resolution of the system, when the detector is coupled to a front-end electronics, depends on the characteristics of the electronics itself and can be minimized with a proper design. This paper aims to analyze the possible performance in the timing of a typically-used front-end circuit, the Trans-Impedance Amplifier, considering different possible configurations. Evidence of the preferred modes of operation in sensor read-out for timing measurement will be given., Comment: arXiv admin note: text overlap with arXiv:2008.09867

Let $H$ be a monoid, $\mathscr F(X)$ be the free monoid on a set $X$, and $\pi_H$ be the unique extension of the identity map on $H$ to a monoid homomorphism $\mathscr F(H) \to H$. Given $A \subseteq H$, an $A$-word $\mathfrak z$ (i.e., an element of $\mathscr F(A)$) is minimal if $\pi_H(\mathfrak z) \ne \pi_H(\mathfrak z')$ for every permutation $\mathfrak z'$ of a proper subword of $\mathfrak z$. The minimal $A$-elasticity of $H$ is then the supremum of all rational numbers $m/n$ with $m, n \in \mathbb N^+$ such that there exist minimal $A$-words $\mathfrak a$ and $\mathfrak b$ of length $m$ and $n$, resp., with $\pi_H(\mathfrak a) = \pi_H(\mathfrak b)$. Among other things, we show that if $H$ is commutative and $A$ is finite, then the minimal $A$-elasticity of $H$ is finite. This yields a non-trivial generalization of the finiteness part of a classical theorem of Anderson et al. from the case where $H$ is cancellative, commutative, and finitely generated (f.g.) modulo units and $A$ is the set $\mathscr A(H)$ of its atoms. We also check that commutativity is somewhat essential here, by proving the existence of an atomic, cancellative, f.g. monoid with trivial group of units whose minimal $\mathscr A(H)$-elasticity is infinite., Comment: 13 pages, no figures. To appear in Arkiv f\"or Matematik

In this paper the results of a beam test characterization campaign of 3D trench silicon pixel sensors are presented. A time resolution in the order of 10 ps was measured both for non-irradiated and irradiated sensors up to a fluence of $2.5 \cdot 10^{16}\,1\,MeV\, n_{eq}\,cm^{-2}$. This feature and a detection efficiency close to $99\%$ make this sensors one of the best candidates for 4D tracking detectors in High-Energy-Physics experiments., Comment: Prepared for submission to JINST, IWORID 2022

The next generation of collider experiments require tracking detectors with extreme performance capabilities in terms of spatial resolution (tens of $\mu \text{m}$), radiation hardness ($10^{17}~1~$MeV n$_{eq}/$cm$^2$) and timing resolution (tens of ps). 3D silicon sensors, recently developed within the TimeSPOT initiative, offer a viable solution to cope with such demanding requirements. In order to accurately characterize the timing performance of these new sensors, several read-out boards, based on discrete active components, have been designed, assembled, and tested. The same electronics is also suitable for characterization of similar pixel sensors whenever timing performance in the order and below 10 ps is a requirement. This paper describes the general characteristics needed by front-end electronics to exploit solid-state sensors with fast timing capabilities and in particular, showcases the performance of the developed electronics in the testing and characterization of fast 3D silicon sensors.

It has been recently observed that fundamental aspects of the classical theory of factorization can be greatly generalized by combining the languages of monoids and preorders. This has led to various theorems on the existence of certain factorizations, herein called $\preceq$-factorizations, for the $\preceq$-non-units of a (multiplicatively written) monoid $H$ endowed with a preorder $\preceq$, where an element $u \in H$ is a $\preceq$-unit if $u \preceq 1_H \preceq u$ and a $\preceq$-non-unit otherwise. The ``building blocks'' of these factorizations are the $\preceq$-irreducibles of $H$ (i.e., the $\preceq$-non-units $a \in H$ that cannot be written as a product of two $\preceq$-non-units each of which is strictly $\preceq$-smaller than $a$); and it is interesting to look for sufficient conditions for the $\preceq$-factorizations of a $\preceq$-non-unit to be bounded in length or finite in number (if measured or counted in a suitable way). This is precisely the kind of questions addressed in the present work, whose main novelty is the study of the interaction between minimal $\preceq$-factorizations (i.e., a refinement of $\preceq$-factorizations used to counter the ``blow-up phenomena'' that are inherent to factorization in non-commutative or non-cancellative monoids) and some finiteness conditions describing the ``local behaviour'' of the pair $(H, \preceq)$. Besides a number of examples and remarks, the paper includes many arithmetic results, a part of which are new already in the basic case where $\preceq$ is the divisibility preorder on $H$ (and hence in the setup of the classical theory)., Comment: 27 pages, 2 figures. Final version to appear in Journal of Algebra

Continual Learning (CL) on time series data represents a promising but under-studied avenue for real-world applications. We propose two new CL benchmarks for Human State Monitoring. We carefully designed the benchmarks to mirror real-world environments in which new subjects are continuously added. We conducted an empirical evaluation to assess the ability of popular CL strategies to mitigate forgetting in our benchmarks. Our results show that, possibly due to the domain-incremental properties of our benchmarks, forgetting can be easily tackled even with a simple finetuning and that existing strategies struggle in accumulating knowledge over a fixed, held-out, test subject., Comment: 6 pages, 4 figures, 2 tables, Accepted as oral at ESANN 2022

Online Continual learning is a challenging learning scenario where the model must learn from a non-stationary stream of data where each sample is seen only once. The main challenge is to incrementally learn while avoiding catastrophic forgetting, namely the problem of forgetting previously acquired knowledge while learning from new data. A popular solution in these scenario is to use a small memory to retain old data and rehearse them over time. Unfortunately, due to the limited memory size, the quality of the memory will deteriorate over time. In this paper we propose OLCGM, a novel replay-based continual learning strategy that uses knowledge condensation techniques to continuously compress the memory and achieve a better use of its limited size. The sample condensation step compresses old samples, instead of removing them like other replay strategies. As a result, the experiments show that, whenever the memory budget is limited compared to the complexity of the data, OLCGM improves the final accuracy compared to state-of-the-art replay strategies., Comment: Accepted as a conference paper at 2022 International Joint Conference on Neural Networks (IJCNN 2022). Part of 2022 IEEE World Congress on Computational Intelligence (IEEE WCCI 2022)

In this work, we report on an $ab-initio$ computational study of the electronic and magnetic properties of transition metal adatoms on a monolayer of NbSe$_2$. We demonstrate that Cr, Mn, Fe and Co prefer all to sit above the Nb atom, where the $d$ states experience a substantial hybridization. The inter-atomic exchange coupling is shown to have an oscillatory nature accompanied by an exponential decay, in accordance with what theory predicts for a damped Ruderman-Kittel-Kasuya-Yosida (RKKY) interaction. Our results indicate that the qualitative features of the magnetic coupling for the four investigated adatoms can be connected to the fine details of their Fermi surface. In particular, the oscillations of the exchange in Fe and Co are found to be related to a single nesting vector, connecting large electrons and hole pockets. Most interestingly, this behavior is found to be unaffected by changes induced on the height of the impurity, which makes the magnetism robust to external perturbations. Considering that NbSe$_2$ is a superconductor down to a single layer, our research might open the path for further research into the interplay between magnetic and superconducting characteristics, which could lead to novel superconductivity engineering.

Pre-trained models are nowadays a fundamental component of machine learning research. In continual learning, they are commonly used to initialize the model before training on the stream of non-stationary data. However, pre-training is rarely applied during continual learning. We formalize and investigate the characteristics of the continual pre-training scenario in both language and vision environments, where a model is continually pre-trained on a stream of incoming data and only later fine-tuned to different downstream tasks. We show that continually pre-trained models are robust against catastrophic forgetting and we provide strong empirical evidence supporting the fact that self-supervised pre-training is more effective in retaining previous knowledge than supervised protocols. Code is provided at https://github.com/AndreaCossu/continual-pretraining-nlp-vision ., Comment: under review

Continual Learning requires the model to learn from a stream of dynamic, non-stationary data without forgetting previous knowledge. Several approaches have been developed in the literature to tackle the Continual Learning challenge. Among them, Replay approaches have empirically proved to be the most effective ones. Replay operates by saving some samples in memory which are then used to rehearse knowledge during training in subsequent tasks. However, an extensive comparison and deeper understanding of different replay implementation subtleties is still missing in the literature. The aim of this work is to compare and analyze existing replay-based strategies and provide practical recommendations on developing efficient, effective and generally applicable replay-based strategies. In particular, we investigate the role of the memory size value, different weighting policies and discuss about the impact of data augmentation, which allows reaching better performance with lower memory sizes.

We discuss a substantial update to the Grid software library for Lattice QCD, enabling it to port to multiple GPU architectures while retaining CPU vectorisation and SIMD execution within OpenMP threads. The GPU environments supported include vendor specific Nvidia CUDA and AMD HIP environments and a (mostly) standards based SYCL implementation. This is performed by an internal abstraction interface giving single source cross-platform performance portability across all number of planned Exascale architectures, and all those planned by the US Department of Energy., Comment: 10 pages, 6 figures

Learning continually from non-stationary data streams is a challenging research topic of growing popularity in the last few years. Being able to learn, adapt, and generalize continually in an efficient, effective, and scalable way is fundamental for a sustainable development of Artificial Intelligent systems. However, an agent-centric view of continual learning requires learning directly from raw data, which limits the interaction between independent agents, the efficiency, and the privacy of current approaches. Instead, we argue that continual learning systems should exploit the availability of compressed information in the form of trained models. In this paper, we introduce and formalize a new paradigm named "Ex-Model Continual Learning" (ExML), where an agent learns from a sequence of previously trained models instead of raw data. We further contribute with three ex-model continual learning algorithms and an empirical setting comprising three datasets (MNIST, CIFAR-10 and CORe50), and eight scenarios, where the proposed algorithms are extensively tested. Finally, we highlight the peculiarities of the ex-model paradigm and we point out interesting future research directions.

The ability of a model to learn continually can be empirically assessed in different continual learning scenarios. Each scenario defines the constraints and the opportunities of the learning environment. Here, we challenge the current trend in the continual learning literature to experiment mainly on class-incremental scenarios, where classes present in one experience are never revisited. We posit that an excessive focus on this setting may be limiting for future research on continual learning, since class-incremental scenarios artificially exacerbate catastrophic forgetting, at the expense of other important objectives like forward transfer and computational efficiency. In many real-world environments, in fact, repetition of previously encountered concepts occurs naturally and contributes to softening the disruption of previous knowledge. We advocate for a more in-depth study of alternative continual learning scenarios, in which repetition is integrated by design in the stream of incoming information. Starting from already existing proposals, we describe the advantages such class-incremental with repetition scenarios could offer for a more comprehensive assessment of continual learning models., Comment: Under review

The increasing attention on Artificial Intelligence (AI) regulation has led to the definition of a set of ethical principles grouped into the Sustainable AI framework. In this article, we identify Continual Learning, an active area of AI research, as a promising approach towards the design of systems compliant with the Sustainable AI principles. While Sustainable AI outlines general desiderata for ethical applications, Continual Learning provides means to put such desiderata into practice., Comment: Accepted at the 2021 International Conference on AI for People (CAIP)

Let $\preceq$ be a preorder on a monoid $H$ and $s$ be an integer $\ge 2$. The $\preceq$-height of an $x \in H$ is the sup of the integers $k \ge 1$ for which there is a (strictly) $\preceq$-decreasing sequence $x_1,\ldots,x_k$ of $\preceq$-non-units of $H$ with $x_1 = x$ (with $\sup\emptyset:=0$), where $u\in H$ is a $\preceq$-unit if $u\preceq 1_H\preceq u$ and a $\preceq$-non-unit otherwise. We say $H$ is $\preceq$-artinian if there exists no $\preceq$-decreasing sequence $x_1,x_2,\ldots$ of elements of $H$; and strongly $\preceq$-artinian if the $\preceq$-height of each element is finite. We establish that, if $H$ is $\preceq$-artinian, then each $\preceq$-non-unit $x\in H$ factors through the $\preceq$-irreducibles of degree $s$, where a $\preceq$-irreducible of degree $s$ is a $\preceq$-non-unit $a\in H$ that cannot be written as a product of $s$ or fewer $\preceq$-non-units each of which is (strictly) smaller than $a$ with respect to $\preceq$. In addition, we show that, if $H$ is strongly $\preceq$-artinian, then $x$ factors through the $\preceq$-quarks of $H$, where a $\preceq$-quark is a $\preceq$-min $\preceq$-non-unit. In the process, we also obtain upper bounds for the length of a shortest factorization of $x$ (into either $\preceq$-irreducible of degree $s$ or $\preceq$-quarks) in terms of its $\preceq$-height. Next, we specialize these abstract results to the case in which $H$ is the multiplicative submonoid of a ring $R$ formed by the zero divisors and the identity $1_R$, and $\preceq$ is the preorder on $H$ defined by $a\preceq b$ iff $r_R(1_R-b)\subseteq r_R(1_R-a)$, where $r_R(\cdot)$ denotes a right annihilator. We can thus recover and improve on classical theorems of J.A. Erdos (1967), R.J.H. Dawlings (1981), and J. Fountain (1991) on idempotent factorizations in the endomorphism ring of a free module of finite rank over a skew field or a commutative DVD., Comment: 29 pages, no figures. To appear in Israel J. Math

Heterostructures of mixed-valence manganites are still under intense scrutiny, due to the occurrence of exotic quantum phenomena linked to electronic correlation and interfacial composition. For instance, if two anti-ferromagnetic insulators as LaMnO$_3$ and SrMnO$_3$ are grown in a (001)-oriented superlattice, a half-metallic ferromagnet may form, provided that the thickness is sufficiently small to allow tunneling across interfaces. In this article, we employ electronic structure calculations to show that all the layers of a (111)-oriented LaMnO$_3$|SrMnO$_3$ superlattice retain a half-metallic ferromagnetic character for a much larger thickness than in its (001) counterpart. This behavior is shown to be linked to the charge transfer across the interface, favored by the octahedral connectivity between the layers. This also results in a symmetry-induced quenching of the Jahn-Teller distortions, which are replaced by breathing modes. The latter are coupled to charge and spin oscillations, whose components have a pure e g character. Most interestingly, the magnetization reaches its maximum value inside the LaMnO$_3$ region and not at the interface, which is fundamentally different from what observed for the (001) orientation. The analysis of the inter-atomic exchange coupling shows that the magnetic order arises from the double-exchange mechanism, despite competing interactions inside the SrMnO$_3$ region. Finally, the van Vleck distortions and the spin oscillations are found to be crucially affected by the variation of Hund's exchange and charge doping, which allows us to speculate that our system behaves as a Hund's metal, creating an interesting connection between manganites and nickelates., Comment: 24 pages, 6 figures

This paper presents the detailed simulation of a double-pixel structure for charged particle detection based on the 3D-trench silicon sensor developed for the TIMESPOT project and a comparison of the simulation results with measurements performed at $\pi-$M1 beam at PSI laboratory. The simulation is based on the combined use of several software tools (TCAD, GEANT4, TCoDe and TFBoost) which allow to fully design and simulate the device physics response in very short computational time, O(1-100 s) per simulated signal, by exploiting parallel computation using single or multi-thread processors. This allowed to produce large samples of simulated signals, perform detailed studies of the sensor characteristics and make precise comparisons with experimental results., Comment: 23 pages, 18 figures

Continual Learning (CL) refers to a learning setup where data is non stationary and the model has to learn without forgetting existing knowledge. The study of CL for sequential patterns revolves around trained recurrent networks. In this work, instead, we introduce CL in the context of Echo State Networks (ESNs), where the recurrent component is kept fixed. We provide the first evaluation of catastrophic forgetting in ESNs and we highlight the benefits in using CL strategies which are not applicable to trained recurrent models. Our results confirm the ESN as a promising model for CL and open to its use in streaming scenarios., Comment: Accepted as oral at ESANN 2021

Learning continually from non-stationary data streams is a long-standing goal and a challenging problem in machine learning. Recently, we have witnessed a renewed and fast-growing interest in continual learning, especially within the deep learning community. However, algorithmic solutions are often difficult to re-implement, evaluate and port across different settings, where even results on standard benchmarks are hard to reproduce. In this work, we propose Avalanche, an open-source end-to-end library for continual learning research based on PyTorch. Avalanche is designed to provide a shared and collaborative codebase for fast prototyping, training, and reproducible evaluation of continual learning algorithms., Comment: Official Website: https://avalanche.continualai.org

Replay strategies are Continual Learning techniques which mitigate catastrophic forgetting by keeping a buffer of patterns from previous experiences, which are interleaved with new data during training. The amount of patterns stored in the buffer is a critical parameter which largely influences the final performance and the memory footprint of the approach. This work introduces Distilled Replay, a novel replay strategy for Continual Learning which is able to mitigate forgetting by keeping a very small buffer (1 pattern per class) of highly informative samples. Distilled Replay builds the buffer through a distillation process which compresses a large dataset into a tiny set of informative examples. We show the effectiveness of our Distilled Replay against popular replay-based strategies on four Continual Learning benchmarks.

In this work, we study the phenomenon of catastrophic forgetting in the graph representation learning scenario. The primary objective of the analysis is to understand whether classical continual learning techniques for flat and sequential data have a tangible impact on performances when applied to graph data. To do so, we experiment with a structure-agnostic model and a deep graph network in a robust and controlled environment on three different datasets. The benchmark is complemented by an investigation on the effect of structure-preserving regularization techniques on catastrophic forgetting. We find that replay is the most effective strategy in so far, which also benefits the most from the use of regularization. Our findings suggest interesting future research at the intersection of the continual and graph representation learning fields. Finally, we provide researchers with a flexible software framework to reproduce our results and carry out further experiments., Comment: Accepted at the 2021 Web Conference Workshop on Graph Learning Benchmarks (GLB 2021). Code available at https://github.com/diningphil/continual_learning_for_graphs

Protein-protein interactions are the basis of many important physiological processes and are currently promising, yet difficult, targets for drug discovery. In this context, inhibitor of apoptosis proteins (IAPs)-mediated interactions are pivotal for cancer cell survival; the interaction of the BIR1 domain of cIAP2 with TRAF2 was shown to lead the recruitment of cIAPs to the TNF receptor, promoting the activation of the NF-\kappa B survival pathway. In this work, using a combined in silico-in vitro approach, we identified a drug-like molecule, NF023, able to disrupt cIAP2 interaction with TRAF2. We demonstrated in vitro its ability to interfere with the assembly of the cIAP2-BIR1/TRAF2 complex and performed a thorough characterization of the compound's mode of action through 248 parallel unbiased molecular dynamics simulations of 300 ns (totaling almost 75 {\mu}s of all-atom sampling), which identified multiple binding modes to the BIR1 domain of cIAP2 via clustering and ensemble docking. NF023 is, thus, a promising protein-protein interaction disruptor, representing a starting point to develop modulators of NF-\kappa B-mediated cell survival in cancer. This study represents a model procedure that shows the use of large-scale molecular dynamics methods to typify promiscuous interactors.

Learning continuously during all model lifetime is fundamental to deploy machine learning solutions robust to drifts in the data distribution. Advances in Continual Learning (CL) with recurrent neural networks could pave the way to a large number of applications where incoming data is non stationary, like natural language processing and robotics. However, the existing body of work on the topic is still fragmented, with approaches which are application-specific and whose assessment is based on heterogeneous learning protocols and datasets. In this paper, we organize the literature on CL for sequential data processing by providing a categorization of the contributions and a review of the benchmarks. We propose two new benchmarks for CL with sequential data based on existing datasets, whose characteristics resemble real-world applications. We also provide a broad empirical evaluation of CL and Recurrent Neural Networks in class-incremental scenario, by testing their ability to mitigate forgetting with a number of different strategies which are not specific to sequential data processing. Our results highlight the key role played by the sequence length and the importance of a clear specification of the CL scenario., Comment: Published in Neural Networks

In lattice calculations, the approach to the continuum limit is hindered by the severe freezing of the topological charge, which prevents ergodic sampling in configuration space. In order to significantly reduce the autocorrelation time of the topological charge, we develop a density of states approach with a smooth constraint and use it to study SU(3) pure Yang Mills gauge theory near the continuum limit. Our algorithm relies on simulated tempering across a range of couplings, which guarantees the decorrelation of the topological charge and ergodic sampling of topological sectors. Particular emphasis is placed on testing the accuracy, efficiency and scaling properties of the method. In their most conservative interpretation, our results provide firm evidence of a sizeable reduction of the exponent z related to the growth of the autocorrelation time as a function of the inverse lattice spacing., Comment: 22 pages, 16 figure

We investigate the axial U(1) anomaly of two-flavor QCD at temperatures 190--330 MeV. In order to preserve precise chiral symmetry on the lattice, we employ the Mobius domain-wall fermion action as well as overlap fermion action implemented with a stochastic reweighting technique. Compared to our previous studies, we reduce the lattice spacing to 0.07 fm, simulate larger multiple volumes to estimate finite size effect, and take more than four quark mass points, including one below physical point to investigate the chiral limit. We measure the topological susceptibility, axial U(1) susceptibility, and examine the degeneracy of U(1) partners in meson and baryon correlators. All the data above the critical temperature indicate that the axial U(1) violation is consistent with zero within statistical errors. The quark mass dependence suggests disappearance of the U(1) anomaly at a rate comparable to that of the SU(2)_L x SU(2)_R symmetry breaking., Comment: 40 pages, 19 figures, the version published in PRD

We present a study of the IR behaviour of a three-dimensional super-renormalisable quantum field theory (QFT) consisting of a scalar field in the adjoint of $SU(N)$ with a $\varphi^4$ interaction. A bare mass is required for the theory to be massless at the quantum level. In perturbation theory the critical mass is ambiguous due to infrared (IR) divergences and we indeed find that at two-loops in lattice perturbation theory the critical mass diverges logarithmically. It was conjectured long ago in [Jackiw 1980, Appelquist 1981] that super-renormalisable theories are nonperturbatively IR finite, with the coupling constant playing the role of an IR regulator. Using a combination of Markov-Chain-Monte-Carlo simulations of the lattice-regularised theory, both frequentist and Bayesian data analysis, and considerations of a corresponding effective theory we gather evidence that this is indeed the case., Comment: Version accepted for publication in PRL, added references, updated plots, extended discussion of IR behaviour in perturbation theory and EFT

Wake redirection is a promising approach designed to mitigate turbine-wake interactions which have a negative impact on the performance and lifetime of wind farms. It has recently been found that substantial power gains can be obtained by tilting the rotors of spanwise-periodic wind-turbine arrays. Rotor tilt is associated to the generation of coherent streamwise vortices which deflect wakes towards the ground and, by exploiting the vertical wind shear, replace them with higher-momentum fluid (high-speed streaks). The objective of this work is to evaluate power gains that can be obtained by tilting rotors in spanwise-periodic wind-turbine arrays immersed in the atmospheric boundary layer and, in particular, to analyze the influence of the rotor size on power gains in the case where the turbines emerge from the atmospheric surface layer. We show that, for the case of wind-aligned arrays, large power gains can be obtained for positive tilt angles of the order of 30{\deg}. Power gains are substantially enhanced by operating tilted-rotor turbines at thrust coefficients higher than in the reference configuration. These power gains initially increase with the rotor size reaching a maximum for rotor diameters of the order of 3.6 boundary layer momentum thicknesses (for the considered cases) and decrease for larger sizes. Maximum power gains are obtained for wind-turbine spanwise spacings which are very similar to those of large-scale and very large scale streaky motions which are naturally amplified in turbulent boundary layers. These results are all congruent with the findings of previous investigations of passive control of canonical boundary layers for drag-reduction applications where high-speed streaks replaced wakes of spanwise-periodic rows of wall-mounted roughness elements., Comment: submitted to Wind Energy Science (revised version)

Detectors based on pixels with timing capabilities are gaining increasing importance in the last years. Next-to-come high-energy physics experiments at colliders requires the use of time information in tracking, due to the increasing levels of track densities in the foreseen experimental conditions. Various different developments are ongoing on solid state sensors to gain high-resolution performance at the sensor level, as for example LGAD sensors or 3D sensors. Intrinsic sensor time resolution around 20 ps have been recently obtained. The increasing performance on the sensor side strongly demands an adequate development on the front-end electronics side, which now risks to become the performance bottle-neck in a tracking or vertex-detecting system. This paper aims to analyse the ultimate possible performance in timing of a typically-used front-end circuit, the Trans-Impedance Amplifier, considering different possible circuit configurations. Evidence to the preferable modes of operation in sensor read-out for timing measurement will be given., Comment: Prepared for submission to JINST

We consider the smallest subring $D$ of $\mathbb{R}(X)$ containing every element of the form $1/(1+x^2)$, with $x\in \mathbb{R}(X)$. $D$ is a Pr\"ufer domain called the minimal Dress ring of $\mathbb{R}(X)$. In this paper, addressing a general open problem for Pr\"ufer non B\'ezout domains, we investigate whether $2\times 2$ singular matrices over $D$ can be decomposed as products of idempotent matrices. We show some conditions that guarantee the idempotent factorization in $M_2(D)$.

Wind turbine wakes negatively impact downwind turbines in wind farms reducing their global efficiency. The reduction of wake-turbine interactions by actuating control on yaw angles and induction factors is an active area of research. In this study, the capability of spanwise-periodic wind turbine arrays with tilted rotors to reduce negative turbine-wakes interaction is investigated by means of large-eddy simulations. It is shown that by means of rotor tilt it is possible to replace turbine far wakes with high-speed streaks where the streamwise velocity exceeds the freestream velocity at hub height. Considering three aligned arrays of wind turbines, it is found that the global power extracted from the wind can be increased by tilting rotors of upwind turbine arrays similarly to what already known for the case of a single row of aligned turbines. It is further shown that global tilt-induced power gains can be significantly increased by operating the tilted turbines at higher induction rates. Power gains can be further increased by increasing the ratio of the rotor diameters and turbine spacing to the boundary layer thickness. All these findings are consistent with those of previous studies where streamwise streaks were artificially forced by means of arrays of wall-mounted roughness elements in order to control canonical boundary layers for drag-reduction purposes., Comment: revised version

Oxidation strongly influences the properties of magnetic layers employed in spintronic devices. We study the effect of oxidation on the structural, magnetic, and electrical properties as well as current-induced spin-orbit torques (SOTs) in Pt/Co/AlOx, Pt/CoOx/Co/AlOx, and PtOx/Co/AlOx layers. We show how the saturation magnetization, perpendicular magnetic anisotropy, anomalous Hall resistance, and SOT are systematically affected by the degree of oxidation of both the Pt/Co and Co/Al interfaces. Oxidation of the Co/Al interface results in a 21% and 42% variation of the dampinglike and fieldlike SOT efficiencies, which peak at 0.14 and 0.07, respectively. The insertion of a paramagnetic CoOx layer between Pt and Co maintains a very strong perpendicular magnetic anisotropy and improves the dampinglike and fieldlike SOT efficiencies, up to 0.26 and 0.20, respectively. In contrast with recent reports, we do not find that the oxidation of Pt leads to a significant enhancement of the torques. Rather, we find that oxygen migrates from Pt to the Co and Al layers, leading to a time-dependent oxidation profile and an effective spin Hall conductivity that decreases with increasing oxygen concentration. Finally, we study current-induced switching in Pt/Co/AlOx with different degrees of oxidation and find a linear relationship between the critical switching current and the effective magnetic anisotropy controlled by the oxidation of Al. These results highlight the importance of interfaces and oxidation effects on the SOT and magnetotransport properties of heavy metal/ferromagnet/oxide trilayers and provide information on how to improve the SOT efficiency and magnetization-switching characteristics of these systems.

Large-scale motions, also known as superstructures, are dynamically relevant coherent structures in a wall-bounded turbulent flow, that span the entire domain in wall-normal direction and significantly contribute to the global energy and momentum transport. Recent investigations in channel and Couette flow, suggest that these large-scale motions are self-sustained, implying they are not driven by small-scale motions at the wall. Whether large-scale motions are self-sustained has however not yet been answered for open boundary layers, which are relevant for many applications. Here, using the asymptotic suction boundary layer flow at the friction Reynolds number $Re_\tau = 1\,168$ as a testbed, we show that large-scale motions are self-sustained also in boundary layers. Together with the previous investigations in confined flows, this observation provides strong evidence of the robust and general nature of coherent self-sustained processes in turbulent wall-bounded flows. The dynamics of the large-scale self-sustained process involving the growth, breakdown and regeneration of quasi-streamwise coherent streaks and vortices within the boundary layer shows temporal phase relations reminiscent of bursting as observed in buffer layer structures. The dynamics however differs from the quasi-periodic large-scale streak-vortex regeneration cycle observed in confined flows. Based on the similarity of the dynamics of large-scale motions in boundary layers and of small-scale buffer layer structures, we conjecture that the bursting behaviour is associated with the dynamical relevance of only one wall, while two confining walls lead to the quasi-periodic cycle.

The ability to learn in dynamic, nonstationary environments without forgetting previous knowledge, also known as Continual Learning (CL), is a key enabler for scalable and trustworthy deployments of adaptive solutions. While the importance of continual learning is largely acknowledged in machine vision and reinforcement learning problems, this is mostly under-documented for sequence processing tasks. This work proposes a Recurrent Neural Network (RNN) model for CL that is able to deal with concept drift in input distribution without forgetting previously acquired knowledge. We also implement and test a popular CL approach, Elastic Weight Consolidation (EWC), on top of two different types of RNNs. Finally, we compare the performances of our enhanced architecture against EWC and RNNs on a set of standard CL benchmarks, adapted to the sequential data processing scenario. Results show the superior performance of our architecture and highlight the need for special solutions designed to address CL in RNNs., Comment: Accepted as a conference paper at 2020 International Joint Conference on Neural Networks (IJCNN 2020). Part of 2020 IEEE World Congress on Computational Intelligence (IEEE WCCI 2020)