Search

Online Continual Learning (CL) solves the problem of learning the ever-emerging new classification tasks from a continuous data stream. Unlike its offline counterpart, in online CL, the training data can only be seen once. Most existing online CL research regards catastrophic forgetting (i.e., model stability) as almost the only challenge. In this paper, we argue that the model's capability to acquire new knowledge (i.e., model plasticity) is another challenge in online CL. While replay-based strategies have been shown to be effective in alleviating catastrophic forgetting, there is a notable gap in research attention toward improving model plasticity. To this end, we propose Collaborative Continual Learning (CCL), a collaborative learning based strategy to improve the model's capability in acquiring new concepts. Additionally, we introduce Distillation Chain (DC), a collaborative learning scheme to boost the training of the models. We adapt CCL-DC to existing representative online CL works. Extensive experiments demonstrate that even if the learners are well-trained with state-of-the-art online CL methods, our strategy can still improve model plasticity dramatically, and thereby improve the overall performance by a large margin. The source code of our work is available at https://github.com/maorong-wang/CCL-DC., Comment: Update Camera-ready revision for CVPR'24

Continual Learning has been challenging, especially when dealing with unsupervised scenarios such as Unsupervised Online General Continual Learning (UOGCL), where the learning agent has no prior knowledge of class boundaries or task change information. While previous research has focused on reducing forgetting in supervised setups, recent studies have shown that self-supervised learners are more resilient to forgetting. This paper proposes a novel approach that enhances memory usage for contrastive learning in UOGCL by defining and using stream-dependent data augmentations together with some implementation tricks. Our proposed method is simple yet effective, achieves state-of-the-art results compared to other unsupervised approaches in all considered setups, and reduces the gap between supervised and unsupervised continual learning. Our domain-aware augmentation procedure can be adapted to other replay-based methods, making it a promising strategy for continual learning., Comment: Accepted to BMVC'23

Online Continual Learning (OCL) addresses the problem of training neural networks on a continuous data stream where multiple classification tasks emerge in sequence. In contrast to offline Continual Learning, data can be seen only once in OCL, which is a very severe constraint. In this context, replay-based strategies have achieved impressive results and most state-of-the-art approaches heavily depend on them. While Knowledge Distillation (KD) has been extensively used in offline Continual Learning, it remains under-exploited in OCL, despite its high potential. In this paper, we analyze the challenges in applying KD to OCL and give empirical justifications. We introduce a direct yet effective methodology for applying Momentum Knowledge Distillation (MKD) to many flagship OCL methods and demonstrate its capabilities to enhance existing approaches. In addition to improving existing state-of-the-art accuracy by more than $10\%$ points on ImageNet100, we shed light on MKD internal mechanics and impacts during training in OCL. We argue that similar to replay, MKD should be considered a central component of OCL. The code is available at \url{https://github.com/Nicolas1203/mkd_ocl}., Comment: Accepted to ICML 2024

Deep neural networks have shown remarkable performance when trained on independent and identically distributed data from a fixed set of classes. However, in real-world scenarios, it can be desirable to train models on a continuous stream of data where multiple classification tasks are presented sequentially. This scenario, known as Continual Learning (CL) poses challenges to standard learning algorithms which struggle to maintain knowledge of old tasks while learning new ones. This stability-plasticity dilemma remains central to CL and multiple metrics have been proposed to adequately measure stability and plasticity separately. However, none considers the increasing difficulty of the classification task, which inherently results in performance loss for any model. In that sense, we analyze some limitations of current metrics and identify the presence of setup-induced forgetting. Therefore, we propose new metrics that account for the task's increasing difficulty. Through experiments on benchmark datasets, we demonstrate that our proposed metrics can provide new insights into the stability-plasticity trade-off achieved by models in the continual learning environment., Comment: 6 pages, presented at MIRU 2023

We use the maximum a posteriori estimation principle for learning representations distributed on the unit sphere. We propose to use the angular Gaussian distribution, which corresponds to a Gaussian projected on the unit-sphere and derive the associated loss function. We also consider the von Mises-Fisher distribution, which is the conditional of a Gaussian in the unit-sphere. The learned representations are pushed toward fixed directions, which are the prior means of the Gaussians; allowing for a learning strategy that is resilient to data drift. This makes it suitable for online continual learning, which is the problem of training neural networks on a continuous data stream, where multiple classification tasks are presented sequentially so that data from past tasks are no longer accessible, and data from the current task can be seen only once. To address this challenging scenario, we propose a memory-based representation learning technique equipped with our new loss functions. Our approach does not require negative data or knowledge of task boundaries and performs well with smaller batch sizes while being computationally efficient. We demonstrate with extensive experiments that the proposed method outperforms the current state-of-the-art methods on both standard evaluation scenarios and realistic scenarios with blurry task boundaries. For reproducibility, we use the same training pipeline for every compared method and share the code at https://github.com/Nicolas1203/ocl-fd., Comment: Fix some typo. Accepted to AAAI24

As a Physicist and a Chemist , we have had the opportunity to teach Algebra in our engineering school in Aix-en-Provence. The article deals with the interactions made between Physics, Chemistry and Algebra., Comment: in French

We study Online Continual Learning with missing labels and propose SemiCon, a new contrastive loss designed for partly labeled data. We demonstrate its efficiency by devising a memory-based method trained on an unlabeled data stream, where every data added to memory is labeled using an oracle. Our approach outperforms existing semi-supervised methods when few labels are available, and obtain similar results to state-of-the-art supervised methods while using only 2.6% of labels on Split-CIFAR10 and 10% of labels on Split-CIFAR100., Comment: Accepted at ICIP'22 Oral presentation

We study a problem of $\pi$ production in heavy ion collisions in the context of the Isospin-dependent Boltzmann-Uehling-Uhlenbeck (IBUU) transport model. We generated nucleon densities using two different models, the Skyrme-Hartree-Fock (SHF) model and configuration interaction shell model (SM). Indeed, inter-nucleon correlations are explicitly taken into account in SM, while they are averaged in the SHF model. As an application of our theoretical frameworks, we calculated the $\pi^{-}$ and $\pi^{+}$ yields in collisions of nuclei with $A = 30-40$ nucleons. We used different harmonic oscillator lengths $b_{HO}$ to generate the harmonic oscillator basis for SM in order to study both theoretical and experimental cases. It is found that SM framework with $b_{HO}$ = 2.5 fm and SHF can be distinguished by the yield of $\pi$ mesons, in this case the density distribution calculated by the shell model produces more $\pi$ in the collision. In comparison, SM with $b_{HO}$ = 2.0 fm is characterized from SHF by the double $\pi^{-}/\pi^{+}$ ratios with different large impact parameters, from which one can find the double $\pi^{-}/\pi^{+}$ ratios of SM change smoother and are less than those of SHF., Comment: 7 pages, 4 figures

The Gamow shell model is utilized to describe nuclear observables of the weakly bound and resonance isotonic states of $^{16}$O at proton drip-line. It is hereby shown that the presence of continuum coupling leads to complex Coulomb contributions in the spectrum of these isotones. The necessity to include the effects of three-body forces, either by a direct calculation or by adding an $A$-dependence to the nucleon-nucleon interaction, already noticed in other theoretical models, is pointed out. It is also demonstrated that our approach is predictive for reaction observables., Comment: 12 pages, 10 figures

The existence of multi-neutron systems has always been a debatable question. Indeed, both inter-nucleon correlations and a large continuum coupling occur in these states. We then employ the ab-initio no-core Gamow shell model to calculate the resonant energies and widths of the trineutron and tetraneutron systems with realistic interactions. Our results indicate that trineutron and tetraneutron are both unbound and bear broad widths. The calculated energy and width of tetraneutron are also comparable with recent experimental data. Moreover, our calculations suggest that the energy of trineutron is lower than that of tetraneutron, while its resonance width is also narrower. This strongly suggests that trineutron is more likely to be experimentally observed than tetraneutron. We thus suggest experimentalists to search for trineutron at low energy., Comment: 5 pages; 2 figures

Drip-line nuclei have very different properties from those of the valley of stability, as they are weakly bound and resonant. Therefore, the models devised for stable nuclei can no longer be applied therein. Hence, a new theoretical tool, the Gamow Shell Model (GSM), has been developed to study the many-body states occurring at the limits of the nuclear chart. GSM is a configuration interaction model based on the use of the so-called Berggren basis, which contains bound, resonant and scattering states, so that inter-nucleon correlations are fully taken into account and the asymptotes of extended many-body wave functions are precisely handled. However, large complex symmetric matrices must be diagonalized in this framework, therefore the use of very powerful parallel machines is needed therein. In order to fully take advantage of their power, a 2D partitioning scheme using hybrid MPI/OpenMP parallelization has been developed in our GSM code. The specificities of the 2D partitioning scheme in the GSM framework will be described and illustrated with numerical examples. It will then be shown that the introduction of this scheme in the GSM code greatly enhances its capabilities., Comment: 23 pages; 5 figures; accepted in Computer Physics Communications, in press

We solve for properties of $^6$Li in the ab initio No-Core Full Configuration approach and we separately solve for its ground state and $J^{\pi}=2_{2}^{+}$ resonance with the Gamow Shell Model in the Berggren basis. We employ both the JISP16 and chiral NNLO$_{opt}$ realistic nucleon-nucleon interactions and investigate the ground state energy, excitation energies, point proton root-mean-square radius and a suite of electroweak observables. We also extend and test methods to extrapolate the ground state energy, point proton root-mean-square radius, and electric quadrupole moment. We attain improved estimates of these observables in the No-Core Full Configuration approach by using basis spaces up through N$_{max}$=18 that enable more definitive comparisons with experiment. Using the Density Matrix Renormalization Group approach with the JISP16 interaction, we find that we can significantly improve the convergence of the Gamow Shell Model treatment of the $^6$Li ground state and $J^{\pi}=2_{2}^{+}$ resonance by adopting a natural orbital single-particle basis., Comment: 25 pages, 18 figures

In nature, one observes that a K-theory of an object is defined in two steps. First a "structured" category is associated to the object. Second, a K-theory machine is applied to the latter category to produce an infinite loop space. We develop a general framework that deals with the first step of this process. The K-theory of an object is defined via a category of "locally trivial" objects with respect to a pretopology. We study conditions ensuring an exact structure on such categories. We also consider morphisms in K-theory that such contexts naturally provide. We end by defining various K-theories of schemes and morphisms between them., Comment: 31 pages