Search

Recent research has seen significant interest in methods for concept removal and targeted forgetting in diffusion models. In this paper, we conduct a comprehensive white-box analysis to expose significant vulnerabilities in existing diffusion model unlearning methods. We show that the objective functions used for unlearning in the existing methods lead to decoupling of the targeted concepts (meant to be forgotten) for the corresponding prompts. This is concealment and not actual unlearning, which was the original goal. The ineffectiveness of current methods stems primarily from their narrow focus on reducing generation probabilities for specific prompt sets, neglecting the diverse modalities of intermediate guidance employed during the inference process. The paper presents a rigorous theoretical and empirical examination of four commonly used techniques for unlearning in diffusion models. We introduce two new evaluation metrics: Concept Retrieval Score (CRS) and Concept Confidence Score (CCS). These metrics are based on a successful adversarial attack setup that can recover forgotten concepts from unlearned diffusion models. The CRS measures the similarity between the latent representations of the unlearned and fully trained models after unlearning. It reports the extent of retrieval of the forgotten concepts with increasing amount of guidance. The CCS quantifies the confidence of the model in assigning the target concept to the manipulated data. It reports the probability of the unlearned model's generations to be aligned with the original domain knowledge with increasing amount of guidance. Evaluating existing unlearning methods with our proposed stringent metrics for diffusion models reveals significant shortcomings in their ability to truly unlearn concepts. Source Code: https://respailab.github.io/unlearning-or-concealment

Continual learning and machine unlearning are crucial challenges in machine learning, typically addressed separately. Continual learning focuses on adapting to new knowledge while preserving past information, whereas unlearning involves selectively forgetting specific subsets of data. In this paper, we introduce a novel framework that jointly tackles both tasks by leveraging controlled knowledge distillation. Our approach enables efficient learning with minimal forgetting and effective targeted unlearning. By incorporating a fixed memory buffer, the system supports learning new concepts while retaining prior knowledge. The distillation process is carefully managed to ensure a balance between acquiring new information and forgetting specific data as needed. Experimental results on benchmark datasets show that our method matches or exceeds the performance of existing approaches in both continual learning and machine unlearning. This unified framework is the first to address both challenges simultaneously, paving the way for adaptable models capable of dynamic learning and forgetting while maintaining strong overall performance.

This paper considers one of the fundamental parallel-in-time methods for the solution of ordinary differential equations, Parareal, and extends it by adopting a neural network as a coarse propagator. We provide a theoretical analysis of the convergence properties of the proposed algorithm and show its effectiveness for several examples, including Lorenz and Burgers' equations. In our numerical simulations, we further specialize the underpinning neural architecture to Random Projection Neural Networks (RPNNs), a 2-layer neural network where the first layer weights are drawn at random rather than optimized. This restriction substantially increases the efficiency of fitting RPNN's weights in comparison to a standard feedforward network without negatively impacting the accuracy, as demonstrated in the SIR system example.

Unlearning methods for recommender systems (RS) have emerged to address privacy issues and concerns about legal compliance. However, evolving user preferences and content licensing issues still remain unaddressed. This is particularly true in case of multi-modal recommender systems (MMRS), which aim to accommodate the growing influence of multi-modal information on user preferences. Previous unlearning methods for RS are inapplicable to MMRS due to incompatibility of multi-modal user-item behavior data graph with the matrix based representation of RS. Partitioning based methods degrade recommendation performance and incur significant overhead costs during aggregation. This paper introduces MMRecUN, a new framework for multi-modal recommendation unlearning, which, to the best of our knowledge, is the first attempt in this direction. Given the trained recommendation model and marked forget data, we devise Reverse Bayesian Personalized Ranking (BPR) objective to force the model to forget it. MMRecUN employs both reverse and forward BPR loss mechanisms to selectively attenuate the impact of interactions within the forget set while concurrently reinforcing the significance of interactions within the retain set. Our experiments demonstrate that MMRecUN outperforms baseline methods across various unlearning requests when evaluated on benchmark multi-modal recommender datasets. MMRecUN achieves recall performance improvements of up to $\mathbf{49.85%}$ compared to the baseline methods. It is up to $\mathbf{1.3}\times$ faster than the \textsc{Gold} model, which is trained on retain data from scratch. MMRecUN offers advantages such as superior performance in removing target elements, preservation of performance for retained elements, and zero overhead costs in comparison to previous methods.

A majority of language technologies are tailored for a small number of high-resource languages, while relatively many low-resource languages are neglected. One such group, Creole languages, have long been marginalized in academic study, though their speakers could benefit from machine translation (MT). These languages are predominantly used in much of Latin America, Africa and the Caribbean. We present the largest cumulative dataset to date for Creole language MT, including 14.5M unique Creole sentences with parallel translations -- 11.6M of which we release publicly, and the largest bitexts gathered to date for 41 languages -- the first ever for 21. In addition, we provide MT models supporting all 41 Creole languages in 172 translation directions. Given our diverse dataset, we produce a model for Creole language MT exposed to more genre diversity than ever before, which outperforms a genre-specific Creole MT model on its own benchmark for 26 of 34 translation directions., Comment: NAACL 2024

Surface and strain engineering are among the cheaper ways to modulate structure property relations in materials. Due to their compositional flexibilities, MXenes, the family of two-dimensional materials, provide enough opportunity for surface engineering. In this work, we have explored the possibility of improving thermoelectric efficiency of MXenes through these routes. The Janus MXenes obtained by modifications of the transition metal constituents and the functional groups passivating their surfaces are considered as surface engineered materials on which bi-axial strain is applied in a systematic way. We find that in the three Janus compounds Zr$_{2}$COS, ZrHfO$_{2}$ and ZrHfCOS, tensile strain modifies the electronic and lattice thermoelectric parameters such that the thermoelectric efficiency can be maximised. A remarkable reduction in the lattice thermal conductivity due to increased anharmonicity and elevation in Seebeck coefficient are obtained by application of moderate tensile strain. With the help of first-principles electronic structure method and semi-classical Boltzmann transport theory we analyse the interplay of structural parameters, electronic and dynamical properties to understand the effects of strain and surface modifications on thermoelectric properties of these systems. Our detailed calculations and in depth analysis lead not only to the microscopic understanding of the influences of surface and strain engineering in these three systems, but also provide enough insights for adopting this approach and improve thermoelectric efficiencies in similar systems., Comment: Supplementary information is available on request

Strong-field ionization can induce electron motion in both the continuum and the valence shell of the parent ion. Here, we explore their interplay by studying laser-induced electron diffraction (LIED) patterns arising from interaction with the potentials of two-hole states of the xenon cation. The quantitative rescattering theory is used to calculate the corresponding photoelectron momentum distributions, providing evidence that the spin-orbit dynamics could be detected by LIED. We identify the contribution of these time-evolving hole states to the angular distribution of the rescattered electrons, particularly noting a distinct change along the backward scattering angles. We benchmark numerical results with experiments using ultrabroad and femtosecond laser pulses centered at \SI{3100}{nm}.

Quantifying the value of data within a machine learning workflow can play a pivotal role in making more strategic decisions in machine learning initiatives. The existing Shapley value based frameworks for data valuation in machine learning are computationally expensive as they require considerable amount of repeated training of the model to obtain the Shapley value. In this paper, we introduce an efficient data valuation framework EcoVal, to estimate the value of data for machine learning models in a fast and practical manner. Instead of directly working with individual data sample, we determine the value of a cluster of similar data points. This value is further propagated amongst all the member cluster points. We show that the overall value of the data can be determined by estimating the intrinsic and extrinsic value of each data. This is enabled by formulating the performance of a model as a \textit{production function}, a concept which is popularly used to estimate the amount of output based on factors like labor and capital in a traditional free economic market. We provide a formal proof of our valuation technique and elucidate the principles and mechanisms that enable its accelerated performance. We demonstrate the real-world applicability of our method by showcasing its effectiveness for both in-distribution and out-of-sample data. This work addresses one of the core challenges of efficient data valuation at scale in machine learning models. The code is available at \underline{https://github.com/respai-lab/ecoval}., Comment: KDD-2024

Despite ample evidence of their influences on the transport properties of two-dimensional solids, the interrelations of reduced symmetry, electronic and thermal transport, have rarely being discussed in the context of thermoelectric materials. With the motivation to design new thermoelectric materials with improved properties, we have addressed these by performing first-principles Density Functional Theory based calculations in conjunction with semi-classical Boltzmann transport theory on a number of compounds in the MXene family. The symmetry lowering in parent M$_{2}$CO$_{2}$ MXenes are done by replacing transition metal $M$ on one surface, resulting in Janus compounds MM$^{\prime}$CO$_{2}$. Our calculations show that the thermoelectric figure-of-merit can be improved significantly by such surface engineering. We discuss in detail, both qualitatively and quantitatively, the origin behind high thermoelectric parameters for these compounds. Our in-depth analysis shows that the modifications in the electronic band structures and degree of anharmonicity driven by the dispersions in the bond strengths due to lowering of symmetry, an artefact of surface engineering, are the factors behind the trends in the thermoelectric parameters of the MXenes considered. The results also substantiate that the compositional flexibility offered by the MXene family of compounds can generate complex interplay of symmetry, electronic structure, bond strengths and anharmonicity which can be exploited to engineer thermoelectric materials with improved properties., Comment: Supplementary information available on request

Euler's elastica is a classical model of flexible slender structures, relevant in many industrial applications. Static equilibrium equations can be derived via a variational principle. The accurate approximation of solutions of this problem can be challenging due to nonlinearity and constraints. We here present two neural network based approaches for the simulation of this Euler's elastica. Starting from a data set of solutions of the discretised static equilibria, we train the neural networks to produce solutions for unseen boundary conditions. We present a $\textit{discrete}$ approach learning discrete solutions from the discrete data. We then consider a $\textit{continuous}$ approach using the same training data set, but learning continuous solutions to the problem. We present numerical evidence that the proposed neural networks can effectively approximate configurations of the planar Euler's elastica for a range of different boundary conditions., Comment: v2: Improved the rigorousness of the experiments considering training, validation and test sets. Fixed language and notation

Graph Neural Networks (GNNs) have established themselves as a key component in addressing diverse graph-based tasks. Despite their notable successes, GNNs remain susceptible to input perturbations in the form of adversarial attacks. This paper introduces an innovative approach to fortify GNNs against adversarial perturbations through the lens of coupled dynamical systems. Our method introduces graph neural layers based on differential equations with contractive properties, which, as we show, improve the robustness of GNNs. A distinctive feature of the proposed approach is the simultaneous learned evolution of both the node features and the adjacency matrix, yielding an intrinsic enhancement of model robustness to perturbations in the input features and the connectivity of the graph. We mathematically derive the underpinnings of our novel architecture and provide theoretical insights to reason about its expected behavior. We demonstrate the efficacy of our method through numerous real-world benchmarks, reading on par or improved performance compared to existing methods., Comment: ECAI 2024

Graph unlearning has emerged as a pivotal method to delete information from a pre-trained graph neural network (GNN). One may delete nodes, a class of nodes, edges, or a class of edges. An unlearning method enables the GNN model to comply with data protection regulations (i.e., the right to be forgotten), adapt to evolving data distributions, and reduce the GPU-hours carbon footprint by avoiding repetitive retraining. Existing partitioning and aggregation-based methods have limitations due to their poor handling of local graph dependencies and additional overhead costs. More recently, GNNDelete offered a model-agnostic approach that alleviates some of these issues. Our work takes a novel approach to address these challenges in graph unlearning through knowledge distillation, as it distills to delete in GNN (D2DGN). It is a model-agnostic distillation framework where the complete graph knowledge is divided and marked for retention and deletion. It performs distillation with response-based soft targets and feature-based node embedding while minimizing KL divergence. The unlearned model effectively removes the influence of deleted graph elements while preserving knowledge about the retained graph elements. D2DGN surpasses the performance of existing methods when evaluated on various real-world graph datasets by up to $43.1\%$ (AUC) in edge and node unlearning tasks. Other notable advantages include better efficiency, better performance in removing target elements, preservation of performance for the retained elements, and zero overhead costs. Notably, our D2DGN surpasses the state-of-the-art GNNDelete in AUC by $2.4\%$, improves membership inference ratio by $+1.3$, requires $10.2\times10^6$ fewer FLOPs per forward pass and up to $\mathbf{3.2}\times$ faster.

A new variant of the superconducting fulleride $\mathrm{Rb_{3}C_{60}}$ is presented, with $\mathrm{^{3}He}$ atoms encapsulated in the $\mathrm{C_{60}}$ cages. The $\mathrm{^{3}He}$ nuclei act as sensitive NMR probes embedded in the material. The superconducting and normal states are characterised by $\mathrm{^{3}He}$ NMR. Evidence is found for co-existing vortex liquid and vortex solid phases below the superconducting transition temperature. A strong dependence of the spin-lattice relaxation time constant on spectral frequency is observed in the superconducting state, as revealed by two-dimensional NMR utilising an inverse Laplace transform. Surprisingly, this phenomenon persists, in attenuated form, at temperatures well above the superconducting transition., Comment: 20 pages, 15 figures

This paper presents a comprehensive technical overview of the Linac Coherent Light Source II (LCLS-II) photoinjector laser system, its first and foremost component. The LCLS-II photoinjector laser system serves as an upgrade to the original LCLS at SLAC National Accelerator Laboratory. This advanced laser system generates high-quality laser beams for LCLS-II, contributing to the instrument's unprecedented brightness, precision, and flexibility. Our discussion extends to the various subsystems that comprise the photoinjector, including the photocathode laser, laser heater, and beam transport systems. Lastly, we draw attention to the ongoing research and development infrastructure underway to enhance the functionality and efficiency of the LCLS-II, and similar X-ray free-electron laser facilities around the world, thereby contributing to the future of laser technology and its applications., Comment: Published by High Power Laser Science and Engineering

The NMR spectroscopy of spin-1/2 nuclei with low gyromagnetic ratio is challenging, due to the low NMR signal strength. Methodology for the rapid acquisition of $^{103}$Rh NMR parameters is demonstrated for the case of the rhodium formate "paddlewheel" complex $\mathrm{Rh_2(HCO_2)_4}$. A scheme is described for enhancing the $^{103}$Rh signal strength by polarization transfer from $^{1}$H nuclei and which also greatly reduces the interference from ringing artifacts, a common hurdle for the direct observation of low-$\gamma$ nuclei. The $^{103}$Rh relaxation time constants $T_1$ and $T_2$ are measured within 20 minutes using $^{1}$H-detected experiments. The field-dependence of the $^{103}$Rh $T_1$ is measured. The high-field relaxation is dominated by the chemical shift anisotropy (CSA) mechanism. The $^{103}$Rh shielding anisotropy is found to be very large: $|\Delta\sigma|=9900\pm540\mathrm{\,ppm}$. This estimate is compared with density functional theory calculations., Comment: submitted to JCP

Motivated by classical work on the numerical integration of ordinary differential equations we present a ResNet-styled neural network architecture that encodes non-expansive (1-Lipschitz) operators, as long as the spectral norms of the weights are appropriately constrained. This is to be contrasted with the ordinary ResNet architecture which, even if the spectral norms of the weights are constrained, has a Lipschitz constant that, in the worst case, grows exponentially with the depth of the network. Further analysis of the proposed architecture shows that the spectral norms of the weights can be further constrained to ensure that the network is an averaged operator, making it a natural candidate for a learned denoiser in Plug-and-Play algorithms. Using a novel adaptive way of enforcing the spectral norm constraints, we show that, even with these constraints, it is possible to train performant networks. The proposed architecture is applied to the problem of adversarially robust image classification, to image denoising, and finally to the inverse problem of deblurring., Comment: 34 pages, 6 figures. This is the accepted version of a paper published in Physica D: Nonlinear Phenomena

Investigation of structure-property relations in chemically and magnetically disordered materials can give rise to interesting physical phenomena. The potential of two-dimensional MXenes as electrodes in supercapacitor applications have been studied extensively. However, the role of chemical and magnetic disorder on their electrochemical parameters like the capacitance have not been explored yet. In this work, we have systematically addressed this for V$_{2-x}$Mn$_{x}$CO$_{2}$ MXene solid solutions with an analysis based upon results from first-principles electronic structure calculations. We find that the variations in the total capacitance over a voltage window depends upon the degree of chemical and magnetic disorder. In course of our investigation, we also found out that the magnetic structure on the surface can substantially influence the redox charge transfer, an yet unexplored phenomenon. A significantly large charge transfer and thus a large capacitance can be obtained by manipulating the chemical composition and the magnetic order of the surfaces.These findings can be useful in designing operational supercapacitor electrodes with magnetic constituents.

As supported by abundant experimental evidence, neural networks are state-of-the-art for many approximation tasks in high-dimensional spaces. Still, there is a lack of a rigorous theoretical understanding of what they can approximate, at which cost, and at which accuracy. One network architecture of practical use, especially for approximation tasks involving images, is (residual) convolutional networks. However, due to the locality of the linear operators involved in these networks, their analysis is more complicated than that of fully connected neural networks. This paper deals with approximation of time sequences where each observation is a matrix. We show that with relatively small networks, we can represent exactly a class of numerical discretizations of PDEs based on the method of lines. We constructively derive these results by exploiting the connections between discrete convolution and finite difference operators. Our network architecture is inspired by those typically adopted in the approximation of time sequences. We support our theoretical results with numerical experiments simulating the linear advection, heat, and Fisher equations.

The Divertor Tokamak Test (DTT) is a new experimental facility whose construction is starting in Frascati, Rome, Italy; its main goals are improving the understanding of plasma-wall interactions and supporting the development of ITER and DEMO. DTT will be equipped with a Neutral Beam Injector (NBI) based on negative deuterium ions, designed to inject 10 MW of power to the tokamak. A fundamental system for the good operations of the DTT NBI will be its Gas injection and Vacuum System (GVS). Indeed, the efficiency of the entire NBI strongly depends on the good performance of its GVS. The GVS for DTT NBI will be composed of two systems working in parallel: a grounded section connected to the main vacuum vessel, and a high voltage part connected to the ion source vessel and working at -510 kV voltage. The grounded part will feature a fore vacuum system (given by screw and roots pumps) plus a high vacuum system based on turbo-molecular pumps located on the side walls of the vessel and Non-Evaporable Getter (NEG) pumps located inside the vessel on the upper and lower surfaces. On the other hand, the high voltage part will feature a fore vacuum system (given by two compact screw pumps mounted on the external surface for the ion source vessel) plus a high vacuum system based on turbo-molecular pumps also located on the sidewalls of the ion source vessel. A dedicated deuterium gas injection will feed the process gas to the ion source and the neutralizer. This paper gives a description of the conceptual design of the GVS for DTT NBI, and of the procedure followed to optimize this system considering the operational requirements and the other constraints of the DTT NBI., Comment: 12 pages, 8 figures, presented at the SOFT 2022 conference

Automated human emotion recognition from facial expressions is a well-studied problem and still remains a very challenging task. Some efficient or accurate deep learning models have been presented in the literature. However, it is quite difficult to design a model that is both efficient and accurate at the same time. Moreover, identifying the minute feature variations in facial regions for both macro and micro-expressions requires expertise in network design. In this paper, we proposed to search for a highly efficient and robust neural architecture for both macro and micro-level facial expression recognition. To the best of our knowledge, this is the first attempt to design a NAS-based solution for both macro and micro-expression recognition. We produce lightweight models with a gradient-based architecture search algorithm. To maintain consistency between macro and micro-expressions, we utilize dynamic imaging and convert microexpression sequences into a single frame, preserving the spatiotemporal features in the facial regions. The EmoNAS has evaluated over 13 datasets (7 macro expression datasets: CK+, DISFA, MUG, ISED, OULU-VIS CASIA, FER2013, RAF-DB, and 6 micro-expression datasets: CASME-I, CASME-II, CAS(ME)2, SAMM, SMIC, MEGC2019 challenge). The proposed models outperform the existing state-of-the-art methods and perform very well in terms of speed and space complexity.

Applying very small purely radial strains on amorphous solids in radial geometry one observes elastic responses that break the radial symmetry. Without any plasticity involved, the responses indicate nonlinear mode coupling contributions even for minute strains. We show that these symmetry-breaking responses are due to disorder, typical to amorphous configurations. The symmetry breaking responses are quantitatively explained using the classical Michell solutions which are excited by mode coupling.