Search

Many hardware proposals have aimed to accelerate inference in AI workloads. Less attention has been paid to hardware acceleration of training, despite the enormous societal impact of rapid training of AI models. Physics-based computers, such as thermodynamic computers, offer an efficient means to solve key primitives in AI training algorithms. Optimizers that normally would be computationally out-of-reach (e.g., due to expensive matrix inversions) on digital hardware could be unlocked with physics-based hardware. In this work, we propose a scalable algorithm for employing thermodynamic computers to accelerate a popular second-order optimizer called Kronecker-factored approximate curvature (K-FAC). Our asymptotic complexity analysis predicts increasing advantage with our algorithm as $n$, the number of neurons per layer, increases. Numerical experiments show that even under significant quantization noise, the benefits of second-order optimization can be preserved. Finally, we predict substantial speedups for large-scale vision and graph problems based on realistic hardware characteristics., Comment: 17 pages, 5 figures

We analyze the phase transition between a symmetric metallic parent state and itinerant altermagnetic order. The underlying mechanism we reveal in our microscopic model of electrons on a Lieb lattice does not involve orbital ordering, but derives from sublattice interference., Comment: 7 pages, 3 figures, supplementary material

A fully Bayesian treatment of complicated predictive models (such as deep neural networks) would enable rigorous uncertainty quantification and the automation of higher-level tasks including model selection. However, the intractability of sampling Bayesian posteriors over many parameters inhibits the use of Bayesian methods where they are most needed. Thermodynamic computing has emerged as a paradigm for accelerating operations used in machine learning, such as matrix inversion, and is based on the mapping of Langevin equations to the dynamics of noisy physical systems. Hence, it is natural to consider the implementation of Langevin sampling algorithms on thermodynamic devices. In this work we propose electronic analog devices that sample from Bayesian posteriors by realizing Langevin dynamics physically. Circuit designs are given for sampling the posterior of a Gaussian-Gaussian model and for Bayesian logistic regression, and are validated by simulations. It is shown, under reasonable assumptions, that the Bayesian posteriors for these models can be sampled in time scaling with $\ln(d)$, where $d$ is dimension. For the Gaussian-Gaussian model, the energy cost is shown to scale with $ d \ln(d)$. These results highlight the potential for fast, energy-efficient Bayesian inference using thermodynamic computing., Comment: 20 pages, 8 figures

Despite the remarkable success of the Standard Model, the hierarchy and patterns of fermion masses and mixings remain a profound mystery. To address this, we propose a model employing the rank mechanism, where the originally massless quarks and leptons sequentially get masses. The third-generation masses originate from the seesaw mechanism at the tree level, while those of the second and first generations emerge from one-loop and two-loop radiative corrections, respectively, with a progressive increase in the rank of the mass matrix. This approach does not require new discrete or global symmetries. Unlike other theories of this type that require the introduction of additional scalars, we employ the double seesaw mechanism within a left-right symmetric framework, which allows us to realize this scenario solely through gauge interactions., Comment: 33 pages + references, 6 figures

Pre-trained language models demonstrate general intelligence and common sense, but long inputs quickly become a bottleneck for memorizing information at inference time. We resurface a simple method, Memorizing Transformers (Wu et al., 2022), that gives the model access to a bank of pre-computed memories. We show that it is possible to fix many of the shortcomings of the original method, such as the need for fine-tuning, by critically assessing how positional encodings should be updated for the keys and values retrieved. This intuitive method uses the model's own key/query system to select and attend to the most relevant memories at each generation step, rather than using external embeddings. We demonstrate the importance of external information being retrieved in a majority of decoder layers, contrary to previous work. We open source a new counterfactual long-range retrieval benchmark, and show that Extended Mind Transformers outperform today's state of the art by 6% on average.

Although theoretically compelling, Bayesian learning with modern machine learning models is computationally challenging since it requires approximating a high dimensional posterior distribution. In this work, we (i) introduce posteriors, an easily extensible PyTorch library hosting general-purpose implementations making Bayesian learning accessible and scalable to large data and parameter regimes; (ii) present a tempered framing of stochastic gradient Markov chain Monte Carlo, as implemented in posteriors, that transitions seamlessly into optimization and unveils a minor modification to deep ensembles to ensure they are asymptotically unbiased for the Bayesian posterior, and (iii) demonstrate and compare the utility of Bayesian approximations through experiments including an investigation into the cold posterior effect and applications with large language models.

The phase diagrams of quasi two-dimensional organic superconductors display a plethora of fundamental phenomena associated with strong electron correlations, such as unconventional superconductivity, metal-insulator transitions, frustrated magnetism and spin liquid behavior. We analyze a minimal model for these compounds, the Hubbard model on an anisotropic triangular lattice, using cutting-edge quantum embedding methods respecting the lattice symmetry. We demonstrate the existence of unconventional superconductivity by directly entering the symmetry-broken phase. We show that the crossover from the Fermi liquid metal to the Mott insulator is associated with the formation of a pseudogap. The predicted momentum-selective destruction of the Fermi surface into hot and cold regions provides motivation for further spectroscopic studies. Our results are in remarkable agreement with experimental phase diagrams of $\kappa$-BEDT organics., Comment: 9 pages, 3 figures; Supplemental Material: 8 pages, 10 figures

Intertwined charge and magnetic fluctuations in high-$T_\text{c}$ copper oxide superconductors (cuprates) are hypothesized to be a consequence of their correlated electronic nature. Among other observables, this is apparent in the doping dependence of incommensurate magnetic order, known as the Yamada relation (YR). We analyze the Hubbard model to challenge the universality of YR as a function of interaction strength $U$ through Kotliar-Ruckenstein slave-boson (SB) mean-field theory and truncated unity functional renormalization group (TUFRG). While TUFRG tends to lock in to a doping dependence of the incommensurate magnetic ordering vector obtained for the perturbative weak-coupling limit, SB not only exhibits an enhanced sensitivity upon a variation of $U$ from weak to strong coupling, but also shows good agreement with experimental data. It supports the placement of weakly hole-doped cuprates in the intermediate-to-strong coupling regime.

In recent years, several unsupervised cell segmentation methods have been presented, trying to omit the requirement of laborious pixel-level annotations for the training of a cell segmentation model. Most if not all of these methods handle the instance segmentation task by focusing on the detection of different cell instances ignoring their type. While such models prove adequate for certain tasks, like cell counting, other applications require the identification of each cell's type. In this paper, we present CellMixer, an innovative annotation-free approach for the semantic segmentation of heterogeneous cell populations. Our augmentation-based method enables the training of a segmentation model from image-level labels of homogeneous cell populations. Our results show that CellMixer can achieve competitive segmentation performance across multiple cell types and imaging modalities, demonstrating the method's scalability and potential for broader applications in medical imaging, cellular biology, and diagnostics., Comment: Medical Imaging Meets NeurIPS 2023

We investigate dynamical mean-field calculations of the three-band Emery model at the one- and two-particle level for material-realistic parameters of high-$T_c$ superconductors. Our study shows that even within dynamical mean-field theory, which accounts solely for temporal fluctuations, the intrinsic multi-orbital nature of the Emery model introduces effective non-local correlations. These correlations lead to a non-Curie-like temperature dependence of the magnetic susceptibility, consistent with nuclear magnetic resonance experiments in the pseudogap regime. By analyzing the temperature dependence of the uniform static spin susceptibility obtained by single-site and cluster dynamical mean-field theory, we find indications of emerging oxygen-copper singlet fluctuations, explicitly captured by the model. Despite correctly describing the hallmark of the pseudogap at the two-particle level, such as the drop in the Knight shift of nuclear magnetic resonance, dynamical mean-field theory fails to capture the spectral properties of the pseudogap., Comment: 15 pages, 7 figures

The recently proposed center-focused post-processing procedure [Phys. Rev. Research 2, 033476 (2020)] of cellular dynamical mean-field theory suggests that central sites of large impurity clusters are closer to the exact solution of the Hubbard model than the edge sites. In this paper, we systematically investigate results in the spirit of this center-focused scheme for several cluster sizes up to $8\times 8$ in and out of particle-hole symmetry. First we analyze the metal-insulator crossovers and transitions of the half-filled Hubbard model on a simple square lattice. We find that the critical interaction of the crossover is reduced with increasing cluster sizes and the critical temperature abruptly drops for the $4\times 4$ cluster. Second, for this cluster size, we apply the center-focused scheme to a system with more realistic tight-binding parameters, investigating its pseudogap regime as a function of temperature and doping, where we find doping dependent metal-insulator crossovers, Lifshitz transitions and a strongly renormalized Fermi-liquid regime. Additionally to diagnosing the real space origin of the suppressed antinodal spectral weight in the pseudogap regime, we can infer hints towards underlying charge ordering tendencies., Comment: 31 pages, 16 figures

The discovery of neutrino oscillations implies that neutrinos are massive and mixed, necessitating an extension of the Standard Model, which may require the introduction of non-standard neutrino interactions (NSI). We investigate the potential of a high-energy muon collider to probe such NSIs with muons, specifically focusing on muonic forces. By analyzing the monophoton signal from the process $\mu^+ \mu^- \rightarrow \nu \overline{\nu } \gamma$, we explore four-fermion contact interactions involving two muons and two neutrinos. Moreover, we examine minimal models that generate scalar and vector-mediated NSIs and study their phenomenology. Projected sensitivities for the strength of NSIs, $\epsilon_{\alpha \beta}^{\mu \mu}$, are presented at a 95$\%$ confidence level for a center-of-mass energy of 3 TeV and integrated luminosities of $\mathcal{L} = 1$ and $10~\mathrm{ab}^{-1}$, showcasing the complementarity between a muon collider and other experimental probes., Comment: 4 figures, 6 pages + references

Continuous ultrasonic atomization in a closed chamber is expected to generate a mist with an equilibrium droplet concentration and size distribution. Such a mist of microdroplets with controllable mist density has been used for Aerosol Jet printing in the fabrication of a variety of additively manufactured microscale devices. Despite many unique capabilities demonstrated with the Aerosol Jet printing technology, its ultrasonic atomization behavior appears to be rather sensitive to the ink properties with gaps in our understanding of the fundamental physics underlying its operation. In this work, we investigate some basic mechanisms in the Aerosol Jet ultrasonic atomizer with a lumped-parameter kinetic coagulation model for highly concentrated mist. To mitigate the difficulty with unavailable knowledge about the complex turbulent flow inside the atomizer chamber, we present results for several orders of magnitude of the turbulent energy dissipation rates in order to examine a range of possibilities. The same approach is taken for analyzing the scavenging effect of the swirling bulk liquid. Our results also demonstrate the theoretical possibility for achieving a mist saturation condition where the mist output from the atomizer can become insensitive to process variables. As observed in experiments, such a saturated mist is highly desirable for Aerosol Jet printing with maximized and well-controlled throughput in additive manufacturing., Comment: 18 pages, 9 figures

We study magnetic and charge order in the electron-doped high-$T_c$ cuprates based on the one-band Hubbard model with onsite ($U$) and nearest-neighbor $(V)$ interactions. To investigate the interplay between the orders, we employ the Kotliar-Ruckenstein slave-boson method and analyze fluctuations descending from an antiferromagnetic parent state. Our analysis reveals incommensurate charge order whose ordering vector matches the doping-dependence of resonant inelastic x-ray scattering (RIXS) measurements in Nd$_{2-x}$Ce$_x$CuO$_4$ (NCCO). From our calculations of paramagnon dispersion as well as dynamical charge and spin structure factors, we reproduce all qualitative features of the RIXS signal., Comment: 17 pages, 11 figures, more condensed version

The ability to control the properties of twisted bilayer transition metal dichalcogenides in situ makes them an ideal platform for investigating the interplay of strong correlations and geometric frustration. Of particular interest are the low energy scales, which make it possible to experimentally access both temperature and magnetic fields that are of the order of the bandwidth or the correlation scale. In this manuscript we analyze the moir\'e Hubbard model, believed to describe the low energy physics of an important subclass of the twisted bilayer compounds. We establish its magnetic and the metal-insulator phase diagram for the full range of magnetic fields up to the fully spin polarized state. We find a rich phase diagram including fully and partially polarized insulating and metallic phases of which we determine the interplay of magnetic order, Zeeman-field, and metallicity, and make connection to recent experiments., Comment: 7+10 pages, 4+9 figures

The topological classification of electronic band structures is based on symmetry properties of Bloch eigenstates of single-particle Hamiltonians. In parallel, topological field theory has opened the doors to the formulation and characterization of non-trivial phases of matter driven by strong electron-electron interaction. Even though important examples of topological Mott insulators have been constructed, the relevance of the underlying non-interacting band topology to the physics of the Mott phase has remained unexplored. Here, we show that the momentum structure of the Green's function zeros defining the ``Luttinger surface" provides a topological characterization of the Mott phase related, in the simplest description, to the one of the single-particle electronic dispersion. Considerations on the zeros lead to the prediction of new phenomena: a topological Mott insulator with an inverted gap for the bulk zeros must possess gapless zeros at the boundary, which behave as a form of ``topological antimatter'' annihilating conventional edge states. Placing band and Mott topological insulators in contact produces distinctive observable signatures at the interface, revealing the otherwise spectroscopically elusive Green's function zeros., Comment: 7 pages, 4 figures; supplementary material available at https://doi.org/10.1038/s41467-023-42773-7; final published version

We show that vector-like quarks in the fundamental or higher-dimensional representations of QCD can generate the electro-weak scale in a phenomenologically viable way by chiral symmetry breaking condensates. The thereby generated scales are determined by numerically solving the Dyson-Schwinger equation and these scales are sizable, because they grow with the hard vector-like mass. Communicating such a scale to the Standard Model via a conformally invariant scalar sector can dynamically generate the electro-weak scale without a naturalness problem, because all non-dynamical mass scales are protected by chiral symmetry. We present a minimal setup which requires only a new neutral scalar with mass not too far above the electro-weak scale, as well as vector-like quarks at the (multi-)TeV scale. Both are consistent with current bounds and are attractive for future experimental searches at the LHC and future colliders. Depending on the hypercharge of the vector-like quarks, hadrons made of them are color-neutral bound states which would be interesting Dark Matter candidates., Comment: 19 pages, 9 figures, 1 table

In the Standard Model, Yukawa couplings parametrize the fermion masses and mixing angles with the exception of neutrino masses. The hierarchies and apparent regularities among the quark and lepton masses are, however, otherwise a mystery. We propose a new class of models having vector-like fermions that can potentially address this problem and provide a new mechanism for fermion mass generation. The masses of the third and second generations of quarks and leptons arise at tree level via the seesaw mechanism from new physics at moderately higher scales, while loop corrections produce the masses for the first generation. This mechanism has a number of interesting and testable consequences. Among them are unavoidable flavor-violating signals at the upcoming experiments and the fact that neutrinos have naturally only Dirac masses., Comment: 25 pages + references, 5 figures, 5 tables, matches published version

We study the magnetic and spectral properties of a single-band Hubbard model for the infinite-layer nickelate compound LaNiO$_2$. As spatial correlations turn out to be the key ingredient for understanding its physics, we use two complementary extensions of the dynamical mean-field theory to take them into account: the cellular dynamical mean-field theory and the dynamical vertex approximation. Additionally to the systematic analysis of the doping dependence of the non-Curie-Weiss behavior of the uniform magnetic susceptibility, we provide insight into its relation to the formation of a pseudogap regime by the calculation of the one-particle spectral function and the magnetic correlation length. The latter is of the order of a few lattice spacings when the pseudogap opens, indicating a strong-coupling pseudogap formation in analogy to cuprates., Comment: 8 pages, 5 figures, submitted to the Condensed Matter Physics section in Frontiers in Physics for the research topic "Advances in Superconducting Infinite-Layer and Related Nickelates"

We report a comprehensive study of magnetic correlations in LaNiO$_{2}$, a parent compound of the recently discovered family of infinite-layer (IL) nickelate superconductors, using multiple experimental and theoretical methods. Our specific heat, muon-spin rotation ($\mu$SR), and magnetic susceptibility measurements on polycrystalline LaNiO$_{2}$ show that long-range magnetic order remains absent down to 2 K. Nevertheless, we detect residual entropy in the low-temperature specific heat, which is compatible with a model fit that includes paramagnon excitations. The $\mu$SR and low-field static and dynamic magnetic susceptibility measurements indicate the presence of short-range magnetic correlations and glassy spin dynamics, which we attribute to local oxygen non-stoichiometry in the average infinite-layer crystal structure. This glassy behavior can be suppressed in strong external fields, allowing us to extract the intrinsic paramagnetic susceptibility. Remarkably, we find that the intrinsic susceptibility shows non-Curie-Weiss behavior at high temperatures, in analogy to doped cuprates that possess robust non-local spin fluctuations. The distinct temperature dependence of the intrinsic susceptibility of LaNiO$_{2}$ can be theoretically understood by a multi-method study of the single-band Hubbard model in which we apply complementary cutting-edge quantum many-body techniques (dynamical mean-field theory, cellular dynamical mean-field theory and the dynamical vertex approximation) to investigate the influence of both short- and long-ranged correlations. Our results suggest a profound analogy between the magnetic correlations in parent (undoped) IL nickelates and doped cuprates., Comment: 18 pages, 14 figures