Search

Predicting high-fidelity ground motions for future earthquakes is crucial for seismic hazard assessment and infrastructure resilience. Conventional empirical simulations suffer from sparse sensor distribution and geographically localized earthquake locations, while physics-based methods are computationally intensive and require accurate representations of Earth structures and earthquake sources. We propose a novel artificial intelligence (AI) simulator, Conditional Generative Modeling for Ground Motion (CGM-GM), to synthesize high-frequency and spatially continuous earthquake ground motion waveforms. CGM-GM leverages earthquake magnitudes and geographic coordinates of earthquakes and sensors as inputs, learning complex wave physics and Earth heterogeneities, without explicit physics constraints. This is achieved through a probabilistic autoencoder that captures latent distributions in the time-frequency domain and variational sequential models for prior and posterior distributions. We evaluate the performance of CGM-GM using small-magnitude earthquake records from the San Francisco Bay Area, a region with high seismic risks. CGM-GM demonstrates a strong potential for outperforming a state-of-the-art non-ergodic empirical ground motion model and shows great promise in seismology and beyond.

All-optical switching (AOS) involves manipulating magnetization using only pulsed laser, presenting a promising approach for next-generation magnetic recording devices. NiCo2O4 (NCO) thin films, a rare-earth-free ferrimagnetic oxide, exhibit a high Curie temperature and strong perpendicular magnetic anisotropy. This study demonstrates AOS in NCO thin films at room temperature using long-duration laser pulses and high repetition rates. Unlike previous findings, the AOS phenomena we report here are helicity-dependent and observable with an ultrashort pulsed laser. Consequently, two distinct types of AOS can be observed in a single NCO thin film, contingent on the characteristics of the laser pulses and temperature., Comment: 5 pages, 4 figures

In this paper, we rethink sparse lexical representations for image retrieval. By utilizing multi-modal large language models (M-LLMs) that support visual prompting, we can extract image features and convert them into textual data, enabling us to utilize efficient sparse retrieval algorithms employed in natural language processing for image retrieval tasks. To assist the LLM in extracting image features, we apply data augmentation techniques for key expansion and analyze the impact with a metric for relevance between images and textual data. We empirically show the superior precision and recall performance of our image retrieval method compared to conventional vision-language model-based methods on the MS-COCO, PASCAL VOC, and NUS-WIDE datasets in a keyword-based image retrieval scenario, where keywords serve as search queries. We also demonstrate that the retrieval performance can be improved by iteratively incorporating keywords into search queries., Comment: Accepted to ECCV 2024 Workshops: 2nd Workshop on Traditional Computer Vision in the Age of Deep Learning (TradiCV)

Recently, representation learning with contrastive learning algorithms has been successfully applied to challenging unlabeled datasets. However, these methods are unable to distinguish important features from unimportant ones under simply unsupervised settings, and definitions of importance vary according to the type of downstream task or analysis goal, such as the identification of objects or backgrounds. In this paper, we focus on unsupervised image clustering as the downstream task and propose a representation learning method that enhances features critical to the clustering task. We extend a clustering-friendly contrastive learning method and incorporate a contrastive analysis approach, which utilizes a reference dataset to separate important features from unimportant ones, into the design of loss functions. Conducting an experimental evaluation of image clustering for three datasets with characteristic backgrounds, we show that for all datasets, our method achieves higher clustering scores compared with conventional contrastive analysis and deep clustering methods., Comment: 12 pages, 6 figures, 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD2024)

Spoken dialogue plays a crucial role in human-AI interactions, necessitating dialogue-oriented spoken language models (SLMs). To develop versatile SLMs, large-scale and diverse speech datasets are essential. Additionally, to ensure hiqh-quality speech generation, the data must be spontaneous like in-wild data and must be acoustically clean with noise removed. Despite the critical need, no open-source corpus meeting all these criteria has been available. This study addresses this gap by constructing and releasing a large-scale spoken dialogue corpus, named Japanese Corpus for Human-AI Talks (J-CHAT), which is publicly accessible. Furthermore, this paper presents a language-independent method for corpus construction and describes experiments on dialogue generation using SLMs trained on J-CHAT. Experimental results indicate that the collected data from multiple domains by our method improve the naturalness and meaningfulness of dialogue generation., Comment: 8 pages, 6 figures

The Simons Observatory (SO) is a ground-based cosmic microwave background experiment currently being deployed to Cerro Toco in the Atacama Desert of Chile. The initial deployment of SO, consisting of three 0.46m-diameter small-aperture telescopes and one 6m-primary large-aperture telescope, will field over 60,000 transition-edge sensors that will observe at frequencies between 30 GHz and 280 GHz. SO will read out its detectors using Superconducting Quantum Interference Device (SQUID) microwave-frequency multiplexing $\mu$mux, a form of frequency division multiplexing where an RF-SQUID couples each TES bolometer to a superconducting resonator tuned to a unique frequency. Resonator frequencies are spaced roughly every 2 MHz between 4 and 6 GHz, allowing for multiplexing factors on the order of 1000. One challenge of $\mu$mux is matching each tracked resonator with its corresponding physical detector. Variations in resonator fabrication, and frequency shifts between cooldowns caused by trapped flux can cause the measured resonance frequencies to deviate significantly from their designed values. In this study, we introduce a method for pairing measured and designed resonators by constructing a bipartite graph based on the two resonator sets, and assigning edge weights based on measured resonator and detector properties such as resonance frequency, detector pointing, and assigned bias lines. Finding the minimum-cost matching for a given set of edge weights is a well-studied problem that can be solved very quickly, and this matching tells us the best assignment of measured resonators to designed detectors for our input parameters. We will present results based on the first on-sky measurements from SAT1, the first SO MF small-aperture telescope.

A novel information entropy of turbulence systems with multiple field quantities is formulated. Inspired by quantum mechanics, the von Neumann entropy (vNE) and the entanglement entropy(EE) are derived from a density matrix for the turbulence state in terms of the multi-field singular value decomposition (MFSVD). Applying the information-theoretic entropy analyses to spatio-temporal dynamics in turbulent plasmas with phase-transition-like behavior, we discover a new nontrivial transition threshold regarding the vNE, which significantly deviates from the transition threshold of the field energy considered in the conventional approaches. These findings provide us with new classifications of the turbulence state in terms of combined energy and information. It is also shown that the EE for nonlinear interactions in turbulence simultaneously describes not only the information for the strength of nonlinear mode couplings but also the direction of net energy transfer., Comment: 13 page, 5 figures

We propose a scar model to reproduce how double-strand breaks occur in the telomeric DNA damaged by the effect of the $\beta$-decay of tritium to helium. In this scar model, the two hydrogens bonded to the 5$^\prime$ carbon connecting the pent saccharides and phosphate are removed. Molecular dynamics simulations using the reactive force field are carried out for 10 cases for the telomeric DNA consisting of 16 base pairs (32 nucleotides). It results in double-strand breaks (DSBs) being observed for structures with more than 24 scars. For 16 scar cases, only single-strand breaks (SSB) are observed. Moreover, in the case of $\left\{16, 0 \right\}$ and $\left\{ 0, 16 \right\},$ where only one of the strands had scars, SSB occurs only in the scarred strand. Secondly, in the $\left\{16, 8 \right\}$ and $\left\{ 8, 16 \right\}$ cases, DSBs occurs. Therefore, we conclude that the following conditions are necessary for DSBs:(i) Scars must occur on both the L and R strands. (ii) A large number of scars (24 or more) must occur in close proximity to each other.

Coupon allocation drives customer purchases and boosts revenue. However, it presents a fundamental trade-off between exploiting the current optimal policy to maximize immediate revenue and exploring alternative policies to collect data for future policy improvement via off-policy evaluation (OPE). While online A/B testing can validate new policies, it risks compromising short-term revenue. Conversely, relying solely on an exploitative policy hinders the ability to reliably estimate and enhance future policies. To balance this trade-off, we propose a novel approach that combines a model-based revenue maximization policy and a randomized exploration policy for data collection. Our framework enables flexibly adjusting the mixture ratio between these two policies to optimize the balance between short-term revenue and future policy improvement. We formulate the problem of determining the optimal mixture ratio between a model-based revenue maximization policy and a randomized exploration policy for data collection. We empirically verified the effectiveness of the proposed mixed policy using both synthetic and real-world data. Our main contributions are: (1) Demonstrating a mixed policy combining deterministic and probabilistic policies, flexibly adjusting the data collection vs. revenue trade-off. (2) Formulating the optimal mixture ratio problem as multi-objective optimization, enabling quantitative evaluation of this trade-off. By optimizing the mixture ratio, businesses can maximize revenue while ensuring reliable future OPE and policy improvement. This framework is applicable in any context where the exploration-exploitation trade-off is relevant.

The impact of quantization on the overall performance of deep learning models is a well-studied problem. However, understanding and mitigating its effects on a more fine-grained level is still lacking, especially for harder tasks such as object detection with both classification and regression objectives. This work defines the performance for a subset of task-critical categories, i.e. the critical-category performance, as a crucial yet largely overlooked fine-grained objective for detection tasks. We analyze the impact of quantization at the category-level granularity, and propose methods to improve performance for the critical categories. Specifically, we find that certain critical categories have a higher sensitivity to quantization, and are prone to overfitting after quantization-aware training (QAT). To explain this, we provide theoretical and empirical links between their performance gaps and the corresponding loss landscapes with the Fisher information framework. Using this evidence, we apply a Fisher-aware mixed-precision quantization scheme, and a Fisher-trace regularization for the QAT on the critical-category loss landscape. The proposed methods improve critical-category metrics of the quantized transformer-based DETR detectors. They are even more significant in case of larger models and higher number of classes where the overfitting becomes more severe. For example, our methods lead to 10.4% and 14.5% mAP gains for, correspondingly, 4-bit DETR-R50 and Deformable DETR on the most impacted critical classes in the COCO Panoptic dataset., Comment: Poster presentation at the 2nd Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@ICML 2024)

Large earthquakes can be destructive and quickly wreak havoc on a landscape. To mitigate immediate threats, early warning systems have been developed to alert residents, emergency responders, and critical infrastructure operators seconds to a minute before seismic waves arrive. These warnings provide time to take precautions and prevent damage. The success of these systems relies on fast, accurate predictions of ground motion intensities, which is challenging due to the complex physics of earthquakes, wave propagation, and their intricate spatial and temporal interactions. To improve early warning, we propose a novel AI-enabled framework, WaveCastNet, for forecasting ground motions from large earthquakes. WaveCastNet integrates a novel convolutional Long Expressive Memory (ConvLEM) model into a sequence to sequence (seq2seq) forecasting framework to model long-term dependencies and multi-scale patterns in both space and time. WaveCastNet, which shares weights across spatial and temporal dimensions, requires fewer parameters compared to more resource-intensive models like transformers and thus, in turn, reduces inference times. Importantly, WaveCastNet also generalizes better than transformer-based models to different seismic scenarios, including to more rare and critical situations with higher magnitude earthquakes. Our results using simulated data from the San Francisco Bay Area demonstrate the capability to rapidly predict the intensity and timing of destructive ground motions. Importantly, our proposed approach does not require estimating earthquake magnitudes and epicenters, which are prone to errors using conventional approaches; nor does it require empirical ground motion models, which fail to capture strongly heterogeneous wave propagation effects.

The Simons Observatory (SO) is a Cosmic Microwave Background experiment located in the Atacama Desert in Chile. SO consists of three small aperture telescopes (SATs) and one large aperture telescope (LAT) with a total of 60,000 detectors in six frequency bands. As an observatory, SO encompasses hundreds of hardware components simultaneously running at different readout rates, all separate from its 60,000 detectors on-sky and their metadata. We provide an overview of commissioning SO's data acquisition software system for SAT-MF1, the first SAT deployed to the Atacama site. Additionally, we share insights from deploying data access software for all four telescopes, detailing how performance limitations affected data loading and quality investigations, which led to site-compatible software improvements., Comment: 15 pages, 9 figures, 3 tables. Presented at SPIE Astronomical Telescopes + Instrumentation 2024 Second submission: fix typos

The Simons Observatory (SO) is a cosmic microwave background (CMB) observatory consisting of three small aperture telescopes and one large aperture telescope. SO is located in the Atacama Desert in Chile at an elevation of 5180m. Distributed among the four telescopes are over 60,000 transition-edge sensor (TES) bolometers across six spectral bands centered between 27 and 280 GHz. A large collection of ancillary hardware devices which produce lower rate `housekeeping' data are used to support the detector data collection. We developed a distributed control system, which we call the observatory control system (ocs), to coordinate data collection among all systems within the observatory. ocs is a core component of the deployed site software, interfacing with all on-site hardware. Alongside ocs we utilize a combination of internally and externally developed open source projects to enable remote monitoring, data management, observation coordination, and data processing. Deployment of a majority of the software is done using Docker containers. The deployment of software packages is partially done via automated Ansible scripts, utilizing a GitOps based approach for updating infrastructure on site. We describe an overview of the software and computing systems deployed within SO, including how those systems are deployed and interact with each other. We also discuss the timing distribution system and its configuration as well as lessons learned during the deployment process and where we plan to make future improvements., Comment: 19 pages, 4 figures

Normalizing flow-based generative models have been widely used in applications where the exact density estimation is of major importance. Recent research proposes numerous methods to improve their expressivity. However, conditioning on a context is largely overlooked area in the bijective flow research. Conventional conditioning with the vector concatenation is limited to only a few flow types. More importantly, this approach cannot support a practical setup where a set of context-conditioned (specialist) models are trained with the fixed pretrained general-knowledge (generalist) model. We propose ContextFlow++ approach to overcome these limitations using an additive conditioning with explicit generalist-specialist knowledge decoupling. Furthermore, we support discrete contexts by the proposed mixed-variable architecture with context encoders. Particularly, our context encoder for discrete variables is a surjective flow from which the context-conditioned continuous variables are sampled. Our experiments on rotated MNIST-R, corrupted CIFAR-10C, real-world ATM predictive maintenance and SMAP unsupervised anomaly detection benchmarks show that the proposed ContextFlow++ offers faster stable training and achieves higher performance metrics. Our code is publicly available at https://github.com/gudovskiy/contextflow., Comment: Accepted to UAI 2024. Preprint

The Simons Observatory (SO) is a cosmic microwave background (CMB) survey experiment that includes small-aperture telescopes (SATs) observing from an altitude of 5,200 m in the Atacama Desert in Chile. The SO SATs will cover six spectral bands between 27 and 280 GHz to search for primordial B-modes to a sensitivity of $\sigma(r)=0.002$, with quantified systematic errors well below this value. Each SAT is a self-contained cryogenic telescope with a 35$^\circ$ field of view, 42 cm diameter optical aperture, 40 K half-wave plate, 1 K refractive optics, and $<0.1$ K focal plane that holds $>12,000$ TES detectors. We describe the nominal design of the SATs and present details about the integration and testing for one operating at 93 and 145 GHz.

Recovering quantum information from a noisy quantum system is one of the central challenges in quantum information science and fundamental physics. The key to this goal is explicitly constructing a decoder. In this paper, we provide two explicit decoding quantum circuits that are both capable of recovering quantum information when a decoupling condition is satisfied, i.e., when quantum information is in principle recoverable. The decoders are constructed by using the fixed-point amplitude amplification algorithm based on the quantum singular value transformation, which significantly extends an approach by Yoshida and Kitaev in a specific noise model to general situations. We also show that the proposed decoding circuits reduce the computational cost compared to a previously known explicit decoder. Our constructions not only show an intriguing intersection between decoders and quantum algorithms but also reveal the power of an algorithmic approach to recovering quantum information., Comment: 23 pages, 12 figures, 2 tables

We present Ir $4f$ core-level hard-x-ray photoemission spectroscopy (HAXPES) experiments conducted across antiferromagnetic (AFM) ordering transition in Ruddlesden-Popper iridates Sr$_2$IrO$_4$ and Sr$_3$Ir$_2$O$_7$. The Ir $4f$ spectra exhibit distinct changes between the AFM and paramagnetic (PM) phases, with the spectral difference $I_\text{PM}-I_\text{AFM}$ showing a contrasting behavior in the two compounds. By employing computational simulations using the local-density approximation combined with the dynamical mean-field theory method, we elucidate that $I_\text{PM}-I_\text{AFM}$ primary reflects the Slater or Mott-Hubbard character of the AFM insulating state rather than material specific details. This sensitivity to fine low-energy electronic structure arises from the dependence of charge-transfer responses to the sudden creation of a localized core hole on both metal-insulator transitions and long-range AFM ordering. Our result broadens the applications of core-level HAXPES as a tool for characterization of electronic structure., Comment: Main: 7 pages, 3 figures ; SM: 4 pages, 4 figures

We present UTDUSS, the UTokyo-SaruLab system submitted to Interspeech2024 Speech Processing Using Discrete Speech Unit Challenge. The challenge focuses on using discrete speech unit learned from large speech corpora for some tasks. We submitted our UTDUSS system to two text-to-speech tracks: Vocoder and Acoustic+Vocoder. Our system incorporates neural audio codec (NAC) pre-trained on only speech corpora, which makes the learned codec represent rich acoustic features that are necessary for high-fidelity speech reconstruction. For the acoustic+vocoder track, we trained an acoustic model based on Transformer encoder-decoder that predicted the pre-trained NAC tokens from text input. We describe our strategies to build these models, such as data selection, downsampling, and hyper-parameter tuning. Our system ranked in second and first for the Vocoder and Acoustic+Vocoder tracks, respectively., Comment: 5 pages, 3 figures

In text-to-speech synthesis, the ability to control voice characteristics is vital for various applications. By leveraging thriving text prompt-based generation techniques, it should be possible to enhance the nuanced control of voice characteristics. While previous research has explored the prompt-based manipulation of voice characteristics, most studies have used pre-recorded speech, which limits the diversity of voice characteristics available. Thus, we aim to address this gap by creating a novel corpus and developing a model for prompt-based manipulation of voice characteristics in text-to-speech synthesis, facilitating a broader range of voice characteristics. Specifically, we propose a method to build a sizable corpus pairing voice characteristics descriptions with corresponding speech samples. This involves automatically gathering voice-related speech data from the Internet, ensuring its quality, and manually annotating it using crowdsourcing. We implement this method with Japanese language data and analyze the results to validate its effectiveness. Subsequently, we propose a construction method of the model to retrieve speech from voice characteristics descriptions based on a contrastive learning method. We train the model using not only conservative contrastive learning but also feature prediction learning to predict quantitative speech features corresponding to voice characteristics. We evaluate the model performance via experiments with the corpus we constructed above., Comment: Submitted to IEEE/ACM Transactions on Audio, Speech, and Language Processing. arXiv admin note: text overlap with arXiv:2309.13509

The $d^5$ electron configurations under the crystal field, spin-orbit coupling, and Coulomb interaction give rise to a plethora of profound ground states. Ruthenium perovskite oxides exhibit a number of unconventional properties yet the Ru$^{4+}$ state ($4d^4$) is usually stable in these materials. In this regard, Ru$^{3+}$ ions in perovskite materials are expected to be a mesmerising playground of $4d^5$ electron configurations. Here, we report measurements of x-ray photoemission spectroscopy on recently synthesized perovskite ruthenium oxide thin films, LaRuO$_3$ and NdRuO$_3$, whose valence state of the ruthenium ions is trivalent. We discuss correlation and spin-orbit effects from the valence-band spectra, in particular an additional peak structure around 3-5 eV, reminiscent of the so-called 3 eV peak observed in Sr$_2$RuO$_4$. Moreover, we find that the core-level spectra of these materials are quantitatively different from those in other ruthenates which possess Ru$^{4+}$ ions, e.g., SrRuO$_3$. We therefore argue that the core level spectra of LaRuO$_3$ and NdRuO$_3$ are peculiar to the Ru$^{3+}$ states., Comment: 5 pages, 4 figures

There has been a growing interest in non-Hermitian physics. One of its main goals is to engineer dissipation and to explore ensuing functionality. In magnonics, the effect of dissipation due to local damping on magnon transport has been explored. However, the effects of non-local damping on the magnonic analog of the Josephson effect remain missing, despite that non-local damping is inevitable and has been playing a central role in magnonics. Here, we uncover theoretically that a surprisingly rich dynamics can emerge in magnetic junctions due to intrinsic non-local damping, using analytical and numerical methods. In particular, under microwave pumping, we show that coherent spin precession in the right and left insulating ferromagnet (FM) of the junction becomes synchronized by non-local damping and thereby a magnonic analog of the $\varphi$ Josephson junction emerges, where $\varphi$ stands here for the relative precession phase of right and left FM in the stationary limit. Remarkably, $\varphi$ decreases monotonically from $ \pi$ to $\pi/2$ as the magnon-magnon interaction, arising from spin anisotropies, increases. Moreover, we also find a magnonic diode effect giving rise to rectification of magnon currents. Our predictions are readily testable with current device and measurement technologies at room temperatures., Comment: 5+7 pages, 3+3 figures

While branching network structures abound in nature, their objective analysis is more difficult than expected because existing quantitative methods often rely on the subjective judgment of branch structures. This problem is particularly pronounced when dealing with images comprising discrete particles. Here we propose an objective framework for quantitative analysis of branching networks by introducing the mathematical definitions for internal and external structures based on topological data analysis, specifically, persistent homology. We compare persistence diagrams constructed from images with and without plots on the convex hull. The unchanged points in the two diagrams are the internal structures and the difference between the two diagrams is the external structures. We construct a mathematical theory for our method and show that the internal structures have a monotonicity relationship with respect to the plots on the convex hull, while the external structures do not. This is the phenomenon related to the resolution of the image. Our method can be applied to a wide range of branch structures in biology, enabling objective analysis of numbers, spatial distributions, sizes, and more. Additionally, our method has the potential to be combined with other tools in topological data analysis, such as the generalized persistence landscape.

The delay Lotka-Volterra and delay Toda lattice equations are delay-differential extensions of the well-known soliton equations, the Lotka-Volterra and Toda lattice equations, respectively. This paper investigates integrable properties of the delay Lotka-Volterra and delay Toda lattice equations, and study the relationships to the already known delay Painleve equations. First, Backlund transformations, Lax pairs and an infinite number of conserved quantities of these delay soliton equations are constructed. Then, applying spatial 2-periodic reductions to them, we show the known delay Painleve equations are derived. Using these reductions, we construct the N-soliton-type determinant solutions of the autonomous versions of delay Painleve equations, and the Casorati determinant solution of a higher order analogue of the discrete Painleve II equation., Comment: 32 pages

We present an evaluation of 32-bit POSIT arithmetic through its implementation as accelerators on FPGAs and GPUs. POSIT, a floating-point number format, adaptively changes the size of its fractional part. We developed hardware designs for FPGAs and software for GPUs to accelerate linear algebra operations using Posit(32,2) arithmetic. Our FPGA- and GPU-based accelerators in Posit(32,2) arithmetic significantly accelerated the Cholesky and LU decomposition algorithms for dense matrices. In terms of numerical accuracy, Posit(32,2) arithmetic is approximately 0.5 - 1.0 digits more accurate than the standard 32-bit format, especially when the norm of the elements of the input matrix is close to 1. Evaluating power consumption, we observed that the power efficiency of the accelerators ranged between 0.043 - 0.076 Gflops/watts for the LU decomposition in Posit(32,2) arithmetic. The power efficiency of the latest GPUs as accelerators of Posit(32,2) arithmetic is better than that of the evaluated FPGA chip., Comment: 11 pages, 8 figures; Published in HPCAsia '24: Proceedings of the International Conference on High Performance Computing in Asia-Pacific Region

We study the interplay between the Kondo and superconducting (SC) proximity effects, taking place in a triangular triple quantum dot (TTQD) connected to one normal and two SC leads. This system shows various quantum phases. Without the SC leads, the lowest two states that belong to the different spin sectors, $S=0$ and $S=1/2$, become energetically very close to each other near half filling. The singlet one is a Kondo-screened state by conduction electrons from the normal lead, and the doublet one is a resonating valence bond state with unpaired free spin which remains unscreened. Furthermore, when one additional electron enters the TTQD, the ground state becomes a doublet in which the $S=1$ local moment due to the Nagaoka ferromagnetism is partially screened by conduction electrons.The Cooper pairs penetrating into the TTQD from the SC leads reconstruct the wavefunctions and vary the phase boundaries between these quantum states in the parameter space. We calculate ground-state phase diagrams using the numerical renormalization group, and show that the SC proximity effect induces a reentrant transition in-between the three- and four-electron fillings., Comment: 7 pages, 3 figures

Finetuning a pretrained vision model (PVM) is a common technique for learning downstream vision tasks. However, the conventional finetuning process with randomly sampled data points results in diminished training efficiency. To address this drawback, we propose a novel approach, Vision-language Collaborative Active Finetuning (VeCAF). With the emerging availability of labels and natural language annotations of images through web-scale crawling or controlled generation, VeCAF makes use of these information to perform parametric data selection for PVM finetuning. VeCAF incorporates the finetuning objective to select significant data points that effectively guide the PVM towards faster convergence to meet the performance goal. This process is assisted by the inherent semantic richness of the text embedding space which we use to augment image features. Furthermore, the flexibility of text-domain augmentation allows VeCAF to handle out-of-distribution scenarios without external data. Extensive experiments show the leading performance and high computational efficiency of VeCAF that is superior to baselines in both in-distribution and out-of-distribution image classification tasks. On ImageNet, VeCAF uses up to 3.3x less training batches to reach the target performance compared to full finetuning, and achieves an accuracy improvement of 2.7% over the state-of-the-art active finetuning method with the same number of batches., Comment: 13 pages

The physics of driven collective modes in quantum materials underpin a number of striking non-equilibrium functional responses, which include enhanced magnetism, ferroelectricity and superconductivity. However, the coherent coupling between multiple modes at once are difficult to capture by single-pump probe (one-dimensional) spectroscopy, and often remain poorly understood. One example is phonon-mediated amplification of Josephson plasmons in YBa$_2$Cu$_3$O$_{6+x}$, in which at least three normal modes of the solid are coherently mixed as a source of enhanced superconductivity. Here, we go beyond previous pump-probe experiments in this system and acquire two-dimensional frequency maps using pairs of mutually delayed, carrier envelope phase stable mid-infrared pump pulses, combined with measurements of the time-modulated second-order nonlinear optical susceptibility. We find that the driven zone-center phonons amplify coherent pairs of opposite-momentum Josephson plasma polaritons, generating a squeezed state of interlayer phase fluctuations. The squeezed state is a potentially important ingredient in the microscopic physics of photo-induced superconductivity in this and other materials., Comment: 20 pages, 7 figures

The Mixture-of-Experts (MoE) approach has demonstrated outstanding scalability in multi-task learning including low-level upstream tasks such as concurrent removal of multiple adverse weather effects. However, the conventional MoE architecture with parallel Feed Forward Network (FFN) experts leads to significant parameter and computational overheads that hinder its efficient deployment. In addition, the naive MoE linear router is suboptimal in assigning task-specific features to multiple experts which limits its further scalability. In this work, we propose an efficient MoE architecture with weight sharing across the experts. Inspired by the idea of linear feature modulation (FM), our architecture implicitly instantiates multiple experts via learnable activation modulations on a single shared expert block. The proposed Feature Modulated Expert (FME) serves as a building block for the novel Mixture-of-Feature-Modulation-Experts (MoFME) architecture, which can scale up the number of experts with low overhead. We further propose an Uncertainty-aware Router (UaR) to assign task-specific features to different FM modules with well-calibrated weights. This enables MoFME to effectively learn diverse expert functions for multiple tasks. The conducted experiments on the multi-deweather task show that our MoFME outperforms the baselines in the image restoration quality by 0.1-0.2 dB and achieves SOTA-compatible performance while saving more than 72% of parameters and 39% inference time over the conventional MoE counterpart. Experiments on the downstream segmentation and classification tasks further demonstrate the generalizability of MoFME to real open-world applications., Comment: aaai2024

Uncertainty estimation is crucial for machine learning models to detect out-of-distribution (OOD) inputs. However, the conventional discriminative deep learning classifiers produce uncalibrated closed-set predictions for OOD data. A more robust classifiers with the uncertainty estimation typically require a potentially unavailable OOD dataset for outlier exposure training, or a considerable amount of additional memory and compute to build ensemble models. In this work, we improve on uncertainty estimation without extra OOD data or additional inference costs using an alternative Split-Ensemble method. Specifically, we propose a novel subtask-splitting ensemble training objective, where a common multiclass classification task is split into several complementary subtasks. Then, each subtask's training data can be considered as OOD to the other subtasks. Diverse submodels can therefore be trained on each subtask with OOD-aware objectives. The subtask-splitting objective enables us to share low-level features across submodels to avoid parameter and computational overheads. In particular, we build a tree-like Split-Ensemble architecture by performing iterative splitting and pruning from a shared backbone model, where each branch serves as a submodel corresponding to a subtask. This leads to improved accuracy and uncertainty estimation across submodels under a fixed ensemble computation budget. Empirical study with ResNet-18 backbone shows Split-Ensemble, without additional computation cost, improves accuracy over a single model by 0.8%, 1.8%, and 25.5% on CIFAR-10, CIFAR-100, and Tiny-ImageNet, respectively. OOD detection for the same backbone and in-distribution datasets surpasses a single model baseline by, correspondingly, 2.2%, 8.1%, and 29.6% mean AUROC., Comment: ICML2024. Project website is available at https://antonioo-c.github.io/projects/split-ensemble