Search

The advent of Large Language Models (LLMs) have shown promise in various creative domains, including culinary arts. However, many LLMs still struggle to deliver the desired level of culinary creativity, especially when tasked with adapting recipes to meet specific cultural requirements. This study focuses on cuisine transfer-applying elements of one cuisine to another-to assess LLMs' culinary creativity. We employ a diverse set of LLMs to generate and evaluate culturally adapted recipes, comparing their evaluations against LLM and human judgments. We introduce the ASH (authenticity, sensitivity, harmony) benchmark to evaluate LLMs' recipe generation abilities in the cuisine transfer task, assessing their cultural accuracy and creativity in the culinary domain. Our findings reveal crucial insights into both generative and evaluative capabilities of LLMs in the culinary domain, highlighting strengths and limitations in understanding and applying cultural nuances in recipe creation. The code and dataset used in this project will be openly available in \url{http://github.com/dmis-lab/CulinaryASH}.

While textless Spoken Language Models (SLMs) have shown potential in end-to-end speech-to-speech modeling, they still lag behind text-based Large Language Models (LLMs) in terms of semantic coherence and relevance. This work introduces the Align-SLM framework, which leverages preference optimization inspired by Reinforcement Learning with AI Feedback (RLAIF) to enhance the semantic understanding of SLMs. Our approach generates multiple speech continuations from a given prompt and uses semantic metrics to create preference data for Direct Preference Optimization (DPO). We evaluate the framework using ZeroSpeech 2021 benchmarks for lexical and syntactic modeling, the spoken version of the StoryCloze dataset for semantic coherence, and other speech generation metrics, including the GPT4-o score and human evaluation. Experimental results show that our method achieves state-of-the-art performance for SLMs on most benchmarks, highlighting the importance of preference optimization to improve the semantics of SLMs.

Chemical vapour deposition enables large-domain growth of ideal graphene, yet many applications of graphene require the controlled inclusion of specific defects. We present a one-step chemical vapour deposition procedure aimed at retaining the precursor topology when incorporated into the grown carbonaceous film. When azupyrene, the molecular analogue of the Stone-Wales defect in graphene, is used as a precursor, carbonaceous monolayers with a range of morphologies are produced as a function of the copper substrate growth temperature. The higher the substrate temperature during deposition, the closer the resulting monolayer is to ideal graphene. Analysis, with a set of complementary materials characterisation techniques, reveals morphological changes closely correlated with changes in the atomic adsorption heights, network topology, and concentration of 5-/7-membered carbon rings. The engineered defective carbon monolayers can be transferred to different substrates, potentially enabling applications in nanoelectronics, sensorics, and catalysis., Comment: manuscript and SI provided

Background: Cone-beam computed tomography (CBCT) plays a crucial role in image-guided radiotherapy, but artifacts and noise make them unsuitable for accurate dose calculation. Artificial intelligence methods have shown promise in enhancing CBCT quality to produce synthetic CT (sCT) images. However, existing methods either produce images of suboptimal quality or incur excessive time costs, failing to satisfy clinical practice standards. Methods and materials: We propose a novel hybrid conditional latent diffusion model for efficient and accurate CBCT-to-CT synthesis, named HC$^3$L-Diff. We employ the Unified Feature Encoder (UFE) to compress images into a low-dimensional latent space, thereby optimizing computational efficiency. Beyond the use of CBCT images, we propose integrating its high-frequency knowledge as a hybrid condition to guide the diffusion model in generating sCT images with preserved structural details. This high-frequency information is captured using our designed High-Frequency Extractor (HFE). During inference, we utilize denoising diffusion implicit model to facilitate rapid sampling. We construct a new in-house prostate dataset with paired CBCT and CT to validate the effectiveness of our method. Result: Extensive experimental results demonstrate that our approach outperforms state-of-the-art methods in terms of sCT quality and generation efficiency. Moreover, our medical physicist conducts the dosimetric evaluations to validate the benefit of our method in practical dose calculation, achieving a remarkable 93.8% gamma passing rate with a 2%/2mm criterion, superior to other methods. Conclusion: The proposed HC$^3$L-Diff can efficiently achieve high-quality CBCT-to-CT synthesis in only over 2 mins per patient. Its promising performance in dose calculation shows great potential for enhancing real-world adaptive radiotherapy., Comment: 13 pages, 5 figures

We study methods for efficiently aligning large language models (LLMs) with human preferences given budgeted online feedback. We first formulate the LLM alignment problem in the frame of contextual dueling bandits. This formulation, subsuming recent paradigms such as online RLHF and online DPO, inherently quests for sample-efficient algorithms that incorporate online active exploration. Leveraging insights from bandit theory, we introduce a unified algorithm based on Thompson sampling and highlight its applications in two distinct LLM alignment scenarios. The practical agent that efficiently implements this algorithm, named SEA (Sample-Efficient Alignment), is empirically validated through extensive experiments across three model scales (1B, 2.8B, 6.9B) and three preference learning algorithms (DPO, IPO, SLiC). The results demonstrate that SEA achieves highly sample-efficient alignment with oracle's preferences, outperforming recent active exploration methods for LLMs. Additionally, we release the implementation of SEA together with an efficient codebase designed for online alignment of LLMs, aiming to accelerate future research in this field.

Multi-scene absolute pose regression addresses the demand for fast and memory-efficient camera pose estimation across various real-world environments. Nowadays, transformer-based model has been devised to regress the camera pose directly in multi-scenes. Despite its potential, transformer encoders are underutilized due to the collapsed self-attention map, having low representation capacity. This work highlights the problem and investigates it from a new perspective: distortion of query-key embedding space. Based on the statistical analysis, we reveal that queries and keys are mapped in completely different spaces while only a few keys are blended into the query region. This leads to the collapse of the self-attention map as all queries are considered similar to those few keys. Therefore, we propose simple but effective solutions to activate self-attention. Concretely, we present an auxiliary loss that aligns queries and keys, preventing the distortion of query-key space and encouraging the model to find global relations by self-attention. In addition, the fixed sinusoidal positional encoding is adopted instead of undertrained learnable one to reflect appropriate positional clues into the inputs of self-attention. As a result, our approach resolves the aforementioned problem effectively, thus outperforming existing methods in both outdoor and indoor scenes., Comment: Accepted to NeurIPS 2024

Despite extensive microscale studies, the macroscopic mechanical properties of monolayer graphene remain underexplored. Here, we report the Young's modulus ($E$ = 1.11 $\pm$ 0.04 TPa), tensile strength (${\sigma}$ = 27.40 $\pm$ 4.36 GPa), and failure strain (${\epsilon}_f$ = 6.01 $\pm$ 0.92 %) of centimeter-scale single-crystal monolayer graphene (SCG) 'dog bone' samples with edges aligned along the zigzag (zz) direction, supported by an ultra-thin polymer (polycarbonate) film. For samples with edges along the armchair (ac) direction, we obtain $E$ = 1.01 $\pm$ 0.10 TPa, ${\sigma}$ = 20.21 $\pm$ 3.22 GPa, ${\epsilon}_f$ = 3.69 $\pm$ 0.38 %, and for chiral samples whose edges were between zz and ac, we obtain $E$= 0.75 $\pm$ 0.12 TPa, ${\sigma}$ = 23.56 $\pm$ 3.42 GPa, and ${\epsilon}_f$ = 4.53 $\pm$ 0.40 %. The SCG is grown on single crystal Cu(111) foils by chemical vapor deposition (CVD). We used a home-built 'float-on-water' (FOW) tensile testing system for tensile loading measurements that also enabled in situ crack observation. The quantized fracture mechanics (QFM) analysis predicts an edge defect size from several to tens of nanometers based on chirality and notch angle. Through Weibull analysis and given that the fatal defects are confined on the edges of macroscale samples, we projected strength ranging from 13.67 to 18.43 GPa for an A4-size SCG according to their chirality. The exceptional mechanical performance of macroscale single crystal graphene (SCG) paves the way for its widespread use in a very wide variety of applications., Comment: 76 pages, 44 figures

We introduce Constrained Human-AI Cooperation (CHAIC), an inclusive embodied social intelligence challenge designed to test social perception and cooperation in embodied agents. In CHAIC, the goal is for an embodied agent equipped with egocentric observations to assist a human who may be operating under physical constraints -- e.g., unable to reach high places or confined to a wheelchair -- in performing common household or outdoor tasks as efficiently as possible. To achieve this, a successful helper must: (1) infer the human's intents and constraints by following the human and observing their behaviors (social perception), and (2) make a cooperative plan tailored to the human partner to solve the task as quickly as possible, working together as a team (cooperative planning). To benchmark this challenge, we create four new agents with real physical constraints and eight long-horizon tasks featuring both indoor and outdoor scenes with various constraints, emergency events, and potential risks. We benchmark planning- and learning-based baselines on the challenge and introduce a new method that leverages large language models and behavior modeling. Empirical evaluations demonstrate the effectiveness of our benchmark in enabling systematic assessment of key aspects of machine social intelligence. Our benchmark and code are publicly available at https://github.com/UMass-Foundation-Model/CHAIC., Comment: NeurIPS 2024 Dataset and Benchmark Track. The first two authors contributed equally. Project Website at https://vis-www.cs.umass.edu/CHAIC/

Recently, transformer-based techniques incorporating superpoints have become prevalent in 3D instance segmentation. However, they often encounter an over-segmentation problem, especially noticeable with large objects. Additionally, unreliable mask predictions stemming from superpoint mask prediction further compound this issue. To address these challenges, we propose a novel framework called MSTA3D. It leverages multi-scale feature representation and introduces a twin-attention mechanism to effectively capture them. Furthermore, MSTA3D integrates a box query with a box regularizer, offering a complementary spatial constraint alongside semantic queries. Experimental evaluations on ScanNetV2, ScanNet200 and S3DIS datasets demonstrate that our approach surpasses state-of-the-art 3D instance segmentation methods., Comment: 14 pages, 9 figures, 7 tables, conference

In the months leading up to political elections in the United States, forecasts are widespread and take on multiple forms, including projections of what party will win the popular vote, state ratings, and predictions of vote margins at the state level. It can be challenging to evaluate how accuracy changes in the lead up to Election Day or to put probabilistic forecasts into historical context. Moreover, forecasts differ between analysts, highlighting the many choices in the forecasting process. With this as motivation, here we take a more comprehensive view and begin to unpack some of the choices involved in election forecasting. Building on a prior compartmental model of election dynamics, we present the forecasts of this model across months, years, and types of race. By gathering together monthly forecasts of presidential, senatorial, and gubernatorial races from 2004--2022, we provide a larger-scale perspective and discuss how treating polling data in different ways affects forecast accuracy. We conclude with our 2024 election forecasts (upcoming at the time of writing)., Comment: For our 2024 forecasts, see: https://c-r-u-d.gitlab.io/2024/. Our code is available at: https://gitlab.com/alexandriavolkening/forecasting-elections-using-compartmental-models-2

Models trained with empirical risk minimization (ERM) are prone to be biased towards spurious correlations between target labels and bias attributes, which leads to poor performance on data groups lacking spurious correlations. It is particularly challenging to address this problem when access to bias labels is not permitted. To mitigate the effect of spurious correlations without bias labels, we first introduce a novel training objective designed to robustly enhance model performance across all data samples, irrespective of the presence of spurious correlations. From this objective, we then derive a debiasing method, Disagreement Probability based Resampling for debiasing (DPR), which does not require bias labels. DPR leverages the disagreement between the target label and the prediction of a biased model to identify bias-conflicting samples-those without spurious correlations-and upsamples them according to the disagreement probability. Empirical evaluations on multiple benchmarks demonstrate that DPR achieves state-of-the-art performance over existing baselines that do not use bias labels. Furthermore, we provide a theoretical analysis that details how DPR reduces dependency on spurious correlations.

We propose an entropy-stable conservative flux form neural network (CFN) that integrates classical numerical conservation laws into a data-driven framework using the entropy-stable, second-order, and non-oscillatory Kurganov-Tadmor (KT) scheme. The proposed entropy-stable CFN uses slope limiting as a denoising mechanism, ensuring accurate predictions in both noisy and sparse observation environments, as well as in both smooth and discontinuous regions. Numerical experiments demonstrate that the entropy-stable CFN achieves both stability and conservation while maintaining accuracy over extended time domains. Furthermore, it successfully predicts shock propagation speeds in long-term simulations, {\it without} oracle knowledge of later-time profiles in the training data., Comment: 27 pages, 18 figures

We present the Atacama Large Millimeter/submillimeter Array Band 7 observations of the CO (J=3-2) line emission of the protostellar system HH 212 at $\sim$24 au spatial resolution and compare them to those of the SiO (J=8-7) and SO (J=8-7) line emission reported in the literature. We find that the CO line traces four distinct regions: (1) an outer outflow shell, (2) a rotating wind region between the SiO and CO shells, (3) the shocked and wide-angle inner X-wind inside a SiO shell, and (4) the jet. The origin of the CO outer outflow shell could be associated with the entrained material of the envelope, or an extended disk wind. The rotating wind, which is shocked, is launched from a radius of 9-15 au, slightly exterior to that of the previously detected SO shell, which marks the boundary where the wide-angle X-wind is interacting with and shocking the disk wind. Additionally, the SO is found to be mixed with the CO emission within the thick and extended rotating wind region. The large scale CO shocked wind coexists with the SO emission near the upper portion of the inner shocked region converged on top of the inner SiO knots. The CO jet is traced by a chain of knots with roughly equal interval, exhibiting quasi-periodicity, as reported in other jets in the literature., Comment: 18 pages, 12 figures, 2 tables. Accepted by ApJ

Reconstructing unknown external source functions is an important perception capability for a large range of robotics domains including manipulation, aerial, and underwater robotics. In this work, we propose a Physics-Informed Neural Network (PINN [1]) based approach for solving the inverse source problems in robotics, jointly identifying unknown source functions and the complete state of a system given partial and noisy observations. Our approach demonstrates several advantages over prior works (Finite Element Methods (FEM) and data-driven approaches): it offers flexibility in integrating diverse constraints and boundary conditions; eliminates the need for complex discretizations (e.g., meshing); easily accommodates gradients from real measurements; and does not limit performance based on the diversity and quality of training data. We validate our method across three simulation and real-world scenarios involving up to 4th order partial differential equations (PDEs), constraints such as Signorini and Dirichlet, and various regression losses including Chamfer distance and L2 norm.

This work presents a design under uncertainty approach for a char combustion process in a limited-data setting, where simulations of the fluid-solid coupled system are computationally expensive. We integrate a polynomial dimensional decomposition (PDD) surrogate model into the design optimization and induce computational efficiency in three key areas. First, we transform the input random variables to have fixed probability measures, which eliminates the need to recalculate the PDD's basis functions associated with these probability quantities. Second, using the limited available data from a physics-based high-fidelity solver, we estimate the PDD coefficients via sparsity-promoting diffeomorphic modulation under observable response preserving homotopy regression. Third, we propose a single-pass surrogate model training process that avoids the need to generate new training data and update the PDD coefficients during the derivative-free optimization process. The results provide insights for optimizing process parameters to ensure consistently high energy production from char combustion., Comment: 19 pages, 6 figures

In optimal experimental design, the objective is to select a limited set of experiments that maximizes information about unknown model parameters based on factor levels. This work addresses the generalized D-optimal design problem, allowing for nonlinear relationships in factor levels. We develop scalable algorithms suitable for cases where the number of candidate experiments grows exponentially with the factor dimension, focusing on both first- and second-order models under design constraints. Particularly, our approach integrates convex relaxation with pricing-based local search techniques, which can provide upper bounds and performance guarantees. Unlike traditional local search methods, such as the ``Fedorov exchange" and its variants, our method effectively accommodates arbitrary side constraints in the design space. Furthermore, it yields both a feasible solution and an upper bound on the optimal value derived from the convex relaxation. Numerical results highlight the efficiency and scalability of our algorithms, demonstrating superior performance compared to the state-of-the-art commercial software, JMP

Far-infrared (FIR) surveys are critical to probing the co-evolution of black holes and galaxies, since of order half the light from accreting black holes and active star formation is emitted in the rest-frame infrared over $0.5\lesssim z \lesssim 10$. For deep fields with areas of 1 deg$^2$ or less, like the legacy surveys GOODS, COSMOS, and CANDELS, source crowding means that sub-arcsecond resolution is essential. In this paper we show with a simulation of the FIR sky that measurements made with a small telescope (2 m) at low angular resolution yield biased results, and we demonstrate the scientific value of a space mission that would offer sub-arcsecond resolution. We envisage a facility that would provide high-resolution imaging and spectroscopy over the wavelength range $25-400\,\mu$m, and we present predictions for an extragalactic survey covering $0.5\,\hbox{deg}^2$. Such a survey is expected to detect tens of thousands of star-forming galaxies and thousands of Active Galactic Nuclei (AGN), in multiple FIR lines (e.g. [CII], [OI], [CI]) and continuum. At the longest wavelengths (200-400$\,\mu$m), it would probe beyond the reionization epoch, up to $z\sim 7$-8. A combination of spectral resolution, line sensitivity, and broad spectral coverage would allow us to learn about the physical conditions (temperature, density, metallicity) characterizing the interstellar medium of galaxies over the past $\sim 12$ billion years and to investigate galaxy-AGN co-evolution., Comment: 23 pages, 13 figures, 1 table, accepted for publication in ApJ

Multiply connected electronic networks threaded by flux tubes have been proposed as a platform for adiabatic quantum transport and topological states. Multi-terminal Josephson junction (MTJJ) has been suggested as a pathway to realize this concept. Yet, the manifestations of topology in MTJJ remain open for experimental study. Here, we investigated the artificial topological band structure of three-terminal graphene Josephson junctions. Employing tunnelling spectroscopy and magnetic flux gates, we captured the tomography of the Andreev bound state (ABS) energy spectrum as a function of two independent phase differences. This ABS spectrum exhibits the topological transition from gapped to gapless states, akin to the band structure of nodal-line semimetals. Our results show the potential of graphene-based MTJJs for engineering band topologies in higher dimensions.

Sodium-ion batteries are emerging as promising alternatives to lithium-ion batteries due to the abundance of sodium resources. Na3V2(PO4)2F3 (NVPF), a cathode material for sodium ion batteries, is attracting attention from its rate capability and high working voltage, but its low discharge capacity is one of the challenges. In this work, we aim to design a high-capacity NASICON-type cathode of sodium-ion battery by discovering element combinations that can stabilize the sodium excess phase in NVPFs. For the efficient discovery of element combinations, we propose a Bayesian optimization-based algorithm for chemical composition discovery. Specifically, we propose an element mapping technique to solve the limitation of Bayesian optimization in discovering chemical composition. By constructing a chemical space applicable to Bayesian optimization through element mapping and optimizing the constructed chemical space, we found optimal binary element combinations. This work not only offers insights into designing high-capacity cathodes, but also demonstrates the efficacy of the proposed algorithm in data-driven materials design., Comment: 20 pages, 5 figures

Measuring nanoscale local temperatures, particularly in vertically integrated and multi-component systems, remains challenging. Spectroscopic techniques like X-ray absorption and core-loss electron energy-loss spectroscopy (EELS) are sensitive to lattice temperature, but understanding thermal effects is nontrivial. This work explores the potential for nanoscale and element-specific core-loss thermometry by comparing the Si L2,3 edge's temperature-dependent redshift against plasmon energy expansion thermometry (PEET) in a scanning TEM. Using density functional theory (DFT), time-dependent DFT, and the Bethe-Salpeter equation, we ab initio model both the Si L2,3 and plasmon redshift. We find that the core-loss redshift is due to bandgap reduction from electron-phonon renormalization. Our results indicate that despite lower core-loss signal intensity and thus accuracy compared to PEET, core-loss thermometry still has important advantages. Specifically, we show that the Varshni equation easily interprets the core-loss redshift, which avoids plasmon spectral convolution for PEET in complex materials. We also find that core-loss thermometry is more accurate than PEET at modeling thermal lattice expansion unless the temperature-dependent effective mass and dielectric constant are known. Furthermore, core-loss thermometry has the potential to measure nanoscale heating in multi-component materials and stacked interfaces with elemental specificity at length scales smaller than the plasmon's wavefunction., Comment: 33 pages, 5 main text figures, 1 table

This paper introduces a new approach to enhance the robustness of humanoid walking under strong perturbations, such as substantial pushes. Effective recovery from external disturbances requires bipedal robots to dynamically adjust their stepping strategies, including footstep positions and timing. Unlike most advanced walking controllers that restrict footstep locations to a predefined convex region, substantially limiting recoverable disturbances, our method leverages reinforcement learning to dynamically adjust the permissible footstep region, expanding it to a larger, effectively non-convex area and allowing cross-over stepping, which is crucial for counteracting large lateral pushes. Additionally, our method adapts footstep timing in real time to further extend the range of recoverable disturbances. Based on these adjustments, feasible footstep positions and DCM trajectory are planned by solving a QP. Finally, we employ a DCM controller and an inverse dynamics whole-body control framework to ensure the robot effectively follows the trajectory.

Designing public spaces requires balancing the interests of diverse stakeholders within a constrained physical and institutional space. Designers usually approach these problems through participatory methods but struggle to incorporate diverse perspectives into design outputs. The growing capabilities of image-generative artificial intelligence (IGAI) could support participatory design. Prior work in leveraging IGAI's capabilities in design has focused on augmenting the experience and performance of individual creators. We study how IGAI could facilitate participatory processes when designing public spaces, a complex collaborative task. We conducted workshops and IGAI-mediated interviews in a real-world participatory process to upgrade a park in Los Angeles. We found (1) a shift from focusing on accuracy to fostering richer conversations as the desirable outcome of adopting IGAI in participatory design, (2) that IGAI promoted more space-aware conversations, and (3) that IGAI-mediated conversations are subject to the abilities of the facilitators in managing the interaction between themselves, the AI, and stakeholders. We contribute by discussing practical implications for using IGAI in participatory design, including success metrics, relevant skills, and asymmetries between designers and stakeholders. We finish by proposing a series of open research questions., Comment: Forthcoming in the Proceedings of the 2025 Conference on Computer Supported Cooperative Work and Social Computing (CSCW)

Controlling the effects of photoexcited polarons in transition metal oxides can enable the long timescale charge separation necessary for renewable energy applications as well as controlling new quantum phases through dynamically tunable electron-phonon coupling. In previously studied transition metal oxides, polaron formation is facilitated by a photoexcited ligand-to-metal charge transfer (LMCT). When the polaron is formed, oxygen atoms move away from iron centers, which increases carrier localization at the metal center and decreases charge hopping. Studies of yttrium iron garnet and erbium iron oxide have suggested that strong electron and spin correlations can modulate photoexcited polaron formation. To understand the interplay between strong spin and electronic correlations in highly polar materials, we studied gadolinium iron oxide (GdFeO3), which selectively forms photoexcited polarons through an Fe-O-Fe superexchange interaction. Excitation-wavelength-dependent transient extreme ultraviolet (XUV) spectroscopy selectively excites LMCT and metal-to-metal charge transfer transitions (MMCT). The LMCT transition suppresses photoexcited polaron formation due to dominant Hubbard interactions, while MMCT transitions result in photoexcited polaron formation within ~373+/-137 fs due to enhanced superexchange interactions. Ab initio theory demonstrates that both electron and hole polarons localize on iron centers following MMCT. In addition to understanding how strong electronic and spin correlations can control strong electron-phonon coupling, these experiments separately measure electron and hole polaron interactions on neighboring metal centers for the first time, providing insight into a large range of charge-transfer and Mott-Hubbard insulators., Comment: 29 pages, 14 figures

Pediatric sleep is an important but often overlooked area in health informatics. We present PedSleepMAE, a generative model that fully leverages multimodal pediatric sleep signals including multichannel EEGs, respiratory signals, EOGs and EMG. This masked autoencoder-based model performs comparably to supervised learning models in sleep scoring and in the detection of apnea, hypopnea, EEG arousal and oxygen desaturation. Its embeddings are also shown to capture subtle differences in sleep signals coming from a rare genetic disorder. Furthermore, PedSleepMAE generates realistic signals that can be used for sleep segment retrieval, outlier detection, and missing channel imputation. This is the first general-purpose generative model trained on multiple types of pediatric sleep signals.

As Large Language Models (LLMs) are increasingly deployed in specialized domains with continuously evolving knowledge, the need for timely and precise knowledge injection has become essential. Fine-tuning with paraphrased data is a common approach to enhance knowledge injection, yet it faces two significant challenges: high computational costs due to repetitive external model usage and limited sample diversity. To this end, we introduce LaPael, a latent-level paraphrasing method that applies input-dependent noise to early LLM layers. This approach enables diverse and semantically consistent augmentations directly within the model. Furthermore, it eliminates the recurring costs of paraphrase generation for each knowledge update. Our extensive experiments on question-answering benchmarks demonstrate that LaPael improves knowledge injection over standard fine-tuning and existing noise-based approaches. Additionally, combining LaPael with data-level paraphrasing further enhances performance., Comment: NeurIPS 2024

The recent segmentation foundation model, Segment Anything Model (SAM), exhibits strong zero-shot segmentation capabilities, but it falls short in generating fine-grained precise masks. To address this limitation, we propose a novel zero-shot image matting model, called ZIM, with two key contributions: First, we develop a label converter that transforms segmentation labels into detailed matte labels, constructing the new SA1B-Matte dataset without costly manual annotations. Training SAM with this dataset enables it to generate precise matte masks while maintaining its zero-shot capability. Second, we design the zero-shot matting model equipped with a hierarchical pixel decoder to enhance mask representation, along with a prompt-aware masked attention mechanism to improve performance by enabling the model to focus on regions specified by visual prompts. We evaluate ZIM using the newly introduced MicroMat-3K test set, which contains high-quality micro-level matte labels. Experimental results show that ZIM outperforms existing methods in fine-grained mask generation and zero-shot generalization. Furthermore, we demonstrate the versatility of ZIM in various downstream tasks requiring precise masks, such as image inpainting and 3D NeRF. Our contributions provide a robust foundation for advancing zero-shot matting and its downstream applications across a wide range of computer vision tasks. The code is available at \url{https://github.com/naver-ai/ZIM}., Comment: preprint (21 pages, 16 figures, and 8 tables)

The Vela pulsar (J0835-4510) is known to exhibit variations in Faraday rotation and dispersion on multi-decade timescales due to the changing sightline through the surrounding Vela supernova remnant and the Gum Nebula. Until now, variations in Faraday rotation towards Vela have not been studied on timescales less than around a decade. We present the results of a high-cadence observing campaign carried out with the Aperture Array Verification System 2 (AAVS2), a prototype SKA-Low station, which received a significant bandwidth upgrade in 2022. We collected observations of the Vela pulsar and PSR J0630-2834 (a nearby pulsar located outside the Gum Nebula), spanning $\sim 1\,\mathrm{yr}$ and $\sim 0.3\,\mathrm{yr}$ respectively, and searched for linear trends in the rotation measure (RM) as a function of time. We do not detect any significant trends on this timescale ($\sim$months) for either pulsar, but the constraints could be greatly improved with more accurate ionospheric models. For the Vela pulsar, the combination of our data and historical data from the published literature have enabled us to model long-term correlated trends in RM and dispersion measure (DM) over the past two decades. We detect a change in DM of $\sim 0.3\,\mathrm{cm}^{-3}\,\mathrm{pc}$ which corresponds to a change in electron density of $\sim 10^5\,\mathrm{cm}^{-3}$ on a transverse length scale of $\sim$1-2 au. The apparent magnetic field strength in the time-varying region changes from $240^{+30}_{-20}\,\mu\mathrm{G}$ to $-6.2^{+0.7}_{-0.9}\,\mu\mathrm{G}$ over the time span of the data set. As well as providing an important validation of polarimetry, this work highlights the pulsar monitoring capabilities of SKA-Low stations, and the niche science opportunities they offer for high-precision polarimetry and probing the microstructure of the magneto-ionic interstellar medium., Comment: 16 pages, 9 figures, 2 tables, published in PASA

Diffusion models have demonstrated remarkable success in various domains, including molecular generation. However, conditional molecular generation remains a fundamental challenge due to an intrinsic trade-off between targeting specific chemical properties and generating meaningful samples from the data distribution. In this work, we present Time-Aware Conditional Synthesis (TACS), a novel approach to conditional generation on diffusion models. It integrates adaptively controlled plug-and-play "online" guidance into a diffusion model, driving samples toward the desired properties while maintaining validity and stability. A key component of our algorithm is our new type of diffusion sampler, Time Correction Sampler (TCS), which is used to control guidance and ensure that the generated molecules remain on the correct manifold at each reverse step of the diffusion process at the same time. Our proposed method demonstrates significant performance in conditional 3D molecular generation and offers a promising approach towards inverse molecular design, potentially facilitating advancements in drug discovery, materials science, and other related fields., Comment: NeurIPS 2024

Determining the 3D orientations of an object in an image, known as single-image pose estimation, is a crucial task in 3D vision applications. Existing methods typically learn 3D rotations parametrized in the spatial domain using Euler angles or quaternions, but these representations often introduce discontinuities and singularities. SO(3)-equivariant networks enable the structured capture of pose patterns with data-efficient learning, but the parametrizations in spatial domain are incompatible with their architecture, particularly spherical CNNs, which operate in the frequency domain to enhance computational efficiency. To overcome these issues, we propose a frequency-domain approach that directly predicts Wigner-D coefficients for 3D rotation regression, aligning with the operations of spherical CNNs. Our SO(3)-equivariant pose harmonics predictor overcomes the limitations of spatial parameterizations, ensuring consistent pose estimation under arbitrary rotations. Trained with a frequency-domain regression loss, our method achieves state-of-the-art results on benchmarks such as ModelNet10-SO(3) and PASCAL3D+, with significant improvements in accuracy, robustness, and data efficiency., Comment: Accepted to NeurIPS 2024, Project webpage at http://cvlab.postech.ac.kr/research/3D_EquiPose

Large language models (LLMs) have shown remarkable versatility across tasks, but aligning them with individual human preferences remains challenging due to the complexity and diversity of these preferences. Existing methods often overlook the fact that preferences are multi-objective, diverse, and hard to articulate, making full alignment difficult. In response, we propose an active preference learning framework that uses binary feedback to estimate user preferences across multiple objectives. Our approach leverages Bayesian inference to update preferences efficiently and reduces user feedback through an acquisition function that optimally selects queries. Additionally, we introduce a parameter to handle feedback noise and improve robustness. We validate our approach through theoretical analysis and experiments on language generation tasks, demonstrating its feedback efficiency and effectiveness in personalizing model responses.

Isospin symmetry is a fundamental concept in nuclear physics. Even though isospin symmetry is partially broken, it holds approximately for most nuclear systems, which makes exceptions very interesting from the nuclear structure perspective. In this framework, it is expected that the spins and parities of the ground states of mirror nuclei should be the same, in particular for the simplest systems where a proton is exchanged with a neutron or vice versa. In this work, we present evidence that this assumption is broken in the mirror pair $^{71}$Br and $^{71}$Kr system. Our conclusions are based on a high-statistics $\beta$ decay study of $^{71}$Kr and on state-of-the-art shell model calculations. In our work, we also found evidence of a new state in $^{70}$Se, populated in the $\beta$-delayed proton emission process which can be interpreted as the long sought coexisting 0$^+$ state., Comment: 8 pages with references, 3 figures. Supplemental material 4 pages (1 table, 3 figures)

This paper explores how non-experts can teach robots desired skills in their environments. We argue that natural language is an intuitive and accessible interface for robot learning. To this end, we investigate two key aspects: (1) how non-experts collect robotic data using natural language supervision and (2) how pre-trained vision-language models learn end-to-end policies directly from this supervision. We propose a data collection framework that collects robot demonstrations based on natural language supervision (e.g., "move forward") and further augments these demonstrations. Next, we introduce a model that learns language-conditioned policies from natural language supervision called CLIP-RT. Our model employs pre-trained CLIP models and learns to predict actions represented in language via contrastive imitation learning. We first train CLIP-RT on large-scale robotic data and then enable it to learn desired skills using data collected from our framework. CLIP-RT shows strong capabilities in acquiring novel manipulation skills, outperforming the state-of-the-art model, OpenVLA (7B parameters), by 17% in average success rates, while using 7x fewer parameters (1B)., Comment: 27 pages, 27 figures

We study the null geodesics in the extremal Kerr-Newman exterior. We clarify the roots of the radial potential and obtain the parameter space of the azimuthal angular momentum and the Carter constant of the light rays for varieties of the orbits. It is known that one of the unique features of extremal black holes for the null geodesics is the existence of the stable double root at the horizon, giving rise to the stable spherical motion. For the black hole's spin $a

Accurate prediction of Global Warming Potential (GWP) is essential for assessing the environmental impact of chemical processes and materials. Traditional GWP prediction models rely predominantly on molecular structure, overlooking critical process-related information. In this study, we present an integrative GWP prediction model that combines molecular descriptors (MACCS keys and Mordred descriptors) with process information (process title, description, and location) to improve predictive accuracy and interpretability. Using a deep neural network (DNN) model, we achieved an R-squared of 86% on test data with Mordred descriptors, process location, and description information, representing a 25% improvement over the previous benchmark of 61%; XAI analysis further highlighted the significant role of process title embeddings in enhancing model predictions. To enhance interpretability, we employed a Kolmogorov-Arnold Network (KAN) to derive a symbolic formula for GWP prediction, capturing key molecular and process features and providing a transparent, interpretable alternative to black-box models, enabling users to gain insights into the molecular and process factors influencing GWP. Error analysis showed that the model performs reliably in densely populated data ranges, with increased uncertainty for higher GWP values. This analysis allows users to manage prediction uncertainty effectively, supporting data-driven decision-making in chemical and process design. Our results suggest that integrating both molecular and process-level information in GWP prediction models yields substantial gains in accuracy and interpretability, offering a valuable tool for sustainability assessments. Future work may extend this approach to additional environmental impact categories and refine the model to further enhance its predictive reliability.

Joel L. Horowitz has made profound contributions to many areas in econometrics and statistics. These include bootstrap methods, semiparametric and nonparametric estimation, specification testing, nonparametric instrumental variables estimation, high-dimensional models, functional data analysis, and shape restrictions, among others. Originally trained as a physicist, Joel made a pivotal transition to econometrics, greatly benefiting our profession. Throughout his career, he has collaborated extensively with a diverse range of coauthors, including students, departmental colleagues, and scholars from around the globe. Joel was born in 1941 in Pasadena, California. He attended Stanford for his undergraduate studies and obtained his Ph.D. in physics from Cornell in 1967. He has been Charles E. and Emma H. Morrison Professor of Economics at Northwestern University since 2001. Prior to that, he was a faculty member at the University of Iowa (1982-2001). He has served as a co-editor of Econometric Theory (1992-2000) and Econometrica (2000-2004). He is a Fellow of the Econometric Society and of the American Statistical Association, and an elected member of the International Statistical Institute. The majority of this interview took place in London during June 2022., Comment

Parameter-efficient fine-tuning (PEFT) has enabled the efficient optimization of cumbersome language models in real-world settings. However, as datasets in such environments often contain noisy labels that adversely affect performance, PEFT methods are inevitably exposed to noisy labels. Despite this challenge, the adaptability of PEFT to noisy environments remains underexplored. To bridge this gap, we investigate various PEFT methods under noisy labels. Interestingly, our findings reveal that PEFT has difficulty in memorizing noisy labels due to its inherently limited capacity, resulting in robustness. However, we also find that such limited capacity simultaneously makes PEFT more vulnerable to interference of noisy labels, impeding the learning of clean samples. To address this issue, we propose Clean Routing (CleaR), a novel routing-based PEFT approach that adaptively activates PEFT modules. In CleaR, PEFT modules are preferentially exposed to clean data while bypassing the noisy ones, thereby minimizing the noisy influence. To verify the efficacy of CleaR, we perform extensive experiments on diverse configurations of noisy labels. The results convincingly demonstrate that CleaR leads to substantially improved performance in noisy environments., Comment: Published at ACL 2024 Main Conference

We measure the time-integrated CP asymmetry in $D^{0} \rightarrow K^{0}_{S}K^{0}_{S}$ decays reconstructed in $e^{+}e^{-} \rightarrow c\overline{c}$ events collected by the Belle and Belle II experiments. The corresponding data samples have integrated luminosities of 980 fb$^{-1}$ and 428 fb$^{-1}$, respectively. The $D^{0}$ decays are required to originate from the $D^{*+} \rightarrow D^{0}\pi^{+}$ decay, which determines the charm flavor at production time. A control sample of $D^{0} \rightarrow K^{+}K^{-}$ decays is used to correct for production and detection asymmetries. The result, $(-1.4\pm1.3{\rm(stat)}\pm0.1{\rm (syst)})\%$, is consistent with previous determinations and with CP symmetry., Comment: 10 pages, 3 figures. arXiv admin note: text overlap with arXiv:2410.22961

We explore a large family of one-dimensional (1D) topological crystalline insulators (TCIs) classified by $\mathbb{Z}$ invariants under space-time inversion symmetry. This finding stands in marked contrast to the conventional classification of 1D band topology protected by inversion symmetry and characterized by $\mathbb{Z}_2$-quantized polarization (Berry-Zak phase). By considering the nontrivial relative polarization among sublattices (orbitals), we introduce the inversion winding number as a topological invariant for characterizing and categorizing band topology. The inversion winding number reliably captures a novel bulk-edge correspondence, where gapped edge states are related to the inversion winding number of the sandwiching bands. Leveraging real-space analysis, we discover disorder-induced topological Anderson insulators and propose to experimentally discern band topology through relative polarization of edge states or bulk states. Our comprehensive findings present a paradigmatic illustration for the ongoing investigation and classification of band topology in TCIs.

Despite the versatility of pre-trained language models (PLMs) across domains, their large memory footprints pose significant challenges in federated learning (FL), where the training model has to be distributed between a server and clients. One potential solution to bypass such constraints might be the use of parameter-efficient fine-tuning (PEFT) in the context of FL. However, we have observed that typical PEFT tends to severely suffer from heterogeneity among clients in FL scenarios, resulting in unstable and slow convergence. In this paper, we propose Client-Customized Adaptation (C2A), a novel hypernetwork-based FL framework that generates client-specific adapters by conditioning the client information. With the effectiveness of the hypernetworks in generating customized weights through learning to adopt the different characteristics of inputs, C2A can maximize the utility of shared model parameters while minimizing the divergence caused by client heterogeneity. To verify the efficacy of C2A, we perform extensive evaluations on FL scenarios involving heterogeneity in label and language distributions. Comprehensive evaluation results clearly support the superiority of C2A in terms of both efficiency and effectiveness in FL scenarios., Comment: Published at Findings of ACL 2023

Recent advances in deep learning architectures have enabled efficient and accurate classification of pre-trained targets in Synthetic Aperture Radar (SAR) images. Nevertheless, the presence of unknown targets in real battlefield scenarios is unavoidable, resulting in misclassification and reducing the accuracy of the classifier. Over the past decades, various feature-based out-of-distribution (OOD) approaches have been developed to address this issue, yet defining the decision boundary between known and unknown targets remains challenging. Additionally, unlike optical images, detecting unknown targets in SAR imagery is further complicated by high speckle noise, the presence of clutter, and the inherent similarities in back-scattered microwave signals. In this work, we propose transforming feature-based OOD detection into a class-localized feature-residual-based approach, demonstrating that this method can improve stability across varying unknown targets' distribution conditions. Transforming feature-based OOD detection into a residual-based framework offers a more robust reference space for distinguishing between in-distribution (ID) and OOD data, particularly within the unique characteristics of SAR imagery. This adaptive residual transformation method standardizes feature-based inputs into distributional representations, enhancing OOD detection in noisy, low-information images. Our approach demonstrates promising performance in real-world SAR scenarios, effectively adapting to the high levels of noise and clutter inherent in these environments. These findings highlight the practical relevance of residual-based OOD detection for SAR applications and suggest a foundation for further advancements in unknown target detection in complex, operational settings.

Medical Decision-Making (MDM) is a multi-faceted process that requires clinicians to assess complex multi-modal patient data patient, often collaboratively. Large Language Models (LLMs) promise to streamline this process by synthesizing vast medical knowledge and multi-modal health data. However, single-agent are often ill-suited for nuanced medical contexts requiring adaptable, collaborative problem-solving. Our MDAgents addresses this need by dynamically assigning collaboration structures to LLMs based on task complexity, mimicking real-world clinical collaboration and decision-making. This framework improves diagnostic accuracy and supports adaptive responses in complex, real-world medical scenarios, making it a valuable tool for clinicians in various healthcare settings, and at the same time, being more efficient in terms of computing cost than static multi-agent decision making methods.

We present a comparison of different quantum state preparation algorithms and their overall efficiency for the Schwinger model with a theta term. While adiabatic state preparation (ASP) is proved to be effective, in practice it leads to large CNOT gate counts to prepare the ground state. The quantum approximate optimization algorithm (QAOA) provides excellent results while keeping the CNOT counts small by design, at the cost of an expensive classical minimization process. We introduce a ``blocked'' modification of the Schwinger Hamiltonian to be used in the QAOA that further decreases the length of the algorithms as the size of the problem is increased. The rodeo algorithm (RA) provides a powerful tool to efficiently prepare any eigenstate of the Hamiltonian, as long as its overlap with the initial guess is large enough. We obtain the best results when combining the blocked QAOA ansatz and the RA, as this provides an excellent initial state with a relatively short algorithm without the need to perform any classical steps for large problem sizes., Comment: 11 pages, 9 figures, 4 tables

We present Fashion-VDM, a video diffusion model (VDM) for generating virtual try-on videos. Given an input garment image and person video, our method aims to generate a high-quality try-on video of the person wearing the given garment, while preserving the person's identity and motion. Image-based virtual try-on has shown impressive results; however, existing video virtual try-on (VVT) methods are still lacking garment details and temporal consistency. To address these issues, we propose a diffusion-based architecture for video virtual try-on, split classifier-free guidance for increased control over the conditioning inputs, and a progressive temporal training strategy for single-pass 64-frame, 512px video generation. We also demonstrate the effectiveness of joint image-video training for video try-on, especially when video data is limited. Our qualitative and quantitative experiments show that our approach sets the new state-of-the-art for video virtual try-on. For additional results, visit our project page: https://johannakarras.github.io/Fashion-VDM., Comment: Accepted to SIGGRAPH Asia 2024

Conventional methods for Bayesian optimization (BO) primarily involve one-step optimal decisions (e.g., maximizing expected improvement of the next step). To avoid myopic behavior, multi-step lookahead BO algorithms such as rollout strategies consider the sequential decision-making nature of BO, i.e., as a stochastic dynamic programming (SDP) problem, demonstrating promising results in recent years. However, owing to the curse of dimensionality, most of these methods make significant approximations or suffer scalability issues, e.g., being limited to two-step lookahead. This paper presents a novel reinforcement learning (RL)-based framework for multi-step lookahead BO in high-dimensional black-box optimization problems. The proposed method enhances the scalability and decision-making quality of multi-step lookahead BO by efficiently solving the SDP of the BO process in a near-optimal manner using RL. We first introduce an Attention-DeepSets encoder to represent the state of knowledge to the RL agent and employ off-policy learning to accelerate its initial training. We then propose a multi-task, fine-tuning procedure based on end-to-end (encoder-RL) on-policy learning. We evaluate the proposed method, EARL-BO (Encoder Augmented RL for Bayesian Optimization), on both synthetic benchmark functions and real-world hyperparameter optimization problems, demonstrating significantly improved performance compared to existing multi-step lookahead and high-dimensional BO methods., Comment: 14 pages

This paper presents a novel parametric scattering model (PSM) for sensing extended targets in integrated sensing and communication (ISAC) systems. The PSM addresses the limitations of traditional models by efficiently capturing the target's angular characteristics through a compact set of key parameters, including the central angle and angular spread, enabling efficient optimization. Based on the PSM, we first derive the Cramer-Rao Bound (CRB) for parameter estimation and then propose a beamforming design algorithm to minimize the CRB while meeting both communication signal-to-interference-plus-noise ratio (SINR) and power constraints. By integrating the PSM into the beamforming optimization process, the proposed framework achieves superior CRB performance while balancing the tradeoff between sensing accuracy and communication quality. Simulation results demonstrate that the PSM-based approach consistently outperforms traditional unstructured and discrete scattering models, particularly in resource-limited scenarios, highlighting its practical applicability and scalability., Comment: 5 pages, 3 figures

Quantum error correction (QEC) codes are fundamentally linked to quantum phases of matter: the degenerate ground state manifold corresponds to the code space, while topological excitations represent error syndromes. Building on this concept, the Sachdev-Ye-Kitaev (SYK) model, characterized by its extensive quasi-ground state degeneracy, serves as a constant rate approximate QEC code. In this work, we study the impacts of decoherence on the information-theoretic capacity of SYK models and their variants. Such a capacity is closely tied to traversable wormholes via its thermofield double state, which theoretically enables the teleportation of information across a black hole. We calculate the coherent information in the maximally entangled quasi-ground state space of the SYK models under the fermion parity breaking and parity conserving noise. Interestingly, we find that under the strong fermion parity symmetric noise, the mixed state undergoes the strong to weak spontaneous symmetry breaking of fermion parity, which also corresponds to the information-theoretic transition. Our results highlight the degradation of wormhole traversability in realistic quantum scenarios, as well as providing critical insights into the behavior of approximate constant-rate QEC codes under decoherence., Comment: 5 + 7 pages, 2 + 4 figures

Tracking dense 3D motion from monocular videos remains challenging, particularly when aiming for pixel-level precision over long sequences. We introduce DELTA, a novel method that efficiently tracks every pixel in 3D space, enabling accurate motion estimation across entire videos. Our approach leverages a joint global-local attention mechanism for reduced-resolution tracking, followed by a transformer-based upsampler to achieve high-resolution predictions. Unlike existing methods, which are limited by computational inefficiency or sparse tracking, DELTA delivers dense 3D tracking at scale, running over 8x faster than previous methods while achieving state-of-the-art accuracy. Furthermore, we explore the impact of depth representation on tracking performance and identify log-depth as the optimal choice. Extensive experiments demonstrate the superiority of DELTA on multiple benchmarks, achieving new state-of-the-art results in both 2D and 3D dense tracking tasks. Our method provides a robust solution for applications requiring fine-grained, long-term motion tracking in 3D space., Comment: Project Page: https://snap-research.github.io/DELTA/

Optimization in deep learning remains poorly understood, even in the simple setting of deterministic (i.e. full-batch) training. A key difficulty is that much of an optimizer's behavior is implicitly determined by complex oscillatory dynamics, referred to as the "edge of stability." The main contribution of this paper is to show that an optimizer's implicit behavior can be explicitly captured by a "central flow:" a differential equation which models the time-averaged optimization trajectory. We show that these flows can empirically predict long-term optimization trajectories of generic neural networks with a high degree of numerical accuracy. By interpreting these flows, we reveal for the first time 1) the precise sense in which RMSProp adapts to the local loss landscape, and 2) an "acceleration via regularization" mechanism, wherein adaptive optimizers implicitly navigate towards low-curvature regions in which they can take larger steps. This mechanism is key to the efficacy of these adaptive optimizers. Overall, we believe that central flows constitute a promising tool for reasoning about optimization in deep learning., Comment: first two authors contributed equally; author order determined by coin flip

Anomaly detection is widely used for identifying critical errors and suspicious behaviors, but current methods lack interpretability. We leverage common properties of existing methods and recent advances in generative models to introduce counterfactual explanations for anomaly detection. Given an input, we generate its counterfactual as a diffusion-based repair that shows what a non-anomalous version should have looked like. A key advantage of this approach is that it enables a domain-independent formal specification of explainability desiderata, offering a unified framework for generating and evaluating explanations. We demonstrate the effectiveness of our anomaly explainability framework, AR-Pro, on vision (MVTec, VisA) and time-series (SWaT, WADI, HAI) anomaly datasets. The code used for the experiments is accessible at: https://github.com/xjiae/arpro.

In this work, we prove that, in linear MDPs, the feature dimension $d$ is lower bounded by $S/U$ in order to aptly represent transition probabilities, where $S$ is the size of the state space and $U$ is the maximum size of directly reachable states. Hence, $d$ can still scale with $S$ depending on the direct reachability of the environment. To address this limitation of linear MDPs, we propose a novel structural aggregation framework based on dynamics, named as the "dynamics aggregation". For this newly proposed framework, we design a provably efficient hierarchical reinforcement learning algorithm in linear function approximation that leverages aggregated sub-structures. Our proposed algorithm exhibits statistical efficiency, achieving a regret of $ \tilde{O} ( d_{\psi}^{3/2} H^{3/2}\sqrt{ N T} )$, where $d_{\psi}$ represents the feature dimension of aggregated subMDPs and $N$ signifies the number of aggregated subMDPs. We establish that the condition $d_{\psi}^3 N \ll d^{3}$ is readily met in most real-world environments with hierarchical structures, enabling a substantial improvement in the regret bound compared to LSVI-UCB, which enjoys a regret of $ \tilde{O} (d^{3/2} H^{3/2} \sqrt{ T})$. To the best of our knowledge, this work presents the first HRL algorithm with linear function approximation that offers provable guarantees.