Search

In recent years, with the rapid development of machine learning, physicists have been exploring its new applications in solving or alleviating the curse of dimensionality in many-body problems. In order to accurately reflect the underlying physics of the problem, domain knowledge must be encoded into the machine learning algorithms. In this work, inspired by field theory, we propose a new set of machine learning models called effective field neural networks (EFNNs) that can automatically and efficiently capture important many-body interactions through multiple self-refining processes. Taking the classical $3$-spin infinite-range model and the quantum double exchange model as case studies, we explicitly demonstrate that EFNNs significantly outperform fully-connected deep neural networks (DNNs) and the effective model. Furthermore, with the help of convolution operations, the EFNNs learned in a small system can be seamlessly used in a larger system without additional training and the relative errors even decrease, which further demonstrates the efficacy of EFNNs in representing core physical behaviors.

Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM., Comment: Accepted by the ACM Web Conference (WWW) 2025 Industrial Track as Oral Presentation

Offline model-based reinforcement learning (MBRL) serves as a competitive framework that can learn well-performing policies solely from pre-collected data with the help of learned dynamics models. To fully unleash the power of offline MBRL, model selection plays a pivotal role in determining the dynamics model utilized for downstream policy learning. However, offline MBRL conventionally relies on validation or off-policy evaluation, which are rather inaccurate due to the inherent distribution shift in offline RL. To tackle this, we propose BOMS, an active model selection framework that enhances model selection in offline MBRL with only a small online interaction budget, through the lens of Bayesian optimization (BO). Specifically, we recast model selection as BO and enable probabilistic inference in BOMS by proposing a novel model-induced kernel, which is theoretically grounded and computationally efficient. Through extensive experiments, we show that BOMS improves over the baseline methods with a small amount of online interaction comparable to only $1\%$-$2.5\%$ of offline training data on various RL tasks., Comment: 18 pages, 6 figures

Latent 3D reconstruction has shown great promise in empowering 3D semantic understanding and 3D generation by distilling 2D features into the 3D space. However, existing approaches struggle with the domain gap between 2D feature space and 3D representations, resulting in degraded rendering performance. To address this challenge, we propose a novel framework that integrates 3D awareness into the 2D latent space. The framework consists of three stages: (1) a correspondence-aware autoencoding method that enhances the 3D consistency of 2D latent representations, (2) a latent radiance field (LRF) that lifts these 3D-aware 2D representations into 3D space, and (3) a VAE-Radiance Field (VAE-RF) alignment strategy that improves image decoding from the rendered 2D representations. Extensive experiments demonstrate that our method outperforms the state-of-the-art latent 3D reconstruction approaches in terms of synthesis performance and cross-dataset generalizability across diverse indoor and outdoor scenes. To our knowledge, this is the first work showing the radiance field representations constructed from 2D latent representations can yield photorealistic 3D reconstruction performance., Comment: Accepted to ICLR 2025; Project page: https://latent-radiance-field.github.io/LRF

We report the GeV $\gamma$-ray emission around the composite supernova remnant (SNR) CTB 87 with more than 16 yrs PASS 8 data recorded by the Fermi Large Area Telescope. Two separate point sources with the different GeV spectra are identified in this region: one has a soft $\gamma$-ray spectrum, likely due to interactions between the SNR shock and molecular clouds (MCs); and another source with a hard GeV $\gamma$-ray spectrum aligns with the TeV spectrum of VER J2016+371, suggesting it as the GeV counterpart. Considering the observations of CTB 87 in the radio and X-ray bands, VER J2016+371 is proposed to originate from the pulsar wind nebula (PWN) associated with PSR J2016+3711. A leptonic model with a broken power-law electron distribution could explain the multi-wavelength data of VER J2016+371, with fitted parameters matching typical $\gamma$-ray PWNe. Deeper searching for the SNR shock of CTB 87 in other bands and the future TeV observations by LHAASO and CTA are crucial to reveal the nature of CTB 87., Comment: 11 pages, 4 figures, 2 tables, accepted for publicaton in ApJ

Recently, coupled tensor decomposition has been widely used in data fusion of a hyperspectral image (HSI) and a multispectral image (MSI) for hyperspectral super-resolution (HSR). However, exsiting works often ignore the inherent non-negative (NN) property of the image data, or impose the NN constraint via hard-thresholding which may interfere with the optimization procedure and cause the method to be sub-optimal. As such, we propose a novel NN coupled canonical polyadic decomposition (NN-C-CPD) algorithm, which makes use of the parametric method and nonlinear least squares (NLS) framework to impose the NN constraint into the C-CPD computation. More exactly, the NN constraint is converted into the squared relationship between the NN entries of the factor matrices and a set of latent parameters. Based on the chain rule for deriving the derivatives, the key entities such as gradient and Jacobian with regards to the latent parameters can be derived, thus the NN constraint is naturally integrated without interfering with the optimization procedure. Experimental results are provided to demonstrate the performance of the proposed NN-C-CPD algorithm in HSR applications., Comment: 5 pages, 4 figures,ICASSP

We consider tensor data completion of an incomplete observation of multidimensional harmonic (MH) signals. Unlike existing tensor-based techniques for MH retrieval (MHR), which mostly adopt the canonical polyadic decomposition (CPD) to model the simple "one-to-one" correspondence among harmonics across difference modes, we herein use the more flexible block term decomposition (BTD) model that can be used to describe the complex mutual correspondences among several groups of harmonics across different modes. An optimization principle that aims to fit the BTD model in the least squares sense, subject to rank minimization of hankelized MH components, is set up for the tensor completion task, and an algorithm based on alternating direction method of multipliers is proposed, of which the effectiveness and applicability are validated through both numerical simulations and an application in Sub-6GHz channel state information (CSI) completion.

Owing to recent advances in machine learning and the ability to harvest large amounts of data during robotic-assisted surgeries, surgical data science is ripe for foundational work. We present a large dataset of surgical videos and their accompanying labels for this purpose. We describe how the data was collected and some of its unique attributes. Multiple example problems are outlined. Although the dataset was curated for a particular set of scientific challenges (in an accompanying paper), it is general enough to be used for a broad range machine learning questions. Our hope is that this dataset exposes the larger machine learning community to the challenging problems within surgical data science, and becomes a touchstone for future research. The videos are available at https://storage.googleapis.com/isi-surgvu/surgvu24_videos_only.zip, the labels at https://storage.googleapis.com/isi-surgvu/surgvu24_labels_updated_v2.zip, and a validation set for tool detection problem at https://storage.googleapis.com/isi-surgvu/cat1_test_set_public.zip.

The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture to concurrently enhance both aspects. By integrating Graph Convolutional Networks (GCNs) as efficient retrieval mechanisms, we are able to significantly reduce data retrieval times while maintaining high model performance. The early exit strategy employed allows for dynamic termination of model inference, utilizing real-time predictive confidence assessments across multiple heads. This not only quickens the responsiveness of LLMs but also upholds or improves their accuracy, making it ideal for real-time application scenarios. Our experiments demonstrate how this architecture effectively decreases computation time without sacrificing the accuracy needed for reliable recommendation delivery, establishing a new standard for efficient, real-time LLM deployment in commercial systems.

Topological materials for classical waves offer remarkable potential in applications such as sensing, waveguiding and signal processing, leveraging topological protection effects like strong robustness, immunity to backscattering and unidirectional transmission. This work presents the simultaneous inverse design of pseudospin-dependent topological edge states for acoustic and electromagnetic waves in two-dimensional $C_{\textrm{6v}}$ phoxonic crystals. The phoxonic crystals are created by arranging the silicon columns periodically in the air background. We propose a multi-objective optimization framework based on the NSGA-II collaborated with the finite element approach, where the bandgaps of acoustic and electromagnetic waves are treated separately as the objective values. The topological nature of bandgaps is determined by analyzing the positional relationships of paired degenerate modes through the modal field calculations, enabling the customization of one of the two bandgaps within the same unit cell. Unlike traditional approaches relying on the band inversion to induce topological phase transitions, the proposed approach directly generates a pair of unit cells with distinct topological properties for both wave types, achieving the maximum bandgap matching in each case. We further demonstrate the existence of the pseudospin-dependent topological edge states for both acoustic and electromagnetic waves, verifying their unidirectionality and robustness against backscattering and defects. This work establishes a systematic strategy for customizing phoxonic topological states, offering a new avenue for the inverse design of multi-functional devices based on both sound and light.

Large language models (LLMs) are notoriously memory-intensive during training, particularly with the popular AdamW optimizer. This memory burden necessitates using more or higher-end GPUs or reducing batch sizes, limiting training scalability and throughput. To address this, various memory-efficient optimizers have been proposed to reduce optimizer memory usage. However, they face critical challenges: (i) reliance on costly SVD operations; (ii) significant performance trade-offs compared to AdamW; and (iii) still substantial optimizer memory overhead to maintain competitive performance. In this work, we identify that AdamW's learning rate adaptation rule can be effectively coarsened as a structured learning rate update. Based on this insight, we propose Approximated Gradient Scaling for Memory-Efficient LLM Optimization (APOLLO), which approximates learning rate scaling using an auxiliary low-rank optimizer state based on pure random projection. This structured learning rate update rule makes APOLLO highly tolerant to further memory reductions while delivering comparable pre-training performance. Even its rank-1 variant, APOLLO-Mini, achieves superior pre-training performance compared to AdamW with SGD-level memory costs. Extensive experiments demonstrate that the APOLLO series performs on-par with or better than AdamW, while achieving greater memory savings by nearly eliminating the optimization states of AdamW. These savings provide significant system-level benefits: (1) Enhanced Throughput: 3x throughput on an 8xA100-80GB setup compared to AdamW by supporting 4x larger batch sizes. (2) Improved Model Scalability: Pre-training LLaMA-13B with naive DDP on A100-80GB GPUs without system-level optimizations. (3) Low-End GPU Friendly Pre-training: Pre-training LLaMA-7B on a single GPU using less than 12 GB of memory with weight quantization., Comment: Accepted to MLSys 2025; the newest version with new experiments

Click-through rate (CTR) prediction, which predicts the probability of a user clicking an ad, is a fundamental task in recommender systems. The emergence of heterogeneous information, such as user profile and behavior sequences, depicts user interests from different aspects. A mutually beneficial integration of heterogeneous information is the cornerstone towards the success of CTR prediction. However, most of the existing methods suffer from two fundamental limitations, including (1) insufficient inter-mode interaction due to the unidirectional information flow between modes, and (2) aggressive information aggregation caused by early summarization, resulting in excessive information loss. To address the above limitations, we propose a novel module named InterFormer to learn heterogeneous information interaction in an interleaving style. To achieve better interaction learning, InterFormer enables bidirectional information flow for mutually beneficial learning across different modes. To avoid aggressive information aggregation, we retain complete information in each data mode and use a separate bridging arch for effective information selection and summarization. Our proposed InterFormer achieves state-of-the-art performance on three public datasets and a large-scale industrial dataset., Comment: 10 pages, 6 figures

With the 15 yrs of Pass 8 data recorded by the {\em Fermi} Large Area Telescope, we report the detection of an extended gigaelectronvolt emission component with a 68\% containment radius of $0^{\circ}\!.85$, which is spatially associated with the newly identified supernova remnant (SNR) G321.3-3.9. The $\gamma$-ray spectrum is best described by a log-parabola model in the energy range of 100 MeV - 1 TeV, which shows a significant spectral curvature at $\sim$ 1 GeV. Either a leptonic or a hadronic model could explain the multi-wavelength data of G321.3-3.9, while the leptonic model predicts a too low strength of magnetic field. Also considering the flat radio spectrum of G321.3-3.9 and the $\gamma$-ray upper limit in the low energy band, the hadronic model is favored. The spatial coincidence between the $\gamma$-ray morphology and the diffuse thermal X-ray emission of G321.3-3.9 and the curved gigaelectronvolt $\gamma$-ray spectrum of it make G321.3-3.9 to be similar to the typical middle-aged SNRs interacting with molecular clouds. Such characteristics provide another evidence of the potential hadronic origin for its $\gamma$-ray emission. While there is no molecular cloud detected around G321.3-3.9, which challenges the hadronic model., Comment: 9 pages, 5 figures, 2 tables, accepted for publication in ApJ

Novel-view synthesis aims to generate novel views of a scene from multiple input images or videos, and recent advancements like 3D Gaussian splatting (3DGS) have achieved notable success in producing photorealistic renderings with efficient pipelines. However, generating high-quality novel views under challenging settings, such as sparse input views, remains difficult due to insufficient information in under-sampled areas, often resulting in noticeable artifacts. This paper presents 3DGS-Enhancer, a novel pipeline for enhancing the representation quality of 3DGS representations. We leverage 2D video diffusion priors to address the challenging 3D view consistency problem, reformulating it as achieving temporal consistency within a video generation process. 3DGS-Enhancer restores view-consistent latent features of rendered novel views and integrates them with the input views through a spatial-temporal decoder. The enhanced views are then used to fine-tune the initial 3DGS model, significantly improving its rendering performance. Extensive experiments on large-scale datasets of unbounded scenes demonstrate that 3DGS-Enhancer yields superior reconstruction performance and high-fidelity rendering results compared to state-of-the-art methods. The project webpage is https://xiliu8006.github.io/3DGS-Enhancer-project ., Comment: Accepted by NeurIPS 2024 Spotlight

The interplay between order and disorder is crucial across various fields, especially in understanding oscillatory phenomena. Periodic oscillations are frequently observed in heterogeneous catalysis, yet their underlying mechanisms need deeper exploration. Here, we investigate how periodic oscillations arise during methane oxidation catalysed by palladium nanoparticles (Pd NPs), utilizing a suite of complementary operando techniques across various spatial scales. We found that reaction intensity and collective dynamic modes can be tuned by the reactant gas-flow rate. At lower gas-flow rates, we observed periodic facet reconstruction of Pd NPs correlated with repeated bubbling behaviour at the Pd/PdO interface, without evident global oscillatory responses. Conversely, at higher gas-flow rates, Pd NPs undergo chaotic transformations between metallic and oxidized states, resulting in overall oscillation. Integrating our observations at different gas-flow rates, we attributed the emergence of global oscillation to thermal coupling regulated by gas flow and connected local and global dynamics through a weak synchronization mechanism. This work demonstrates the correlations between open surfaces and interfaces, chaos and regularity, and dissipative processes and coupling behaviour. Our findings offer critical insights into the complexity behind catalytic oscillations and provide guidance for modulating oscillatory behaviours in catalytic processes, with significant implications for both science and industry.

Scaling laws play an instrumental role in the sustainable improvement in model quality. Unfortunately, recommendation models to date do not exhibit such laws similar to those observed in the domain of large language models, due to the inefficiencies of their upscaling mechanisms. This limitation poses significant challenges in adapting these models to increasingly more complex real-world datasets. In this paper, we propose an effective network architecture based purely on stacked factorization machines, and a synergistic upscaling strategy, collectively dubbed Wukong, to establish a scaling law in the domain of recommendation. Wukong's unique design makes it possible to capture diverse, any-order of interactions simply through taller and wider layers. We conducted extensive evaluations on six public datasets, and our results demonstrate that Wukong consistently outperforms state-of-the-art models quality-wise. Further, we assessed Wukong's scalability on an internal, large-scale dataset. The results show that Wukong retains its superiority in quality over state-of-the-art models, while holding the scaling law across two orders of magnitude in model complexity, extending beyond 100 GFLOP/example, where prior arts fall short., Comment: 12 pages

Listening head generation aims to synthesize a non-verbal responsive listener head by modeling the correlation between the speaker and the listener in dynamic conversion.The applications of listener agent generation in virtual interaction have promoted many works achieving the diverse and fine-grained motion generation. However, they can only manipulate motions through simple emotional labels, but cannot freely control the listener's motions. Since listener agents should have human-like attributes (e.g. identity, personality) which can be freely customized by users, this limits their realism. In this paper, we propose a user-friendly framework called CustomListener to realize the free-form text prior guided listener generation. To achieve speaker-listener coordination, we design a Static to Dynamic Portrait module (SDP), which interacts with speaker information to transform static text into dynamic portrait token with completion rhythm and amplitude information. To achieve coherence between segments, we design a Past Guided Generation Module (PGG) to maintain the consistency of customized listener attributes through the motion prior, and utilize a diffusion-based structure conditioned on the portrait token and the motion prior to realize the controllable generation. To train and evaluate our model, we have constructed two text-annotated listening head datasets based on ViCo and RealTalk, which provide text-video paired labels. Extensive experiments have verified the effectiveness of our model., Comment: Accepted by CVPR 2024