Search

Mathematical reasoning remains an ongoing challenge for AI models, especially for geometry problems that require both linguistic and visual signals. As the vision encoders of most MLLMs are trained on natural scenes, they often struggle to understand geometric diagrams, performing no better in geometry problem solving than LLMs that only process text. This limitation is amplified by the lack of effective methods for representing geometric relationships. To address these issues, we introduce the Diagram Formalization Enhanced Geometry Problem Solver (DFE-GPS), a new framework that integrates visual features, geometric formal language, and natural language representations. We propose a novel synthetic data approach and create a large-scale geometric dataset, SynthGeo228K, annotated with both formal and natural language captions, designed to enhance the vision encoder for a better understanding of geometric structures. Our framework improves MLLMs' ability to process geometric diagrams and extends their application to open-ended tasks on the formalgeo7k dataset.

This work proposes FireRedTTS, a foundation text-to-speech framework, to meet the growing demands for personalized and diverse generative speech applications. The framework comprises three parts: data processing, foundation system, and downstream applications. First, we comprehensively present our data processing pipeline, which transforms massive raw audio into a large-scale high-quality TTS dataset with rich annotations and a wide coverage of content, speaking style, and timbre. Then, we propose a language-model-based foundation TTS system. The speech signal is compressed into discrete semantic tokens via a semantic-aware speech tokenizer, and can be generated by a language model from the prompt text and audio. Then, a two-stage waveform generator is proposed to decode them to the high-fidelity waveform. We present two applications of this system: voice cloning for dubbing and human-like speech generation for chatbots. The experimental results demonstrate the solid in-context learning capability of FireRedTTS, which can stably synthesize high-quality speech consistent with the prompt text and audio. For dubbing, FireRedTTS can clone target voices in a zero-shot way for the UGC scenario and adapt to studio-level expressive voice characters in the PUGC scenario via few-shot fine-tuning with 1-hour recording. Moreover, FireRedTTS achieves controllable human-like speech generation in a casual style with paralinguistic behaviors and emotions via instruction tuning, to better serve spoken chatbots.

Multimodal Federated Learning frequently encounters challenges of client modality heterogeneity, leading to undesired performances for secondary modality in multimodal learning. It is particularly prevalent in audiovisual learning, with audio is often assumed to be the weaker modality in recognition tasks. To address this challenge, we introduce ModalityMirror to improve audio model performance by leveraging knowledge distillation from an audiovisual federated learning model. ModalityMirror involves two phases: a modality-wise FL stage to aggregate uni-modal encoders; and a federated knowledge distillation stage on multi-modality clients to train an unimodal student model. Our results demonstrate that ModalityMirror significantly improves the audio classification compared to the state-of-the-art FL methods such as Harmony, particularly in audiovisual FL facing video missing. Our approach unlocks the potential for exploiting the diverse modality spectrum inherent in multi-modal FL.

This correspondence investigates the novel fluid antenna system (FAS) technology, combining with reconfigurable intelligent surface (RIS) for wireless communications, where a base station (BS) communicates with a FAS-enabled user with the assistance of a RIS. To analyze this technology, we derive the outage probability based on the block-diagonal matrix approximation (BDMA) model. With this, we obtain the upper bound, lower bound, and asymptotic approximation of the outage probability to gain more insights. Moreover, we design the phase shift matrix of the RIS in order to minimize the system outage probability. Simulation results confirm the accuracy of our approximations and that the proposed schemes outperform benchmarks significantly.

In this correspondence, we analyze the performance of a reconfigurable intelligent surface (RIS)-aided communication system that involves a fluid antenna system (FAS)-enabled receiver. By applying the central limit theorem (CLT), we derive approximate expressions for the system outage probability when the RIS has a large number of elements. Also, we adopt the block-correlation channel model to simplify the outage probability expressions, reducing the computational complexity and shedding light on the impact of the number of ports. Numerical results validate the effectiveness of our analysis, especially in scenarios with a large number of RIS elements.

In this paper, we investigate the question of which technology, fluid antenna systems (FAS) or active reconfigurable intelligent surfaces (ARIS), plays a more crucial role in FAS-ARIS wireless communication systems. To address this, we develop a comprehensive system model and explore the problem from an optimization perspective. We introduce an alternating optimization (AO) algorithm incorporating majorization-minimization (MM), successive convex approximation (SCA), and sequential rank-one constraint relaxation (SRCR) to tackle the non-convex challenges inherent in these systems. Specifically, for the transmit beamforming of the BS optimization, we propose a closed-form rank-one solution with low-complexity. For the optimization the positions of fluid antennas (FAs) of the BS, the Taylor expansions and MM algorithm are utilized to construct the effective lower bounds and upper bounds of the objective function and constraints, transforming the non-convex optimization problem into a convex one. Furthermore, we use the SCA and SRCR to optimize the reflection coefficient matrix of the ARIS and effectively solve the rank-one constraint. Simulation results reveal that the relative importance of FAS and ARIS varies depending on the scenario: FAS proves more critical in simpler models with fewer reflecting elements or limited transmission paths, while ARIS becomes more significant in complex scenarios with a higher number of reflecting elements or transmission paths. Ultimately, the integration of both FAS and ARIS creates a win-win scenario, resulting in a more robust and efficient communication system. This study underscores the importance of combining FAS with ARIS, as their complementary use provides the most substantial benefits across different communication environments.

In a Poynting-flux-dominated (PFD) jet that exhibits an ordered magnetic field, a transition towards turbulence and magnetic disorder follows after magnetic reconnection and energy dissipation during the prompt emission phase. In this process, the configuration of the magnetic field evolves with time, rendering it impossible to entirely categorize the magnetic field as ordered. Therefore, we assumed a crude model that incorporates a random magnetic field and an ordered magnetic field, and takes into account the proportionality of the random magnetic field strength to the ordered magnetic field, in order to compute the polarization degree (PD) curve for an individual pulse. It has been discovered that the random magnetic field has a significant impact on the PD results of the low-energy X-ray. In an ordered magnetic field, the X-ray segment maintains a significant PD compared to those in the hundreds of keV and MeV ranges even after electron injection ceases, this making PD easier to detect by polarimetry. However, when the random magnetic field is introduced, the low-energy and high-energy PDs exhibit a similar trend, with the X-ray PD being lower than that of the high-energy segment. Of course, this is related to the rate of disorder in the magnetic field. Additionally, there is two rotation of the polarization angles (PAs) that were not present previously, and the rotation of the PA in the high-energy segment occurs slightly earlier. These results are unrelated to the structure of the ordered magnetic field.

In an era defined by the explosive growth of data and rapid technological advancements, Multimodal Large Language Models (MLLMs) stand at the forefront of artificial intelligence (AI) systems. Designed to seamlessly integrate diverse data types-including text, images, videos, audio, and physiological sequences-MLLMs address the complexities of real-world applications far beyond the capabilities of single-modality systems. In this paper, we systematically sort out the applications of MLLM in multimodal tasks such as natural language, vision, and audio. We also provide a comparative analysis of the focus of different MLLMs in the tasks, and provide insights into the shortcomings of current MLLMs, and suggest potential directions for future research. Through these discussions, this paper hopes to provide valuable insights for the further development and application of MLLM.

Gaussian process (GP) models have received increasingly attentions in recent years due to their superb prediction accuracy and modeling flexibility. To address the computational burdens of GP models for large-scale datasets, distributed learning for GPs are often adopted. Current aggregation models for distributed GPs are not time-efficient when incorporating correlations between GP experts. In this work, we propose a novel approach for aggregated prediction in distributed GPs. The technique is suitable for both the exact and sparse variational GPs. The proposed method incorporates correlations among experts, leading to better prediction accuracy with manageable computational requirements. As demonstrated by empirical studies, the proposed approach results in more stable predictions in less time than state-of-the-art consistent aggregation models., Comment: 25 pages, 12 figures, 3 tables

Gaussian processes (GPs) are widely used in non-parametric Bayesian modeling, and play an important role in various statistical and machine learning applications. In a variety tasks of uncertainty quantification, generating random sample paths of GPs is of interest. As GP sampling requires generating high-dimensional Gaussian random vectors, it is computationally challenging if a direct method, such as the Cholesky decomposition, is used. In this paper, we propose a scalable algorithm for sampling random realizations of the prior and posterior of GP models. The proposed algorithm leverages inducing points approximation with sparse grids, as well as additive Schwarz preconditioners, which reduce computational complexity, and ensure fast convergence. We demonstrate the efficacy and accuracy of the proposed method through a series of experiments and comparisons with other recent works., Comment: 20 pages, 12 figures

SDSS J083942.11+380526.3 ($z=2.315$) is a FeLoBAL quasar that exhibits visible Balmer absorption lines (H$\alpha$), implying a significant $n=2$ population. The quasar also shows an array of absorption lines, including \oi, \niii, \feii, \mgii, \aliii\, to \civ\ and \nv. The high-ionization absorption lines such as \civ\ and \siiv\ are revealed by slightly blueshifted BAL troughs. The resonance doublets such as \mgii\ and \aliii\ are saturated but did not reached zero intensity which indicates that the BLR is partially covered. Overall, however, the absorption is predominantly from low-ionization \feii\ lines, emitted from ground and excited states up to at least 3.814 eV. This implies that the absorbing gas spans the hydrogen ionization front and extends into the partially ionized zone where neutral hydrogen is certainly present. Notably, the hydrogen line spectrum of the quasar shows no signature of expected Ly$\alpha$ absorption. Instead, the line spectrum shows an unusual Ly$\alpha$ emission characterized by a fully filled emission line spectrum which is a composite of a strong narrow core superposed on a weak broad base. Taking into account the effect of partial covering to BLR, we have extracted a strong DLA trough in Ly$\alpha$ emission region. To fit the spectrum, we performed photoionized model calculations and compared them to the observations. We found that photoionization modeling using CLOUDY can successfully reproduce the main characteristics of the quasar spectrum, and the predicted neutral hydrogen column density arising from the clouds responsible for the low-ionization absorption provides a good match to the extracted DLA trough. This indicates that both the DLA and the low-ionization absorption arise from the same medium that is roughly collocated with the dusty torus., Comment: 18 pages, 7 figures, accepted for publication in MNRAS

This study investigates finite-volume effects in physical processes that involve the combination of long-range hadronic matrix elements with electroweak loop integrals. We adopt the approach of implementing the electroweak part as the infinite-volume version, which is denoted as the EW$_\infty$ method in this work. A general approach is established for correcting finite-volume effects in cases where the hadronic intermediate states are dominated by either a single particle or two particles. For the single-particle case, this work derives the infinite volume reconstruction (IVR) method from a new perspective. For the two-particle case, we provide the correction formulas for power-law finite-volume effects and unphysical terms with exponentially divergent time dependence. The finite-volume formalism developed in this study has broad applications, including the QED corrections in various processes and the two-photon exchange contribution in $K_L\to\mu^+\mu^-$ or $\eta\to\mu^+\mu^-$ decays., Comment: 55 pages, 16 figures

We perform the first lattice calculation on the semileptonic decay of $J/\psi$ using the (2+1)-flavor Wilson-clover gauge ensembles generated by CLQCD collaboration. Three gauge ensembles with different lattice spacings, from 0.0519 fm to 0.1053 fm, and pion masses, $m_{\pi}\sim$ 300 MeV, are utilized. After a naive continuum extrapolation using three lattice spacings, we obtain $\operatorname{Br}(J/\psi\rightarrow D_s e\nu_e)=1.90(6)(5)_{V_{cs}}\times 10^{-10}$ and $\operatorname{Br}(J/\psi\rightarrow D e\nu_e)=1.21(6)(9)_{V_{cd}}\times 10^{-11}$, where the first errors are statistical, and the second come from the uncertainties of CKM matrix element $V_{cs(d)}$. The ratios of the branching fractions between lepton $\mu$ and $e$ are also calculated as $R_{J/\psi}(D_s)=0.97002(8)$ and $R_{J/\psi}(D)=0.97423(15)$ after performing a continuum limit including only $a^2$ term. The ratios provide necessary theoretical support for the future experimental test of lepton flavor universality., Comment: 13 pages, 7 figures

Story visualization aims to create visually compelling images or videos corresponding to textual narratives. Despite recent advances in diffusion models yielding promising results, existing methods still struggle to create a coherent sequence of subject-consistent frames based solely on a story. To this end, we propose DreamStory, an automatic open-domain story visualization framework by leveraging the LLMs and a novel multi-subject consistent diffusion model. DreamStory consists of (1) an LLM acting as a story director and (2) an innovative Multi-Subject consistent Diffusion model (MSD) for generating consistent multi-subject across the images. First, DreamStory employs the LLM to generate descriptive prompts for subjects and scenes aligned with the story, annotating each scene's subjects for subsequent subject-consistent generation. Second, DreamStory utilizes these detailed subject descriptions to create portraits of the subjects, with these portraits and their corresponding textual information serving as multimodal anchors (guidance). Finally, the MSD uses these multimodal anchors to generate story scenes with consistent multi-subject. Specifically, the MSD includes Masked Mutual Self-Attention (MMSA) and Masked Mutual Cross-Attention (MMCA) modules. MMSA and MMCA modules ensure appearance and semantic consistency with reference images and text, respectively. Both modules employ masking mechanisms to prevent subject blending. To validate our approach and promote progress in story visualization, we established a benchmark, DS-500, which can assess the overall performance of the story visualization framework, subject-identification accuracy, and the consistency of the generation model. Extensive experiments validate the effectiveness of DreamStory in both subjective and objective evaluations. Please visit our project homepage at https://dream-xyz.github.io/dreamstory.

This paper examines a fluid antenna (FA)-assisted simultaneous wireless information and power transfer (SWIPT) system. Unlike traditional SWIPT systems with fixed-position antennas (FPAs), our FA-assisted system enables dynamic reconfiguration of the radio propagation environment by adjusting the positions of FAs. This capability enhances both energy harvesting and communication performance. The system comprises a base station (BS) equipped with multiple FAs that transmit signals to an energy receiver (ER) and an information receiver (IR), both equipped with a single FA. Our objective is to maximize the communication rate between the BS and the IR while satisfying the harvested power requirement of the ER. This involves jointly optimizing the BS's transmit beamforming and the positions of all FAs. To address this complex convex optimization problem, we employ an alternating optimization (AO) approach, decomposing it into three sub-problems and solving them iteratively using first and second-order Taylor expansions. Simulation results validate the effectiveness of our proposed FA-assisted SWIPT system, demonstrating significant performance improvements over traditional FPA-based systems.

Current 3D self-supervised learning methods of 3D scenes face a data desert issue, resulting from the time-consuming and expensive collecting process of 3D scene data. Conversely, 3D shape datasets are easier to collect. Despite this, existing pre-training strategies on shape data offer limited potential for 3D scene understanding due to significant disparities in point quantities. To tackle these challenges, we propose Shape2Scene (S2S), a novel method that learns representations of large-scale 3D scenes from 3D shape data. We first design multiscale and high-resolution backbones for shape and scene level 3D tasks, i.e., MH-P (point-based) and MH-V (voxel-based). MH-P/V establishes direct paths to highresolution features that capture deep semantic information across multiple scales. This pivotal nature makes them suitable for a wide range of 3D downstream tasks that tightly rely on high-resolution features. We then employ a Shape-to-Scene strategy (S2SS) to amalgamate points from various shapes, creating a random pseudo scene (comprising multiple objects) for training data, mitigating disparities between shapes and scenes. Finally, a point-point contrastive loss (PPC) is applied for the pre-training of MH-P/V. In PPC, the inherent correspondence (i.e., point pairs) is naturally obtained in S2SS. Extensive experiments have demonstrated the transferability of 3D representations learned by MH-P/V across shape-level and scene-level 3D tasks. MH-P achieves notable performance on well-known point cloud datasets (93.8% OA on ScanObjectNN and 87.6% instance mIoU on ShapeNetPart). MH-V also achieves promising performance in 3D semantic segmentation and 3D object detection., Comment: ECCV 2024; Project page: https://github.com/FengZicai/S2S

Dead time is a common instrumental effect of X-ray detectors which would alter the behavior of timing properties of astronomical signals, such as distorting the shape of power density spectra (PDS), affecting the root-mean-square of potential quasi-periodic oscillation signals, etc. We revisit the effects of the dead time of Medium Energy X-ray telescope (ME) onboard Insight-HXMT, based on the simulation of electronic read-out mechanism that causes the dead time, and the real data. We investigate dead time effects on the pulse profile as well as the Quasi-Periodic Oscillation (QPO) signals. The dead time coefficient suggests a linear correlation with the observed count rate in each phase bin of the pulse profile according to the simulation of periodic signal as well as the real data observed on Swift J0243.6+6124. The Fourier-amplitude-difference (FAD) method could well recover the intrinsic shape of the observed PDS in the case that the PDS is from two identical detectors. We apply this technique on ME, by splitting the 9 FPGA modules into 2 groups. The results indicate that the FAD technique suits the case when two groups of detectors are not largely different; and the recovered PDS of Sco X-1 observed by ME slightly enhances the significance of the previously known QPO signal, meanwhile the root-mean-square of QPO is significantly improved. We provide the FAD correction tool implemented in HXMTDAS for users in the future to better analyze QPO signals., Comment: 9 pages, 8 figures, accepted for publication in MNRAS main journal

In this paper, we investigate reconfigurable intelligent surface (RIS)-assisted communication systems which involve a fixed-antenna base station (BS) and a mobile user (MU) that is equipped with fluid antenna system (FAS). Specifically, the RIS is utilized to enable communication for the user whose direct link from the base station is blocked by obstacles. We propose a comprehensive framework that provides transmission design for both static scenarios with the knowledge of channel state information (CSI) and harsh environments where CSI is hard to acquire. It leads to two approaches: a CSI-based scheme where CSI is available, and a CSI-free scheme when CSI is inaccessible. Given the complex spatial correlations in FAS, we employ block-diagonal matrix approximation and independent antenna equivalent models to simplify the derivation of outage probabilities in both cases. Based on the derived outage probabilities, we then optimize the throughput of the FAS-RIS system. For the CSI-based scheme, we first propose a gradient ascent-based algorithm to obtain a near-optimal solution. Then, to address the possible high computational complexity in the gradient algorithm, we approximate the objective function and confirm a unique optimal solution accessible through a bisection search method. For the CSI-free scheme, we apply the partial gradient ascent algorithm, reducing complexity further than full gradient algorithms. We also approximate the objective function and derive a locally optimal closed-form solution to maximize throughput. Simulation results validate the effectiveness of the proposed framework for the transmission design in FAS-RIS systems., Comment: submitted to IEEE journal for possible publication

Medical images and radiology reports are crucial for diagnosing medical conditions, highlighting the importance of quantitative analysis for clinical decision-making. However, the diversity and cross-source heterogeneity of these data challenge the generalizability of current data-mining methods. Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence (AGI) for computer vision, showcasing their potential in the biomedical domain. In this study, we evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets, including 5 medical imaging categories (dermatology, radiology, dentistry, ophthalmology, and endoscopy), and 3 radiology report datasets. The investigated tasks encompass disease classification, lesion segmentation, anatomical localization, disease diagnosis, report generation, and lesion detection. Our experimental results demonstrated that Gemini-series models excelled in report generation and lesion detection but faces challenges in disease classification and anatomical localization. Conversely, GPT-series models exhibited proficiency in lesion segmentation and anatomical localization but encountered difficulties in disease diagnosis and lesion detection. Additionally, both the Gemini series and GPT series contain models that have demonstrated commendable generation efficiency. While both models hold promise in reducing physician workload, alleviating pressure on limited healthcare resources, and fostering collaboration between clinical practitioners and artificial intelligence technologies, substantial enhancements and comprehensive validations remain imperative before clinical deployment.

In this paper, we investigate the channel estimation problem for extremely large-scale multi-input and multi-output (XL-MIMO) systems, considering the spherical wavefront effect, spatially non-stationary (SnS) property, and dual-wideband effects. To accurately characterize the XL-MIMO channel, we first derive a novel spatial-and-frequency-domain channel model for XL-MIMO systems and carefully examine the channel characteristics in the angular-and-delay domain. Based on the obtained channel representation, we formulate XL-MIMO channel estimation as a Bayesian inference problem. To fully exploit the clustered sparsity of angular-and-delay channels and capture the inter-antenna and inter-subcarrier correlations, a Markov random field (MRF)-based hierarchical prior model is adopted. Meanwhile, to facilitate efficient channel reconstruction, we propose a sparse Bayesian learning (SBL) algorithm based on approximate message passing (AMP) with a unitary transformation. Tailored to the MRF-based hierarchical prior model, the message passing equations are reformulated using structured variational inference, belief propagation, and mean-field rules. Finally, simulation results validate the convergence and superiority of the proposed algorithm over existing methods., Comment: This paper has been submitted to IEEE journal for possible publication

The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining the next essential steps for their advancement., Comment: 90 pages, 22 figures, roadmap, neuromorphic

Effective expression feature representations generated by a triplet-based deep metric learning are highly advantageous for facial expression recognition (FER). The performance of triplet-based deep metric learning is contingent upon identifying the best threshold for triplet loss. Threshold validation, however, is tough and challenging, as the ideal threshold changes among datasets and even across classes within the same dataset. In this paper, we present the multi-threshold deep metric learning technique, which not only avoids the difficult threshold validation but also vastly increases the capacity of triplet loss learning to construct expression feature representations. We find that each threshold of the triplet loss intrinsically determines a distinctive distribution of inter-class variations and corresponds, thus, to a unique expression feature representation. Therefore, rather than selecting a single optimal threshold from a valid threshold range, we thoroughly sample thresholds across the range, allowing the representation characteristics manifested by thresholds within the range to be fully extracted and leveraged for FER. To realize this approach, we partition the embedding layer of the deep metric learning network into a collection of slices and model training these embedding slices as an end-to-end multi-threshold deep metric learning problem. Each embedding slice corresponds to a sample threshold and is learned by enforcing the corresponding triplet loss, yielding a set of distinct expression features, one for each embedding slice. It makes the embedding layer, which is composed of a set of slices, a more informative and discriminative feature, hence enhancing the FER accuracy. Extensive evaluations demonstrate the superior performance of the proposed approach on both posed and spontaneous facial expression datasets., Comment: accepted by Pattern Recognition

Reinforcement learning from human feedback (RLHF) provides a principled framework for aligning AI systems with human preference data. For various reasons, e.g., personal bias, context ambiguity, lack of training, etc, human annotators may give incorrect or inconsistent preference labels. To tackle this challenge, we propose a robust RLHF approach -- $R^3M$, which models the potentially corrupted preference label as sparse outliers. Accordingly, we formulate the robust reward learning as an $\ell_1$-regularized maximum likelihood estimation problem. Computationally, we develop an efficient alternating optimization algorithm, which only incurs negligible computational overhead compared with the standard RLHF approach. Theoretically, we prove that under proper regularity conditions, $R^3M$ can consistently learn the underlying reward and identify outliers, provided that the number of outlier labels scales sublinearly with the preference sample size. Furthermore, we remark that $R^3M$ is versatile and can be extended to various preference optimization methods, including direct preference optimization (DPO). Our experiments on robotic control and natural language generation with large language models (LLMs) show that $R^3M$ improves robustness of the reward against several types of perturbations to the preference data., Comment: 22 pages, 7 figures

In Federated Learning (FL), clients may have weak devices that cannot train the full model or even hold it in their memory space. To implement large-scale FL applications, thus, it is crucial to develop a distributed learning method that enables the participation of such weak clients. We propose EmbracingFL, a general FL framework that allows all available clients to join the distributed training regardless of their system resource capacity. The framework is built upon a novel form of partial model training method in which each client trains as many consecutive output-side layers as its system resources allow. Our study demonstrates that EmbracingFL encourages each layer to have similar data representations across clients, improving FL efficiency. The proposed partial model training method guarantees convergence to a neighbor of stationary points for non-convex and smooth problems. We evaluate the efficacy of EmbracingFL under a variety of settings with a mixed number of strong, moderate (~40% memory), and weak (~15% memory) clients, datasets (CIFAR-10, FEMNIST, and IMDB), and models (ResNet20, CNN, and LSTM). Our empirical study shows that EmbracingFL consistently achieves high accuracy as like all clients are strong, outperforming the state-of-the-art width reduction methods (i.e. HeteroFL and FjORD).

Pre-trained language models, trained on large-scale corpora, demonstrate strong generalizability across various NLP tasks. Fine-tuning these models for specific tasks typically involves updating all parameters, which is resource-intensive. Parameter-efficient fine-tuning (PEFT) methods, such as the popular LoRA family, introduce low-rank matrices to learn only a few parameters efficiently. However, during inference, the product of these matrices updates all pre-trained parameters, complicating tasks like knowledge editing that require selective updates. We propose a novel PEFT method, which conducts \textbf{r}ow and c\textbf{o}lumn-wise spar\textbf{se} \textbf{lo}w-\textbf{r}ank \textbf{a}daptation (RoseLoRA), to address this challenge. RoseLoRA identifies and updates only the most important parameters for a specific task, maintaining efficiency while preserving other model knowledge. By adding a sparsity constraint on the product of low-rank matrices and converting it to row and column-wise sparsity, we ensure efficient and precise model updates. Our theoretical analysis guarantees the lower bound of the sparsity with respective to the matrix product. Extensive experiments on five benchmarks across twenty datasets demonstrate that RoseLoRA outperforms baselines in both general fine-tuning and knowledge editing tasks.

Large vision-language models (VLMs) have demonstrated remarkable abilities in understanding everyday content. However, their performance in the domain of art, particularly culturally rich art forms, remains less explored. As a pearl of human wisdom and creativity, art encapsulates complex cultural narratives and symbolism. In this paper, we offer the Pun Rebus Art Dataset, a multimodal dataset for art understanding deeply rooted in traditional Chinese culture. We focus on three primary tasks: identifying salient visual elements, matching elements with their symbolic meanings, and explanations for the conveyed messages. Our evaluation reveals that state-of-the-art VLMs struggle with these tasks, often providing biased and hallucinated explanations and showing limited improvement through in-context learning. By releasing the Pun Rebus Art Dataset, we aim to facilitate the development of VLMs that can better understand and interpret culturally specific content, promoting greater inclusiveness beyond English-based corpora.

Sulfur chemistry in the formation process of low-mass stars and planets remains poorly understood. The protoplanetary disks (PPDs) are the birthplace of planets and its distinctive environment provides an intriguing platform for investigating models of sulfur chemistry. We analyzed the ALMA observations of CS 7-6 transitions in the HD 163296 disk and perform astrochemical modeling to explore its sulfur chemistry. We simulated the distribution of sulfur-containing molecules and compared it with observationally deduced fractional column densities. We have found that the simulated column density of CS is consistent with the observationally deduced fractional column densities, while the simulated column density of C$_2$S is lower than the observationally deduced upper limits on column densities. This results indicate that we have a good understanding of the chemical properties of CS and C$_2$S in the disk. We also investigated the influence of the C/O ratio on sulfur-containing molecules and found that the column densities of SO, SO$_2$, and H$_2$S near the central star are dependent on the C/O ratio. Additionally, we found that the $N$[CS]/$N$[SO] ratio can serve as a promising indicator of the disk's C/O ratio in the HD 163296. Overall, the disk of HD 163296 provides a favorable environment for the detection of sulfur-containing molecules., Comment: 16 pages, 7 figures

We investigate the quantum phase transition in the anisotropic Dicke model through an examination of the quantum geometric tensor of the ground state. In this analysis, two distinct classical limits exhibit their unique anisotropic characteristics. The classical spin limit demonstrates a preference for the rotating-wave coupling, whereas the classical oscillator limit exhibits symmetry in the coupling strength of the bias. The anisotropic features of the classical spin limit persist at finite scales. Furthermore, we observe that the interplay among the anisotropic ratio, spin length, and frequency ratio can collectively enhance the critical behaviors. This critical enhancement without trade-off between these factors provides a flexible method for quantum precision measurement.

Reinforcement learning from human feedback (RLHF) is a prevalent approach to align AI systems with human values by learning rewards from human preference data. Due to various reasons, however, such data typically takes the form of rankings over pairs of trajectory segments, which fails to capture the varying strengths of preferences across different pairs. In this paper, we propose a novel adaptive preference loss, underpinned by distributionally robust optimization (DRO), designed to address this uncertainty in preference strength. By incorporating an adaptive scaling parameter into the loss for each pair, our method increases the flexibility of the reward function. Specifically, it assigns small scaling parameters to pairs with ambiguous preferences, leading to more comparable rewards, and large scaling parameters to those with clear preferences for more distinct rewards. Computationally, our proposed loss function is strictly convex and univariate with respect to each scaling parameter, enabling its efficient optimization through a simple second-order algorithm. Our method is versatile and can be readily adapted to various preference optimization frameworks, including direct preference optimization (DPO). Our experiments with robotic control and natural language generation with large language models (LLMs) show that our method not only improves policy performance but also aligns reward function selection more closely with policy optimization, simplifying the hyperparameter tuning process.

This work focuses on the design of experiments of multi-fidelity computer experiments. We consider the autoregressive Gaussian process model proposed by Kennedy and O'Hagan (2000) and the optimal nested design that maximizes the prediction accuracy subject to a budget constraint. An approximate solution is identified through the idea of multi-level approximation and recent error bounds of Gaussian process regression. The proposed (approximately) optimal designs admit a simple analytical form. We prove that, to achieve the same prediction accuracy, the proposed optimal multi-fidelity design requires much lower computational cost than any single-fidelity design in the asymptotic sense. Numerical studies confirm this theoretical assertion.