Search

Reinforcement Learning (RL) mathematically formulates decision-making with Markov Decision Process (MDP). With MDPs, researchers have achieved remarkable breakthroughs across various domains, including games, robotics, and language models. This paper seeks a new possibility, Natural Language Reinforcement Learning (NLRL), by extending traditional MDP to natural language-based representation space. Specifically, NLRL innovatively redefines RL principles, including task objectives, policy, value function, Bellman equation, and policy iteration, into their language counterparts. With recent advancements in large language models (LLMs), NLRL can be practically implemented to achieve RL-like policy and value improvement by either pure prompting or gradient-based training. Experiments over Maze, Breakthrough, and Tic-Tac-Toe games demonstrate the effectiveness, efficiency, and interpretability of the NLRL framework among diverse use cases. Our code will be released at https://github.com/waterhorse1/Natural-language-RL., Comment: Extension of arXiv:2402.07157

In the era of (multi-modal) large language models, most operational processes can be reformulated and reproduced using LLM agents. The LLM agents can perceive, control, and get feedback from the environment so as to accomplish the given tasks in an autonomous manner. Besides the environment-interaction property, the LLM agents can call various external tools to ease the task completion process. The tools can be regarded as a predefined operational process with private or real-time knowledge that does not exist in the parameters of LLMs. As a natural trend of development, the tools for calling are becoming autonomous agents, thus the full intelligent system turns out to be a multi-LLM-agent system (MLAS). This paper discusses the technical and business landscapes of MLAS. Compared to the previous single-LLM-agent system, a MLAS has the advantages of i) higher potential of task-solving performance, ii) higher flexibility for system changing, iii) proprietary data preserving for each participating entity, and iv) feasibility of monetization for each entity. To support the ecosystem of MLAS, we provide a preliminary version of such MLAS protocol considering technical requirements, data privacy, and business incentives. As such, MLAS would be a practical solution to achieve artificial collective intelligence in the near future.

Controlling the style and characteristics of speech synthesis is crucial for adapting the output to specific contexts and user requirements. Previous Text-to-speech (TTS) works have focused primarily on the technical aspects of producing natural-sounding speech, such as intonation, rhythm, and clarity. However, they overlook the fact that there is a growing emphasis on spatial perception of synthesized speech, which may provide immersive experience in gaming and virtual reality. To solve this issue, in this paper, we present a novel multi-modal TTS approach, namely Image-indicated Immersive Text-to-speech Synthesis (I2TTS). Specifically, we introduce a scene prompt encoder that integrates visual scene prompts directly into the synthesis pipeline to control the speech generation process. Additionally, we propose a reverberation classification and refinement technique that adjusts the synthesized mel-spectrogram to enhance the immersive experience, ensuring that the involved reverberation condition matches the scene accurately. Experimental results demonstrate that our model achieves high-quality scene and spatial matching without compromising speech naturalness, marking a significant advancement in the field of context-aware speech synthesis. Project demo page: https://spatialTTS.github.io/ Index Terms-Speech synthesis, scene prompt, spatial perception, Comment: 5pages,4figures

In the field of medical image segmentation, challenges such as indistinct lesion features, ambiguous boundaries,and multi-scale characteristics have long revailed. This paper proposes an improved method named Intensity-Spatial Dual Masked AutoEncoder (ISD-MAE). Based on the tissue-contrast semi-masked autoencoder, a Masked AutoEncoder (MAE) branch is introduced to perform intensity masking and spatial masking operations on chest CT images for multi-scale feature learning and segmentation tasks. The model utilizes a dual-branch structure and contrastive learning to enhance the ability to learn tissue features and boundary details. Experiments are conducted on multiple 2D and 3D datasets. The results show that ISD-MAE significantly outperforms other methods in 2D pneumonia and mediastinal tumor segmentation tasks. For example, the Dice score reaches 90.10% on the COVID19 LESION dataset, and the performance is relatively stable. However, there is still room for improvement on 3D datasets. In response to this, improvement directions are proposed, including optimizing the loss function, using enhanced 3D convolution blocks, and processing datasets from multiple perspectives.Our code is available at:https://github.com/prowontheus/ISD-MAE., Comment: 10 pages,6 figures,3 tables

The alignment of large language models (LLMs) with human preferences remains a key challenge. While post-training techniques like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) have achieved notable success, they often introduce computational inefficiencies and training instability. In this paper, we propose Feature-level constrained Preference Optimization (FPO), a novel method designed to simplify the alignment process while ensuring stability. FPO leverages pre-trained Sparse Autoencoders (SAEs) and introduces feature-level constraints, allowing for efficient, sparsity-enforced alignment. Our approach enjoys efficiency by using sparse features activated in a well-trained sparse autoencoder and the quality of sequential KL divergence by using the feature-level offline reference. Experimental results on benchmark datasets demonstrate that FPO achieves a 5.08% absolute improvement in win rate with much lower computational cost compared to state-of-the-art baselines, making it a promising solution for efficient and controllable LLM alignments.

Mixture-of-Experts (MOE) has recently become the de facto standard in Multi-domain recommendation (MDR) due to its powerful expressive ability. However, such MOE-based method typically employs all experts for each instance, leading to scalability issue and low-discriminability between domains and experts. Furthermore, the design of commonly used domain-specific networks exacerbates the scalability issues. To tackle the problems, We propose a novel method named CESAA consists of Conditional Expert Selection (CES) Module and Adaptive Expert Aggregation (AEA) Module to tackle these challenges. Specifically, CES first combines a sparse gating strategy with domain-shared experts. Then AEA utilizes mutual information loss to strengthen the correlations between experts and specific domains, and significantly improve the distinction between experts. As a result, only domain-shared experts and selected domain-specific experts are activated for each instance, striking a balance between computational efficiency and model performance. Experimental results on both public ranking and industrial retrieval datasets verify the effectiveness of our method in MDR tasks.

We introduce BLIP3-KALE, a dataset of 218 million image-text pairs that bridges the gap between descriptive synthetic captions and factual web-scale alt-text. KALE augments synthetic dense image captions with web-scale alt-text to generate factually grounded image captions. Our two-stage approach leverages large vision-language models and language models to create knowledge-augmented captions, which are then used to train a specialized VLM for scaling up the dataset. We train vision-language models on KALE and demonstrate improvements on vision-language tasks. Our experiments show the utility of KALE for training more capable and knowledgeable multimodal models. We release the KALE dataset at https://huggingface.co/datasets/Salesforce/blip3-kale

Over the past decades, we have witnessed a rapid emergence of soft and reconfigurable robots thanks to their capability to interact safely with humans and adapt to complex environments. However, their softness makes accurate control very challenging. High-fidelity sensing is critical in improving control performance, especially posture and contact estimation. To this end, traditional camera-based sensors and load cells have limited portability and accuracy, and they will inevitably increase the robot's cost and weight. In this study, instead of using specialized sensors, we only collect distributed pressure data inside a pneumatics-driven soft arm and apply the physical reservoir computing principle to simultaneously predict its kinematic posture (i.e., bending angle) and payload status (i.e., payload mass). Our results show that, with careful readout training, one can obtain accurate bending angle and payload mass predictions via simple, weighted linear summations of pressure readings. In addition, our comparative analysis shows that, to guarantee low prediction errors within 10\%, bending angle prediction requires less training data than payload prediction. This result reveals that balanced linear and nonlinear body dynamics are critical for the physical reservoir to accomplish complex proprioceptive and exteroceptive information perception tasks. Finally, the method of exploring the most efficient readout training methods presented in this paper could be extended to other soft robotic systems to maximize their perception capabilities.

We investigate the Berry phase arising from axion-gauge-boson and axion-fermion interactions. The effective Hamiltonians in these two systems are shown to share the same form, enabling a unified description of the Berry phase. This approach offers a new perspective on certain axion experiments, including photon birefringence and storage-ring experiments. Additionally, we conceptually propose a novel photon-ring experiment for axion detection. Furthermore, we demonstrate that measuring the axion-induced Berry phase provides a unique way for probing the global structure of the Standard Model (SM) gauge group and axion-related generalized symmetries., Comment: 9 pages, 2 figures

We introduce Agent K v1.0, an end-to-end autonomous data science agent designed to automate, optimise, and generalise across diverse data science tasks. Fully automated, Agent K v1.0 manages the entire data science life cycle by learning from experience. It leverages a highly flexible structured reasoning framework to enable it to dynamically process memory in a nested structure, effectively learning from accumulated experience stored to handle complex reasoning tasks. It optimises long- and short-term memory by selectively storing and retrieving key information, guiding future decisions based on environmental rewards. This iterative approach allows it to refine decisions without fine-tuning or backpropagation, achieving continuous improvement through experiential learning. We evaluate our agent's apabilities using Kaggle competitions as a case study. Following a fully automated protocol, Agent K v1.0 systematically addresses complex and multimodal data science tasks, employing Bayesian optimisation for hyperparameter tuning and feature engineering. Our new evaluation framework rigorously assesses Agent K v1.0's end-to-end capabilities to generate and send submissions starting from a Kaggle competition URL. Results demonstrate that Agent K v1.0 achieves a 92.5\% success rate across tasks, spanning tabular, computer vision, NLP, and multimodal domains. When benchmarking against 5,856 human Kaggle competitors by calculating Elo-MMR scores for each, Agent K v1.0 ranks in the top 38\%, demonstrating an overall skill level comparable to Expert-level users. Notably, its Elo-MMR score falls between the first and third quartiles of scores achieved by human Grandmasters. Furthermore, our results indicate that Agent K v1.0 has reached a performance level equivalent to Kaggle Grandmaster, with a record of 6 gold, 3 silver, and 7 bronze medals, as defined by Kaggle's progression system.

A comprehensive framework of modeling covariance in angular streaking experiments is presented. Within the impulsive streaking regime, the displacement of electron momentum distribution (MD) provides a tight connection between the dressing-free MD and the dressed MD. Such connection establishes universal structures in the composition of streaking covariance that are common across different MDs, regardless of their exact shape. Building on this robust framework, we have developed methods for retrieving temporal information from angular streaking measurements. By providing a detailed understanding of the covariance structure in angular streaking experiments, our work enables more accurate and robust temporal measurements in a wide range of experimental scenarios., Comment: 22 pages, 13 figures

Large Language Models (LLMs) represent a revolutionary advancement in the contemporary landscape of artificial general intelligence (AGI). As exemplified by ChatGPT, LLM-based applications necessitate minimal response latency and maximal throughput for inference serving. However, due to the unpredictability of LLM execution, the first-come-first-serve (FCFS) scheduling policy employed by current LLM serving systems suffers from head-of-line (HoL) blocking issues and long job response times. In this paper, we propose a new efficient LLM inference serving framework, named ALISE. The key design paradigm of ALISE is to leverage a novel speculative scheduler by estimating the execution time for each job and exploiting such prior knowledge to assign appropriate job priority orders, thus minimizing potential queuing delays for heterogeneous workloads. Furthermore, to mitigate the memory overhead of the intermediate key-value (KV) cache, we employ a priority-based adaptive memory management protocol and quantization-based compression techniques. Evaluations demonstrate that in comparison to the state-of-the-art solution vLLM, ALISE improves the throughput of inference serving by up to 1.8x and 2.1x under the same latency constraint on the Alpaca and ShareGPT datasets, respectively., Comment: ICCAD 2024

Utilizing a range of techniques including multi-band light curves, softness ratio analysis, structure functions, rms spectra, cross-correlation functions, and ratios of spectra from different intervals, we present a comprehensive study of the complex X-ray spectral variability in Seyfert 1 galaxy Ark 120, through re-analyzing its six XMM-Newton observations taken between 2003 and 2014. We find a clear ''softer-when-brighter" trend in the 2--10 keV power-law component over long timescales, with this trend being timescale dependent, as it is much weaker on shorter timescales, similar to that previously detected in NGC 4051. Notably, a rare ''harder-when-brighter" trend is observed during one exposure, indicating dynamic changes in the spectral variability behavior of the power-law component. This exceptional exposure, with the spectral variability indeed marked by a power-law pivoting at an unusually low energy of ~ 2 keV, suggests intricate variations in the thermal Comptonization processes within the corona. Furthermore, when the data below 2 keV are included, we identify that the soft excess component adds significant complexity to the spectral variability, such as evidenced by a transition from ''harder-when-brighter'' to ''softer-when-brighter'' during another single exposure. Such extra complexity arises because the variability of the soft excess sometimes follows and sometimes does not follow the changes in the power-law component. Our findings underscore the necessity of applying multiple analytic techniques to fully capture the multifaceted spectral variability of AGNs., Comment: 13 pages, 11 figures, submitted. Comments are very welcome!

Developing algorithms to differentiate between machine-generated texts and human-written texts has garnered substantial attention in recent years. Existing methods in this direction typically concern an offline setting where a dataset containing a mix of real and machine-generated texts is given upfront, and the task is to determine whether each sample in the dataset is from a large language model (LLM) or a human. However, in many practical scenarios, sources such as news websites, social media accounts, or on other forums publish content in a streaming fashion. Therefore, in this online scenario, how to quickly and accurately determine whether the source is an LLM with strong statistical guarantees is crucial for these media or platforms to function effectively and prevent the spread of misinformation and other potential misuse of LLMs. To tackle the problem of online detection, we develop an algorithm based on the techniques of sequential hypothesis testing by betting that not only builds upon and complements existing offline detection techniques but also enjoys statistical guarantees, which include a controlled false positive rate and the expected time to correctly identify a source as an LLM. Experiments were conducted to demonstrate the effectiveness of our method.

In this paper, we establish the partial order (POs) for both the binary erasure channel (BEC) and the binary memoryless symmetric channel (BMSC) under any block rate-matched polar codes. Firstly, we define the POs in the sense of rate-matched polar codes as a sequential block version. Furthermore, we demonstrate the persistence of POs after block rate matching in the BEC. Finally, leveraging the existing POs in the BEC, we obtain more POs in the BMSC under block rate matching. Simulations show that the PW sequence constructed from \beta-expansion can be improved by the tool of POs. Actually, any fixed reliable sequence in the mother polar codes can be improved by POs for rate matching., Comment: 8 pages, 2 figures, 1 table

By pretraining on trillions of tokens, an LLM gains the capability of text generation. However, to enhance its utility and reduce potential harm, SFT and alignment are applied sequentially to the pretrained model. Due to the differing nature and objective functions of SFT and alignment, catastrophic forgetting has become a significant issue. To address this, we introduce Unified Fine-Tuning (UFT), which integrates SFT and alignment into a single training stage using the same objective and loss functions through an implicit reward function. Our experimental results demonstrate that UFT outperforms SFT on instruction-tuning data alone. Moreover, when combining instruction-tuning data with alignment data, UFT effectively prevents catastrophic forgetting across these two stages and shows a clear advantage over sequentially applying SFT and alignment. This is evident in the significant improvements observed in the \textbf{ifeval} task for instruction-following and the \textbf{truthful-qa} task for factuality. The proposed general fine-tuning framework UFT establishes an effective and efficient pretraining-UFT paradigm for LLM training.

The weight spectrum plays a crucial role in the performance of error-correcting codes. Despite substantial theoretical exploration into polar codes with mother code length, a framework for the weight spectrum of rate-compatible polar codes remains elusive. In this paper, we address this gap by enumerating the number of minimum-weight codewords for quasi-uniform punctured, Wang-Liu shortened, and bit-reversal shortened polar codes. Additionally, we propose efficient algorithms for computing the average spectrum of random upper-triangular pre-transformed shortened and punctured polar codes. Notably, our algorithms operate with polynomial complexity relative to the code length. Simulation results affirm that our findings can substantially enhance the practical construction of rate-compatible polar codes, and leading to an improved weight spectrum.

This paper introduces a novel mobile phone control architecture, termed ``app agents", for efficient interactions and controls across various Android apps. The proposed Lightweight Multi-modal App Control (LiMAC) takes as input a textual goal and a sequence of past mobile observations, such as screenshots and corresponding UI trees, to generate precise actions. To address the computational constraints inherent to smartphones, within LiMAC, we introduce a small Action Transformer (AcT) integrated with a fine-tuned vision-language model (VLM) for real-time decision-making and task execution. We evaluate LiMAC on two open-source mobile control datasets, demonstrating the superior performance of our small-form-factor approach against fine-tuned versions of open-source VLMs, such as Florence2 and Qwen2-VL. It also significantly outperforms prompt engineering baselines utilising closed-source foundation models like GPT-4o. More specifically, LiMAC increases the overall action accuracy by up to 19% compared to fine-tuned VLMs, and up to 42% compared to prompt-engineering baselines.

The success of autoregressive (AR) language models in text generation has inspired the computer vision community to adopt Large Language Models (LLMs) for image generation. However, considering the essential differences between text and image modalities, the design space of language models for image generation remains underexplored. We observe that image tokens exhibit greater randomness compared to text tokens, which presents challenges when training with token prediction. Nevertheless, AR models demonstrate their potential by effectively learning patterns even from a seemingly suboptimal optimization problem. Our analysis also reveals that while all models successfully grasp the importance of local information in image generation, smaller models struggle to capture the global context. In contrast, larger models showcase improved capabilities in this area, helping to explain the performance gains achieved when scaling up model size. We further elucidate the design space of language models for vision generation, including tokenizer choice, model choice, model scalability, vocabulary design, and sampling strategy through extensive comparative experiments. Our work is the first to analyze the optimization behavior of language models in vision generation, and we believe it can inspire more effective designs when applying LMs to other domains. Finally, our elucidated language model for image generation, termed as ELM, achieves state-of-the-art performance on the ImageNet 256*256 benchmark. The code is available at https://github.com/Pepperlll/LMforImageGeneration.git., Comment: Project page: https://pepper-lll.github.io/LMforImageGeneration/

Smartphone agents are increasingly important for helping users control devices efficiently, with (Multimodal) Large Language Model (MLLM)-based approaches emerging as key contenders. Fairly comparing these agents is essential but challenging, requiring a varied task scope, the integration of agents with different implementations, and a generalisable evaluation pipeline to assess their strengths and weaknesses. In this paper, we present SPA-Bench, a comprehensive SmartPhone Agent Benchmark designed to evaluate (M)LLM-based agents in an interactive environment that simulates real-world conditions. SPA-Bench offers three key contributions: (1) A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines; (2) A plug-and-play framework enabling real-time agent interaction with Android devices, integrating over ten agents with the flexibility to add more; (3) A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption. Our extensive experiments across tasks and agents reveal challenges like interpreting mobile user interfaces, action grounding, memory retention, and execution costs. We propose future research directions to ease these difficulties, moving closer to real-world smartphone agent applications.

On-device control agents, especially on mobile devices, are responsible for operating mobile devices to fulfill users' requests, enabling seamless and intuitive interactions. Integrating Multimodal Large Language Models (MLLMs) into these agents enhances their ability to understand and execute complex commands, thereby improving user experience. However, fine-tuning MLLMs for on-device control presents significant challenges due to limited data availability and inefficient online training processes. This paper introduces DistRL, a novel framework designed to enhance the efficiency of online RL fine-tuning for mobile device control agents. DistRL employs centralized training and decentralized data acquisition to ensure efficient fine-tuning in the context of dynamic online interactions. Additionally, the framework is backed by our tailor-made RL algorithm, which effectively balances exploration with the prioritized utilization of collected data to ensure stable and robust training. Our experiments show that, on average, DistRL delivers a 3X improvement in training efficiency and enables training data collection 2.4X faster than the leading synchronous multi-machine methods. Notably, after training, DistRL achieves a 20% relative improvement in success rate compared to state-of-the-art methods on general Android tasks from an open benchmark, significantly outperforming existing approaches while maintaining the same training time. These results validate DistRL as a scalable and efficient solution, offering substantial improvements in both training efficiency and agent performance for real-world, in-the-wild device control tasks., Comment: Paper and Appendix, 25 pages

In this paper, we study the system-level advantages provided by rateless coding, early termination and power allocation strategy for multiple users distributed across multiple cells. In a multi-cell scenario, the early termination of coded transmission not only reduces finite-length loss akin to the single-user scenario but also yields capacity enhancements due to the cancellation of interference across cells. We term this technique \emph{coded water-filling}, a concept that diverges from traditional water-filling by incorporating variable-length rateless coding and interference cancellation. We formulate a series of analytical models to quantify the gains associated with coded water-filling in multi-user scenarios. First, we analyze the capacity gains from interference cancellation in Additive White Gaussian Noise (AWGN) channels, which arises from the disparity in the number of bits transmitted by distinct users. Building upon this, we broaden our analysis to encompass fading channels to show the robustness of the interference cancellation algorithms. Finally, we address the power allocation problem analogous to the water-filling problem under a multi-user framework, proving that an elevation in the water-filling threshold facilitates overall system capacity enhancement. Our analysis reveals the capacity gains achievable through early termination and power allocation techniques in multi-user settings. These results show that coded water-filling is instrumental for further improving spectral efficiency in crowded spectrums.

Federated learning (FL) is a promising machine learning paradigm that collaborates with client models to capture global knowledge. However, deploying FL models in real-world scenarios remains unreliable due to the coexistence of in-distribution data and unexpected out-of-distribution (OOD) data, such as covariate-shift and semantic-shift data. Current FL researches typically address either covariate-shift data through OOD generalization or semantic-shift data via OOD detection, overlooking the simultaneous occurrence of various OOD shifts. In this work, we propose FOOGD, a method that estimates the probability density of each client and obtains reliable global distribution as guidance for the subsequent FL process. Firstly, SM3D in FOOGD estimates score model for arbitrary distributions without prior constraints, and detects semantic-shift data powerfully. Then SAG in FOOGD provides invariant yet diverse knowledge for both local covariate-shift generalization and client performance generalization. In empirical validations, FOOGD significantly enjoys three main advantages: (1) reliably estimating non-normalized decentralized distributions, (2) detecting semantic shift data via score values, and (3) generalizing to covariate-shift data by regularizing feature extractor. The prejoct is open in https://github.com/XeniaLLL/FOOGD-main.git., Comment: NeurIPS 2024

This paper addresses the problem of learning optimal control policies for systems with uncertain dynamics and high-level control objectives specified as Linear Temporal Logic (LTL) formulas. Uncertainty is considered in the workspace structure and the outcomes of control decisions giving rise to an unknown Markov Decision Process (MDP). Existing reinforcement learning (RL) algorithms for LTL tasks typically rely on exploring a product MDP state-space uniformly (using e.g., an $\epsilon$-greedy policy) compromising sample-efficiency. This issue becomes more pronounced as the rewards get sparser and the MDP size or the task complexity increase. In this paper, we propose an accelerated RL algorithm that can learn control policies significantly faster than competitive approaches. Its sample-efficiency relies on a novel task-driven exploration strategy that biases exploration towards directions that may contribute to task satisfaction. We provide theoretical analysis and extensive comparative experiments demonstrating the sample-efficiency of the proposed method. The benefit of our method becomes more evident as the task complexity or the MDP size increases., Comment: arXiv admin note: text overlap with arXiv:2205.04424

The inter-band lags among the optical broad-band continua of active galactic nuclei (AGNs) have been intensively explored over the past decade. However, the nature of the lags remains under debate. Here utilizing two distinct scenarios for AGN variability, i.e., the thermal fluctuation of accretion disk and the reprocessing of both the accretion disk and clouds in the broad line region, we show that, owing to the random nature of AGN variability, the inter-band lags of an individual AGN would vary from one campaign with a finite baseline to another. Specifically, the thermal fluctuation scenario implies larger variations in the lags than the reprocessing scenario. Moreover, the former predicts a positive correlation between the lag and variation amplitude, while the latter does not result in such a correlation. For both scenarios, averaging the lags of an individual AGN measured with repeated and non-overlapping campaigns would give rise to a stable lag, which is larger for a longer baseline and gets saturation for a sufficiently long baseline. However, obtaining the stable lag for an individual AGN is very time-consuming. Alternatively, it can be equivalently inferred by averaging the lags of a sample of AGNs with similar physical properties, thus can be properly compared with predictions of AGN models. In addition, discussed are several new observational tests suggested by our simulations as well as the role of the deep high-cadence surveys of the Wide Field Survey Telescope in enriching our knowledge of the lags., Comment: 16 pages, 10 figures. Accepted for publication in Astrophysical Journal, comments are welcome!

In this technical report, we introduce OpenR, an open-source framework designed to integrate key components for enhancing the reasoning capabilities of large language models (LLMs). OpenR unifies data acquisition, reinforcement learning training (both online and offline), and non-autoregressive decoding into a cohesive software platform. Our goal is to establish an open-source platform and community to accelerate the development of LLM reasoning. Inspired by the success of OpenAI's o1 model, which demonstrated improved reasoning abilities through step-by-step reasoning and reinforcement learning, OpenR integrates test-time compute, reinforcement learning, and process supervision to improve reasoning in LLMs. Our work is the first to provide an open-source framework that explores the core techniques of OpenAI's o1 model with reinforcement learning, achieving advanced reasoning capabilities beyond traditional autoregressive methods. We demonstrate the efficacy of OpenR by evaluating it on the MATH dataset, utilising publicly available data and search methods. Our initial experiments confirm substantial gains, with relative improvements in reasoning and performance driven by test-time computation and reinforcement learning through process reward models. The OpenR framework, including code, models, and datasets, is accessible at https://openreasoner.github.io.

In sequential decision-making (SDM) tasks, methods like reinforcement learning (RL) and heuristic search have made notable advances in specific cases. However, they often require extensive exploration and face challenges in generalizing across diverse environments due to their limited grasp of the underlying decision dynamics. In contrast, large language models (LLMs) have recently emerged as powerful general-purpose tools, due to their capacity to maintain vast amounts of domain-specific knowledge. To harness this rich prior knowledge for efficiently solving complex SDM tasks, we propose treating LLMs as prior action distributions and integrating them into RL frameworks through Bayesian inference methods, making use of variational inference and direct posterior sampling. The proposed approaches facilitate the seamless incorporation of fixed LLM priors into both policy-based and value-based RL frameworks. Our experiments show that incorporating LLM-based action priors significantly reduces exploration and optimization complexity, substantially improving sample efficiency compared to traditional RL techniques, e.g., using LLM priors decreases the number of required samples by over 90% in offline learning scenarios.

This study explores a relationship between the CO luminosity-full width at half-maximum linewidth linear relation (i.e. the CO LFR) and mean galaxy property of the local star-forming galaxy sample in the xCOLDGASS data base, via a mathematical statement. The whole data base galaxies were separated into two subsamples based on their stellar masses and redshifts, being a help to examine the dependence issue of the CO LFR. Selecting the galaxy data with a stringent requirement was also implemented in order to assure the validly of the CO LFR. An algorithm of the linear regression was conducted with the data of the subsample. An assessment about the linear correlation manifested a valid CO LFR occurs in the selected galaxy of the subsample, and the intercept of the CO LFR may be related with the mean galaxy properties such as the molecular gas fraction and galaxy size. For the finding on the intercept of the CO LFR, we aligned that intercept with those galaxy properties via the involvement of a $\psi$ parameter. On evaluating the $\psi$ value with our local star-forming galaxy sample, we numerically determined a relationship between the statistical result and the galaxy property in a different stellar mass range. It also shows a possible method on estimating galaxy property., Comment: 10 pages, 3 tables, 2 figures; we sincerely appreciate the suggestion of the referee and the acceptance of the MNRAS. Y-HW is greatly grateful to all the co-authors for their works on his articles

An important line of research in the field of explainability is to extract a small subset of crucial rationales from the full input. The most widely used criterion for rationale extraction is the maximum mutual information (MMI) criterion. However, in certain datasets, there are spurious features non-causally correlated with the label and also get high mutual information, complicating the loss landscape of MMI. Although some penalty-based methods have been developed to penalize the spurious features (e.g., invariance penalty, intervention penalty, etc) to help MMI work better, these are merely remedial measures. In the optimization objectives of these methods, spurious features are still distinguished from plain noise, which hinders the discovery of causal rationales. This paper aims to develop a new criterion that treats spurious features as plain noise, allowing the model to work on datasets rich in spurious features as if it were working on clean datasets, thereby making rationale extraction easier. We theoretically observe that removing either plain noise or spurious features from the input does not alter the conditional distribution of the remaining components relative to the task label. However, significant changes in the conditional distribution occur only when causal features are eliminated. Based on this discovery, the paper proposes a criterion for \textbf{M}aximizing the \textbf{R}emaining \textbf{D}iscrepancy (MRD). Experiments on six widely used datasets show that our MRD criterion improves rationale quality (measured by the overlap with human-annotated rationales) by up to $10.4\%$ as compared to several recent competitive MMI variants. Code: \url{https://github.com/jugechengzi/Rationalization-MRD}., Comment: Accepted at NeurIPS 2024. arXiv admin note: text overlap with arXiv:2309.13391

Time series forecasting is extensively applied across diverse domains. Transformer-based models demonstrate significant potential in modeling cross-time and cross-variable interaction. However, we notice that the cross-variable correlation of multivariate time series demonstrates multifaceted (positive and negative correlations) and dynamic progression over time, which is not well captured by existing Transformer-based models. To address this issue, we propose a TimeCNN model to refine cross-variable interactions to enhance time series forecasting. Its key innovation is timepoint-independent, where each time point has an independent convolution kernel, allowing each time point to have its independent model to capture relationships among variables. This approach effectively handles both positive and negative correlations and adapts to the evolving nature of variable relationships over time. Extensive experiments conducted on 12 real-world datasets demonstrate that TimeCNN consistently outperforms state-of-the-art models. Notably, our model achieves significant reductions in computational requirements (approximately 60.46%) and parameter count (about 57.50%), while delivering inference speeds 3 to 4 times faster than the benchmark iTransformer model

Preference Optimization (PO) has proven an effective step for aligning language models to human-desired behaviors. Current variants, following the offline Direct Preference Optimization objective, have focused on a strict setting where all tokens are contributing signals of KL divergence and rewards to the loss function. However, human preference is not affected by each word in a sequence equally but is often dependent on specific words or phrases, e.g. existence of toxic terms leads to non-preferred responses. Based on this observation, we argue that not all tokens should be weighted equally during PO and propose a flexible objective termed SparsePO, that aims to automatically learn to weight the KL divergence and reward corresponding to each token during PO training. We propose two different variants of weight-masks that can either be derived from the reference model itself or learned on the fly. Notably, our method induces sparsity in the learned masks, allowing the model to learn how to best weight reward and KL divergence contributions at the token level, learning an optimal level of mask sparsity. Extensive experiments on multiple domains, including sentiment control, dialogue, text summarization and text-to-code generation, illustrate that our approach assigns meaningful weights to tokens according to the target task, generates more responses with the desired preference and improves reasoning tasks by up to 2 percentage points compared to other token- and response-level PO methods., Comment: 20 pages, 9 figures, 5 tables. Under Review

Large language models have demonstrated impressive value in performing as autonomous agents when equipped with external tools and API calls. Nonetheless, effectively harnessing their potential for executing complex tasks crucially relies on enhancements in their function calling capabilities. This paper identifies a critical gap in existing function calling models, where performance varies significantly across benchmarks, often due to being misled by specific naming conventions. To address such an issue, we introduce Hammer, a novel family of foundation models specifically engineered for on-device function calling. Hammer employs an augmented dataset that enhances models' sensitivity to irrelevant functions and incorporates function masking techniques to minimize misleading. Our empirical evaluations reveal that Hammer not only outperforms larger models but also demonstrates robust generalization across diverse benchmarks, achieving sota results. Our open source contributions include a specialized dataset for irrelevance detection, a tuning framework for enhanced generalization, and the Hammer models, establishing a new standard for function calling performance.

With the rapid advancement of large language models (LLMs), recent years have witnessed many promising studies on leveraging LLM-based agents to simulate human social behavior. While prior work has demonstrated significant potential across various domains, much of it has focused on specific scenarios involving a limited number of agents and has lacked the ability to adapt when errors occur during simulation. To overcome these limitations, we propose a novel LLM-agent-based simulation platform called \textit{GenSim}, which: (1) \textbf{Abstracts a set of general functions} to simplify the simulation of customized social scenarios; (2) \textbf{Supports one hundred thousand agents} to better simulate large-scale populations in real-world contexts; (3) \textbf{Incorporates error-correction mechanisms} to ensure more reliable and long-term simulations. To evaluate our platform, we assess both the efficiency of large-scale agent simulations and the effectiveness of the error-correction mechanisms. To our knowledge, GenSim represents an initial step toward a general, large-scale, and correctable social simulation platform based on LLM agents, promising to further advance the field of social science.

The growth in the number of parameters of Large Language Models (LLMs) has led to a significant surge in computational requirements, making them challenging and costly to deploy. Speculative decoding (SD) leverages smaller models to efficiently propose future tokens, which are then verified by the LLM in parallel. Small models that utilise activations from the LLM currently achieve the fastest decoding speeds. However, we identify several limitations of SD models including the lack of on-policyness during training and partial observability. To address these shortcomings, we propose a more grounded architecture for small models by introducing a Mixture of Attentions for SD. Our novel architecture can be applied in two scenarios: a conventional single device deployment and a novel client-server deployment where the small model is hosted on a consumer device and the LLM on a server. In a single-device scenario, we demonstrate state-of-the-art speedups improving EAGLE-2 by 9.5% and its acceptance length by 25%. In a client-server setting, our experiments demonstrate: 1) state-of-the-art latencies with minimal calls to the server for different network conditions, and 2) in the event of a complete disconnection, our approach can maintain higher accuracy compared to other SD methods and demonstrates advantages over API calls to LLMs, which would otherwise be unable to continue the generation process.

The multimodal model has demonstrated promise in histopathology. However, most multimodal models are based on H\&E and genomics, adopting increasingly complex yet black-box designs. In our paper, we propose a novel interpretable multimodal framework named SHAP-CAT, which uses a Shapley-value-based dimension reduction technique for effective multimodal fusion. Starting with two paired modalities -- H\&E and IHC images, we employ virtual staining techniques to enhance limited input data by generating a new clinical-related modality. Lightweight bag-level representations are extracted from image modalities and a Shapley-value-based mechanism is used for dimension reduction. For each dimension of the bag-level representation, attribution values are calculated to indicate how changes in the specific dimensions of the input affect the model output. In this way, we select a few top important dimensions of bag-level representation for each image modality to late fusion. Our experimental results demonstrate that the proposed SHAP-CAT framework incorporating synthetic modalities significantly enhances model performance, yielding a 5\% increase in accuracy for the BCI, an 8\% increase for IHC4BC-ER, and an 11\% increase for the IHC4BC-PR dataset.

The scaling relation of central massive black holes (MBHs) and their host galaxies is well-studied for supermassive BHs (SMBHs, $M_{\rm BH}\ \ge 10^6\, M_{\rm \odot}$). However, this relation has large uncertainties in the mass range of the intermediate-mass BHs (IMBHs, $M_{\rm BH}\ \sim10^3-10^{6}\, M_{\rm \odot}$). Since Green Pea (GP) galaxies are luminous compact dwarf galaxies, which may be likely to host less massive SMBHs or even IMBHs, we systematically search for MBHs in a large sample of 2190 GP galaxies at $z < 0.4$, selected from LAMOST and SDSS spectroscopic surveys. Here, we report a newly discovered sample of 59 MBH candidates with broad H$\alpha$ lines. This sample has a median stellar mass of $10^{8.83\pm0.11}\, M_{\rm \odot}$ and hosts MBHs with single-epoch virial masses ranging from $M_{\rm BH}\ \sim 10^{4.7}$ to $10^{8.5}\, M_{\rm \odot}$ (median $10^{5.85\pm0.64}\, M_{\rm \odot}$). Among the 59 MBH candidates, 36 have black hole masses $M_{\rm BH} \le 10^{6}\, M_{\rm \odot}$ (IMBH candidates), one of which even has $M_{\rm BH} \ \lesssim 10^{5}\, M_{\rm \odot}$. We find that the $M_{\rm BH}-M_{\rm *}$ relation of our MBH sample is consistent with the $M_{\rm BH}-M_{\rm bulge}$ relation for SMBHs, while is above the $M_{\rm BH}-M_{\rm *}$ relation for MBHs in dwarf galaxies in the same mass range. Furthermore, we show that 25 MBH candidates, including 4 IMBH candidates, have additional evidence of black hole activities, assessed through various methods such as the broad-line width, BPT diagram, mid-infrared color, X-ray luminosity, and radio emission. Our studies show that it is very promising to find IMBHs in GP galaxies, and the BH sample so obtained enables us to probe the connection between the MBHs and compact dwarf galaxies in the low-redshift Universe., Comment: 17 pages, 8 figures, 2 tables; Accepted for pubulication in SCPMA

We report a new high-sensitivity HI mapping observation of the NGC 5055 galaxy group over an area of $1.^\circ5\times0.^\circ75$ with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Our observation reveals that the warped H\,{\sc i} disk of NGC~5055 is more extended than what previously observed by WSRT, out to $ 23.'9$ (61.7 kpc). The total HI mass of NGC 5055 is determined to be $\rm\sim 1.1\times10^{10}\,M_\odot$. We identified three HI clouds with HI masses of the order of $\rm \sim 10^7\,M_\odot$ at the southeastern edge of the HI disk, as well as a candidate high-velocity cloud with an HI mass of $\rm (1.2\pm0.5) \times10^6\,M_\odot$ to the north of NGC 5055. The HI content of UGCA 337 is robustly detected for the first time by the FAST observations. It has a narrow HI linewidth of $W_{50}=17.4\pm3.8$ km s$^{-1}$ with a total \HI\ mass of ($\rm 3.5\pm0.3)\times10^6\,M_\odot$. Comparing the gas content and g-r color of UGCA 337 with typical low-mass dwarf galaxies, UGCA~337 appears relatively gas-poor despite its blue color. This suggests that UGCA 337 may have undergone gas stripping in the past. We also analyzed the possible origin of the diffuse HI clouds located at the outskirts of NGC 5055, and speculate that they might be the remnant features of a merger event in the past., Comment: 10 pages, 6 figures

Cosmological first-order phase transitions (FOPTs) serve as comprehensive probes into our early Universe with associated generations of stochastic gravitational waves and superhorizon curvature perturbations or even primordial black holes. In characterizing the FOPT, phenomenological parameters like transition temperatures, strength factors, bubble separations, and energy budgets can be easily extracted from the macroscopic equilibrium features of the underlying particle physics models except for the terminal wall velocity of the bubble expansion, making it the last key parameter to be determined most difficultly due to the non-equilibrium nature of the microscopic transition model. In this paper, we propose a new model-independent approach to calculate the bubble wall velocity by virtue of an extra junction condition from the conservation and violation of the total number density current across the shock front (if any) and bubble wall, respectively., Comment: v2, 43 pages, 7 figures, major revision, a proof is added for the collision integral to vanish in local dynamical equilibrium, the necessity is introduced for the scalar-wall contribution to the total number density current, the clarification is emphasized for the conservation and violation of total number density current across the shock front and bubble wall, respectively

Cross-Domain Recommendation (CDR) have received widespread attention due to their ability to utilize rich information across domains. However, most existing CDR methods assume an ideal static condition that is not practical in industrial recommendation systems (RS). Therefore, simply applying existing CDR methods in the industrial RS environment may lead to low effectiveness and efficiency. To fill this gap, we propose DIIT, an end-to-end Domain-Invariant Information Transfer method for industrial cross-domain recommendation. Specifically, We first simulate the industrial RS environment that maintains respective models in multiple domains, each of them is trained in the incremental mode. Then, for improving the effectiveness, we design two extractors to fully extract domain-invariant information from the latest source domain models at the domain level and the representation level respectively. Finally, for improving the efficiency, we design a migrator to transfer the extracted information to the latest target domain model, which only need the target domain model for inference. Experiments conducted on one production dataset and two public datasets verify the effectiveness and efficiency of DIIT., Comment: Accepted at CIKM 2024

Unpredictable and complex aerodynamic effects pose significant challenges to achieving precise flight control, such as the downwash effect from upper vehicles to lower ones. Conventional methods often struggle to accurately model these interactions, leading to controllers that require large safety margins between vehicles. Moreover, the controller on real drones usually requires high-frequency and has limited on-chip computation, making the adaptive control design more difficult to implement. To address these challenges, we incorporate Gaussian process (GP) to model the adaptive external aerodynamics with linear model predictive control. The GP is linearized to enable real-time high-frequency solutions. Moreover, to handle the error caused by linearization, we integrate end-to-end Bayesian optimization during sample collection stages to improve the control performance. Experimental results on both simulations and real quadrotors show that we can achieve real-time solvable computation speed with acceptable tracking errors.

In recent years, Large Language Models (LLMs) have gained widespread use, raising concerns about their security. Traditional jailbreak attacks, which often rely on the model internal information or have limitations when exploring the unsafe behavior of the victim model, limiting their reducing their general applicability. In this paper, we introduce PathSeeker, a novel black-box jailbreak method, which is inspired by the game of rats escaping a maze. We think that each LLM has its unique "security maze", and attackers attempt to find the exit learning from the received feedback and their accumulated experience to compromise the target LLM's security defences. Our approach leverages multi-agent reinforcement learning, where smaller models collaborate to guide the main LLM in performing mutation operations to achieve the attack objectives. By progressively modifying inputs based on the model's feedback, our system induces richer, harmful responses. During our manual attempts to perform jailbreak attacks, we found that the vocabulary of the response of the target model gradually became richer and eventually produced harmful responses. Based on the observation, we also introduce a reward mechanism that exploits the expansion of vocabulary richness in LLM responses to weaken security constraints. Our method outperforms five state-of-the-art attack techniques when tested across 13 commercial and open-source LLMs, achieving high attack success rates, especially in strongly aligned commercial models like GPT-4o-mini, Claude-3.5, and GLM-4-air with strong safety alignment. This study aims to improve the understanding of LLM security vulnerabilities and we hope that this sturdy can contribute to the development of more robust defenses., Comment: update the abstract and cite a new related work

In this paper, we study the following biharmonic Schr\"odinger equation with potential and mixed nonlinearities \begin{equation*} \left\{\begin{array}{ll}\Delta^2 u +V(x,y)u+\lambda u =\mu|u|^{p-2}u+|u|^{q-2}u,\ (x, y) \in \Omega_r \times \mathbb{T}^n, \\ \int_{\Omega_r\times\mathbb{T}^n}u^2dxdy=\Theta,\end{array} \right. \end{equation*} where $\Omega_r \subset \mathbb{R}^d$ is an open bounded convex domain, $r>0$ is large and $\mu\in\mathbb{R}$. The exponents satisfy $2

The $I=2$ $\pi\pi$ elastic $s$-wave scattering phase shift is measured by lattice QCD with $N_f=3$ flavors of the Asqtad-improved staggered fermions. The lattice-calculated energy-eigenvalues of $\pi\pi$ systems at one center of mass frame and some moving frames using the moving wall source technique are utilized to secure phase shifts by L\"uscher's formula. Our computations are fine enough to obtain threshold parameters: scattering length $a$, effective range $r$, and shape parameter $P$, which can be extrapolated at the physical point by NLO in chiral perturbation theory, and our relevant NNLO predictions from expanding NPLQCD's works are novelly considered as the systematic uncertainties. Our outcomes are consistent with Roy equation determinations, newer experimental data, and lattice estimations. Numerical computations are performed with a coarse ($a\approx0.12$~fm, $L^3 T = 32^3 64$), two fine ($a\approx0.09$~fm, $L^3 T = 40^3 96$) and a superfine ($a\approx0.06$~fm, $L^3 T = 48^3 144$) lattice ensembles at four pion masses of $m_\pi\sim247~{\rm MeV}$, $249~{\rm MeV}$, $275~{\rm MeV}$, and $384~{\rm MeV}$, respectively., Comment: 8 figures

To help address the occlusion problem in panoptic segmentation and image understanding, this paper proposes a new large-scale dataset, COCO-Occ, which is derived from the COCO dataset by manually labelling the COCO images into three perceived occlusion levels. Using COCO-Occ, we systematically assess and quantify the impact of occlusion on panoptic segmentation on samples having different levels of occlusion. Comparative experiments with SOTA panoptic models demonstrate that the presence of occlusion significantly affects performance with higher occlusion levels resulting in notably poorer performance. Additionally, we propose a straightforward yet effective method as an initial attempt to leverage the occlusion annotation using contrastive learning to render a model that learns a more robust representation capturing different severities of occlusion. Experimental results demonstrate that the proposed approach boosts the performance of the baseline model and achieves SOTA performance on the proposed COCO-Occ dataset.

Our research introduces an innovative framework for video-to-audio synthesis, which solves the problems of audio-video desynchronization and semantic loss in the audio. By incorporating a semantic alignment adapter and a temporal synchronization adapter, our method significantly improves semantic integrity and the precision of beat point synchronization, particularly in fast-paced action sequences. Utilizing a contrastive audio-visual pre-trained encoder, our model is trained with video and high-quality audio data, improving the quality of the generated audio. This dual-adapter approach empowers users with enhanced control over audio semantics and beat effects, allowing the adjustment of the controller to achieve better results. Extensive experiments substantiate the effectiveness of our framework in achieving seamless audio-visual alignment.

In the realm of Large Language Models (LLMs), the ability to process long contexts is increasingly crucial for tasks such as multi-round dialogues, code generation, and document summarization. This paper addresses the challenges of enhancing the long-context performance, reducing computational complexity, and leveraging pretrained models collectively termed the "impossible triangle." We introduce E2LLM (Encoder Elongated Large Language Models), a novel approach that effectively navigates this paradox. The method involves splitting long contexts into chunks, compressing each into embedding vectors via a pretrained text encoder, and utilizing an adapter to align these representations with a decoder-only LLM. Two training objectives, focusing on reconstruction of the encoder output and long-context instruction fine-tuning, are employed to facilitate the understanding of soft prompts by the LLM. Experimental results demonstrate that E2LLM achieves superior performance in long-context scenarios while balancing efficiency, performance, and compatibility with pretrained models. Our framework thus represents a significant advancement in the field, contributing to effective long-text modeling., Comment: 12 pages, 4 figures

In recent years, there has been significant progress in Text-to-Speech (TTS) synthesis technology, enabling the high-quality synthesis of voices in common scenarios. In unseen situations, adaptive TTS requires a strong generalization capability to speaker style characteristics. However, the existing adaptive methods can only extract and integrate coarse-grained timbre or mixed rhythm attributes separately. In this paper, we propose AS-Speech, an adaptive style methodology that integrates the speaker timbre characteristics and rhythmic attributes into a unified framework for text-to-speech synthesis. Specifically, AS-Speech can accurately simulate style characteristics through fine-grained text-based timbre features and global rhythm information, and achieve high-fidelity speech synthesis through the diffusion model. Experiments show that the proposed model produces voices with higher naturalness and similarity in terms of timbre and rhythm compared to a series of adaptive TTS models., Comment: Accepted by SLT 2024

In this paper, we introduced the Unified Three-Form Dark Sector (UTFDS) model, a unified dark sector model that combines dark energy and dark matter through a three-form field. In this framework, the potential of the three-form field acts as dark matter, while the kinetic term represents dark energy. The interaction between dark matter and dark energy is driven by the energy exchange between these two terms. Given the dynamical equations of UTFDS, we provide an autonomous system of evolution equations for UTFDS and perform a stability analysis of its fixed points. The result aligns with our expectations for a unified dark sector. Furthermore, we discover that the dual Lagrangian of the UTFDS Lagrangian is equivalent to a Dirac-Born-Infeld (DBI) Lagrangian. By fixing the parameter $\kappa X_0$ to 250, 500, 750, we refer to the resulting models as the $\overline{\rm UTFDS}$ model with $\kappa X_0$=250, 500, 750, respectively. We then place constraints on these three $\overline{\rm UTFDS}$ models and the $\Lambda$CDM model in light of the Planck 2018 Cosmic Microwave Background (CMB) anisotropies, Redshift Space Distortions (RSD) observations, Baryon Acoustic Oscillation (BAO) measurements, and the $S_8$ prior chosen according to the KiDS1000 Weak gravitational Lensing (WL) measuement. We find that the $\overline{\rm UTFDS}$ model with $\kappa X_0$=500 is the only one among the four models where both $\sigma_8$ and $S_8$ tensions, between CMB and RSD+BAO+WL datasets, are below 2.0$\sigma$. Furthermore, the tensions are relieved without exacerbating the $H_0$ tension. Although both the CMB and RSD+BAO+WL datasets provide definite/positive evidence favoring $\Lambda$CDM over the $\overline{\rm UTFDS}$ model with $\kappa X_0$=500, the evidence is not strong enough to rule out further study of this model., Comment: 16 pages, 6 tables, 5 figures

Let $E$ be an elliptic curve defined over $\mathbb{Q}$ with good ordinary reduction at a prime $p\geq 5$, and let $F$ be an imaginary quadratic field. Under appropriate assumptions, we show that the Pontryagin dual of the fine Mordell-Weil group of $E$ over the $\mathbb{Z}_p^2$-extension of $F$ is pseudo-null as a module over the Iwasawa algebra of the group $\mathbb{Z}_p^2$.

Sparse and noisy images (SNIs), like those in spatial gene expression data, pose significant challenges for effective representation learning and clustering, which are essential for thorough data analysis and interpretation. In response to these challenges, we propose Dual Advancement of Representation Learning and Clustering (DARLC), an innovative framework that leverages contrastive learning to enhance the representations derived from masked image modeling. Simultaneously, DARLC integrates cluster assignments in a cohesive, end-to-end approach. This integrated clustering strategy addresses the "class collision problem" inherent in contrastive learning, thus improving the quality of the resulting representations. To generate more plausible positive views for contrastive learning, we employ a graph attention network-based technique that produces denoised images as augmented data. As such, our framework offers a comprehensive approach that improves the learning of representations by enhancing their local perceptibility, distinctiveness, and the understanding of relational semantics. Furthermore, we utilize a Student's t mixture model to achieve more robust and adaptable clustering of SNIs. Extensive experiments, conducted across 12 different types of datasets consisting of SNIs, demonstrate that DARLC surpasses the state-of-the-art methods in both image clustering and generating image representations that accurately capture gene interactions. Code is available at https://github.com/zipging/DARLC.