Search

Recent advancements in Direct Preference Optimization (DPO) have significantly enhanced the alignment of Large Language Models (LLMs) with human preferences, owing to its simplicity and effectiveness. However, existing methods typically optimize a scalar score or ranking reward, thereby overlooking the multi-dimensional nature of human preferences. In this work, we propose to extend the preference of DPO to two dimensions: segments and aspects. We first introduce a 2D supervision dataset called HelpSteer-2D. For the segment dimension, we divide the response into sentences and assign scores to each segment. For the aspect dimension, we meticulously design several criteria covering the response quality rubrics. With the 2-dimensional signals as feedback, we develop a 2D-DPO framework, decomposing the overall objective into multi-segment and multi-aspect objectives. Extensive experiments on popular benchmarks demonstrate that 2D-DPO performs better than methods that optimize for scalar or 1-dimensional preferences., Comment: The first four authors contributed equally, 25 pages

The Chasing All Transients Constellation Hunters (CATCH) space mission is focused on exploring the dynamic universe via X-ray follow-up observations of various transients. The first pathfinder of the CATCH mission, CATCH-1, was launched on June 22, 2024, alongside the Space-based multiband astronomical Variable Objects Monitor (SVOM) mission. CATCH-1 is equipped with narrow-field optimized Micro Pore Optics (MPOs) featuring a large effective area and incorporates four Silicon Drift Detectors (SDDs) in its focal plane. This paper presents the system calibration results conducted before the satellite integration. Utilizing the data on the performance of the mirror and detectors obtained through the system calibration, combined with simulated data, the ground calibration database can be established. Measuring the relative positions of the mirror and detector system, which were adjusted during system calibration, allows for accurate installation of the entire satellite. Furthermore, the paper outlines the operational workflow of the ground network post-satellite launch.

The success of large language models (LLMs) has prompted efforts to integrate speech and audio data, aiming to create general foundation models capable of processing both textual and non-textual inputs. Recent advances, such as GPT-4o, highlight the potential for end-to-end speech LLMs, which preserves non-semantic information and world knowledge for deeper speech understanding. To guide the development of speech LLMs, we propose a five-level roadmap, ranging from basic automatic speech recognition (ASR) to advanced superhuman models capable of integrating non-semantic information with abstract acoustic knowledge for complex tasks. Moreover, we design a benchmark, SAGI Bechmark, that standardizes critical aspects across various tasks in these five levels, uncovering challenges in using abstract acoustic knowledge and completeness of capability. Our findings reveal gaps in handling paralinguistic cues and abstract acoustic knowledge, and we offer future directions. This paper outlines a roadmap for advancing speech LLMs, introduces a benchmark for evaluation, and provides key insights into their current limitations and potential.

This work demonstrates a novel energy-efficient tunnel FET (TFET)-CMOS hybrid foundry platform for ultralow-power AIoT applications. By utilizing the proposed monolithic integration process, the novel complementary n and p-type Si TFET technology with dopant segregated source junction and self-aligned drain underlap design is successfully integrated into a 300mm CMOS baseline process without CMOS performance penalty and any new materials, experimentally demonstrating the large Ion and record high Ion/Ioff ratio of 10^7 among TFETs by industry-manufacturers. The device performance and variability are also co-optimized for high-volume production. Further circuit-level implementations are presented based on the calibrated compact model. The proposed TFET-CMOS hybrid logic and SRAM topologies show significant energy efficiency improvement with comparable operation speed compared with standard CMOS circuits, indicating its great potential for power-constraint AIoT applications.

Recent progress has shown that the generalization error of the Gibbs algorithm can be exactly characterized using the symmetrized KL information between the learned hypothesis and the entire training dataset. However, evaluating such a characterization is cumbersome, as it involves a high-dimensional information measure. In this paper, we address this issue by considering individual sample information measures within the Gibbs algorithm. Our main contribution lies in establishing the asymptotic equivalence between the sum of symmetrized KL information between the output hypothesis and individual samples and that between the hypothesis and the entire dataset. We prove this by providing explicit expressions for the gap between these measures in the non-asymptotic regime. Additionally, we characterize the asymptotic behavior of various information measures in the context of the Gibbs algorithm, leading to tighter generalization error bounds. An illustrative example is provided to verify our theoretical results, demonstrating our analysis holds in broader settings., Comment: IEEE Information Theory Workshop

We propose DOME, a diffusion-based world model that predicts future occupancy frames based on past occupancy observations. The ability of this world model to capture the evolution of the environment is crucial for planning in autonomous driving. Compared to 2D video-based world models, the occupancy world model utilizes a native 3D representation, which features easily obtainable annotations and is modality-agnostic. This flexibility has the potential to facilitate the development of more advanced world models. Existing occupancy world models either suffer from detail loss due to discrete tokenization or rely on simplistic diffusion architectures, leading to inefficiencies and difficulties in predicting future occupancy with controllability. Our DOME exhibits two key features:(1) High-Fidelity and Long-Duration Generation. We adopt a spatial-temporal diffusion transformer to predict future occupancy frames based on historical context. This architecture efficiently captures spatial-temporal information, enabling high-fidelity details and the ability to generate predictions over long durations. (2)Fine-grained Controllability. We address the challenge of controllability in predictions by introducing a trajectory resampling method, which significantly enhances the model's ability to generate controlled predictions. Extensive experiments on the widely used nuScenes dataset demonstrate that our method surpasses existing baselines in both qualitative and quantitative evaluations, establishing a new state-of-the-art performance on nuScenes. Specifically, our approach surpasses the baseline by 10.5% in mIoU and 21.2% in IoU for occupancy reconstruction and by 36.0% in mIoU and 24.6% in IoU for 4D occupancy forecasting., Comment: Please visit our project page at https://gusongen.github.io/DOME

A finite extension of global fields $L/K$ satisfies the Hasse norm principle if any nonzero element of $K$ has the property that it is a norm locally if and only if it is a norm globally. In 1931, Hasse proved that any cyclic extension satisfies the Hasse norm principle, providing a novel approach to extending the local-global principle to equations with degree greater than $2$. In this paper, we introduce the projective Hasse norm principle, generalizing the Hasse norm principle to multiple fields and asking whether a projective line that contains a norm locally in every field must also contain a norm globally in every field. We show that the projective Hasse norm principle is independent from the conjunction of Hasse norm principles in all of the constituent fields in the general case, but that the latter implies the former when the fields are all Galois and independent. We also prove an analogue of the Hasse norm theorem for the projective Hasse norm theorem, namely that the projective Hasse norm principle holds in all cyclic extensions., Comment: 25 pages. Comments are welcome

This paper introduces a novel method for locking the repetition frequency of pulse lasers, adaptable to different frequencies,offering significant improvements in system integration and measurement accuracy. The method consists of two primary components: an error amplification module (EAM) and a digital frequency locking module (DFLM) based on FPGA. The EAM integrates a configurable frequency generator (CFG), a configurable frequency multiplier (CFM) and a mixer,to process the laser pulse alongside a high-stability reference source, such as an atomic clock. By employing frequency multiplication and mixing, the EAM amplifies the laser's frequency error and performs frequency down-conversion,enhancing measurement sensitivity and reducing the hardware requirements of the back-end.The CFG, implemented on a phase-locked loop (PLL) chip, allows for parameter adjustments to accommodate various laser frequencies.The DFLM processes the output from the EAM using a high-speed, ADC-based dual-mixer time-difference (DMTD) method to precisely measure frequency errors.A digital proportional-integral-derivative (PID) controller then provides feedback to achieve accurate frequency locking. To evaluate the proposed method, an FPGA-based electronic system was developed and tested. In laboratory experiment with a custom-built femtosecond fiber laser, the system demonstrated robust locking of the repetition rate, achieving an Allan deviation improvement from $1.51 \times 10^{-7}$ to $1.12 \times 10^{-12}$ at a gate time of 10 s.Further testing with a voltage-controlled oscillator (VCO) confirmed a long-term stability of $9.58 \times 10^{-14} @ 10 s$.

Bipartite graph embedding (BGE) maps nodes to compressed embedding vectors that can reflect the hidden topological features of the network, and learning high-quality BGE is crucial for facilitating downstream applications such as recommender systems. However, most existing methods either struggle to efficiently learn embeddings suitable for users and items with fewer interactions, or exhibit poor scalability to handle large-scale networks. In this paper, we propose a Clustering Constraints induced BIpartite graph Embedding (CCBIE) as an integrated solution to both problems. CCBIE facilitates automatic and dynamic soft clustering of items in a top-down manner, and capturing macro-preference information of users through clusters. Specifically, by leveraging the cluster embedding matrix of items, CCBIE calculates the cluster assignment matrix for items and also captures the extent of user preferences across different clusters, thereby elucidating the similarity between users and items on a macro-scale level. CCBIE effectively preserves the global properties of bipartite graphs, maintaining the cluster structure of isomorphic nodes and accounting for long-range dependencies among heterogeneous nodes. Our approach significantly enhances user-item collaborative relation modeling by integrating adaptive clustering for relationship learning, thereby markedly improving prediction performance, particularly benefiting cold users and items. Extensive experiments indicate that CCBIE consistently and significantly improves accuracy over baseline models, particularly on sparse graphs, while also enhancing training speed and reducing memory requirements on large-scale bipartite graphs., Comment: 10 pages, 7 figures

The increasing demand for versatile robotic systems to operate in diverse and dynamic environments has emphasized the importance of a generalist policy, which leverages a large cross-embodiment data corpus to facilitate broad adaptability and high-level reasoning. However, the generalist would struggle with inefficient inference and cost-expensive training. The specialist policy, instead, is curated for specific domain data and excels at task-level precision with efficiency. Yet, it lacks the generalization capacity for a wide range of applications. Inspired by these observations, we introduce RoboDual, a synergistic dual-system that supplements the merits of both generalist and specialist policy. A diffusion transformer-based specialist is devised for multi-step action rollouts, exquisitely conditioned on the high-level task understanding and discretized action output of a vision-language-action (VLA) based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in real-world setting and 12% gain on CALVIN by introducing a specialist policy with merely 20M trainable parameters. It maintains strong performance with 5% of demonstration data only, and enables a 3.8 times higher control frequency in real-world deployment. Code would be made publicly available. Our project page is hosted at: https://opendrivelab.com/RoboDual/, Comment: Project page: https://opendrivelab.com/RoboDual/

We study the problem of fairly allocating either a set of indivisible goods or a set of mixed divisible and indivisible goods (i.e., mixed goods) to agents with additive utilities, taking the best-of-both-worlds perspective of guaranteeing fairness properties both ex ante and ex post. The ex-post fairness notions considered in this paper are relaxations of envy-freeness, specifically, EFX for indivisible-goods allocation, and EFM for mixed-goods allocation. For two agents, we show that there is a polynomial-time randomized algorithm that achieves ex-ante envy-freeness and ex-post EFX / EFM simultaneously. For $n$ agents with bi-valued utilities, we show there exist randomized allocations that are (i) ex-ante proportional and ex-post EFM, and (ii) ex-ante envy-free, ex-post EFX, and ex-post fractionally Pareto optimal., Comment: Appears in the 20th Conference on Web and Internet Economics (WINE), 2024

The text-to-video (T2V) generation models, offering convenient visual creation, have recently garnered increasing attention. Despite their substantial potential, the generated videos may present artifacts, including structural implausibility, temporal inconsistency, and a lack of motion, often resulting in near-static video. In this work, we have identified a correlation between the disparity of temporal attention maps across different blocks and the occurrence of temporal inconsistencies. Additionally, we have observed that the energy contained within the temporal attention maps is directly related to the magnitude of motion amplitude in the generated videos. Based on these observations, we present BroadWay, a training-free method to improve the quality of text-to-video generation without introducing additional parameters, augmenting memory or sampling time. Specifically, BroadWay is composed of two principal components: 1) Temporal Self-Guidance improves the structural plausibility and temporal consistency of generated videos by reducing the disparity between the temporal attention maps across various decoder blocks. 2) Fourier-based Motion Enhancement enhances the magnitude and richness of motion by amplifying the energy of the map. Extensive experiments demonstrate that BroadWay significantly improves the quality of text-to-video generation with negligible additional cost.

Image watermark techniques provide an effective way to assert ownership, deter misuse, and trace content sources, which has become increasingly essential in the era of large generative models. A critical attribute of watermark techniques is their robustness against various manipulations. In this paper, we introduce a watermark removal approach capable of effectively nullifying the state of the art watermarking techniques. Our primary insight involves regenerating the watermarked image starting from a clean Gaussian noise via a controllable diffusion model, utilizing the extracted semantic and spatial features from the watermarked image. The semantic control adapter and the spatial control network are specifically trained to control the denoising process towards ensuring image quality and enhancing consistency between the cleaned image and the original watermarked image. To achieve a smooth trade-off between watermark removal performance and image consistency, we further propose an adjustable and controllable regeneration scheme. This scheme adds varying numbers of noise steps to the latent representation of the watermarked image, followed by a controlled denoising process starting from this noisy latent representation. As the number of noise steps increases, the latent representation progressively approaches clean Gaussian noise, facilitating the desired trade-off. We apply our watermark removal methods across various watermarking techniques, and the results demonstrate that our methods offer superior visual consistency/quality and enhanced watermark removal performance compared to existing regeneration approaches.

Employing grazing-incidence small-angle X-ray scattering (GISAXS) and X-ray reflectivity (XRR), we demonstrate that films composed of polyethylene glycol (PEG)-grafted silver nanoparticles (AgNP) and gold nanoparticles (AuNP), as well as their binary mixtures, form highly stable hexagonal structures at the vapor-liquid interface. These nanoparticles exhibit remarkable stability under varying environmental conditions, including changes in pH, mixing concentration, and PEG chain length. Short-chain PEG grafting produces dense, well-ordered films, while longer chains produce more complex, less dense quasi-bilayer structures. AuNPs exhibit higher grafting densities than AgNPs, leading to more ordered in-plane arrangements. In binary mixtures, AuNPs dominate the population at the surface, while AgNPs integrate into the system, expanding the lattice without forming a distinct binary superstructure. These results offer valuable insights into the structural behavior of PEG-grafted nanoparticles and provide a foundation for optimizing binary nanoparticle assemblies for advanced nanotechnology applications., Comment: 17 pages, 8 figures, 2 tables

Diffusion models have made substantial progress in facilitating image generation and editing. As the technology matures, we see its potential in the context of driving simulations to enhance the simulated experience. In this paper, we explore this potential through the introduction of a novel system designed to boost visual fidelity. Our system, DRIVE (Diffusion-based Realism Improvement for Virtual Environments), leverages a diffusion model pipeline to give a simulated environment a photorealistic view, with the flexibility to be adapted for other applications. We conducted a preliminary user study to assess the system's effectiveness in rendering realistic visuals and supporting participants in performing driving tasks. Our work not only lays the groundwork for future research on the integration of diffusion models in driving simulations but also provides practical guidelines and best practices for their application in this context.

Differential privacy (DP) offers a robust framework for safeguarding individual data privacy. To utilize DP in training modern machine learning models, differentially private optimizers have been widely used in recent years. A popular approach to privatize an optimizer is to clip the individual gradients and add sufficiently large noise to the clipped gradient. This approach led to the development of DP optimizers that have comparable performance with their non-private counterparts in fine-tuning tasks or in tasks with a small number of training parameters. However, a significant performance drop is observed when these optimizers are applied to large-scale training. This degradation stems from the substantial noise injection required to maintain DP, which disrupts the optimizer's dynamics. This paper introduces DiSK, a novel framework designed to significantly enhance the performance of DP optimizers. DiSK employs Kalman filtering, a technique drawn from control and signal processing, to effectively denoise privatized gradients and generate progressively refined gradient estimations. To ensure practicality for large-scale training, we simplify the Kalman filtering process, minimizing its memory and computational demands. We establish theoretical privacy-utility trade-off guarantees for DiSK, and demonstrate provable improvements over standard DP optimizers like DPSGD in terms of iteration complexity upper-bound. Extensive experiments across diverse tasks, including vision tasks such as CIFAR-100 and ImageNet-1k and language fine-tuning tasks such as GLUE, E2E, and DART, validate the effectiveness of DiSK. The results showcase its ability to significantly improve the performance of DP optimizers, surpassing state-of-the-art results under the same privacy constraints on several benchmarks.

Large Language Models (LLMs) boosts human efficiency but also poses misuse risks, with watermarking serving as a reliable method to differentiate AI-generated content from human-created text. In this work, we propose a novel theoretical framework for watermarking LLMs. Particularly, we jointly optimize both the watermarking scheme and detector to maximize detection performance, while controlling the worst-case Type-I error and distortion in the watermarked text. Within our framework, we characterize the universally minimum Type-II error, showing a fundamental trade-off between detection performance and distortion. More importantly, we identify the optimal type of detectors and watermarking schemes. Building upon our theoretical analysis, we introduce a practical, model-agnostic and computationally efficient token-level watermarking algorithm that invokes a surrogate model and the Gumbel-max trick. Empirical results on Llama-13B and Mistral-8$\times$7B demonstrate the effectiveness of our method. Furthermore, we also explore how robustness can be integrated into our theoretical framework, which provides a foundation for designing future watermarking systems with improved resilience to adversarial attacks.

Previous researches on the Information retrieval (IR) field have focused on summarizing progress and synthesizing knowledge and techniques from individual studies and data-driven experiments, the extent of contributions and collaborations between researchers from different communities (e.g., academia and industry) in advancing IR knowledge remains unclear. To address this gap, this study explores several characteristics of information retrieval research in four areas: productivity patterns and preferred venues, the relationship between citations and downloads, changes in research topics, and changes in patterns of scientific collaboration, by analyzing 53,471 papers published between 2000 and 2018 from the Association for Computing Machinery (ACM) Digital Library dataset. Through the analysis and interpretation on empirical datasets, we find that academic research, industry research, and collaborative research between academia and industry focused on different topics. Among the collaboration models, Academia-Industry Collaboration is more oriented towards large teamwork. Collaborative networks between researchers in academia and industry suggest that the field of information retrieval has become richer over time in terms of themes, foci, and sub-themes, becoming a more diverse field of study., Comment: 43 pages, 11 figures

This paper investigates the performance of several representative large models in the field of literature recommendation and explores potential biases. The results indicate that while some large models' recommendations can be somewhat satisfactory after simple manual screening, overall, the accuracy of these models in specific literature recommendation tasks is generally moderate. Additionally, the models tend to recommend literature that is timely, collaborative, and expands or deepens the field. In scholar recommendation tasks. There is no evidence to suggest that LLMs exacerbate inequalities related to gender, race, or the level of development of countries., Comment: There are some aspects in the original research that need to be supplemented with robustness checks

While the language modeling objective has been shown to be deeply connected with compression, it is surprising that modern LLMs are not employed in practical text compression systems. In this paper, we provide an in-depth analysis of neural network and transformer-based compression techniques to answer this question. We compare traditional text compression systems with neural network and LLM-based text compression methods. Although LLM-based systems significantly outperform conventional compression methods, they are highly impractical. Specifically, LLMZip, a recent text compression system using Llama3-8B requires 9.5 days to compress just 10 MB of text, although with huge improvements in compression ratios. To overcome this, we present FineZip - a novel LLM-based text compression system that combines ideas of online memorization and dynamic context to reduce the compression time immensely. FineZip can compress the above corpus in approximately 4 hours compared to 9.5 days, a 54 times improvement over LLMZip and comparable performance. FineZip outperforms traditional algorithmic compression methods with a large margin, improving compression ratios by approximately 50\%. With this work, we take the first step towards making lossless text compression with LLMs a reality. While FineZip presents a significant step in that direction, LLMs are still not a viable solution for large-scale text compression. We hope our work paves the way for future research and innovation to solve this problem.

Recently, large language model (LLM)-based agents have made significant advances across various fields. One of the most popular research areas involves applying these agents to video games. Traditionally, these methods have relied on game APIs to access in-game environmental and action data. However, this approach is limited by the availability of APIs and does not reflect how humans play games. With the advent of vision language models (VLMs), agents now have enhanced visual understanding capabilities, enabling them to interact with games using only visual inputs. Despite these advances, current approaches still face challenges in action-oriented tasks, particularly in action role-playing games (ARPGs), where reinforcement learning methods are prevalent but suffer from poor generalization and require extensive training. To address these limitations, we select an ARPG, ``Black Myth: Wukong'', as a research platform to explore the capability boundaries of existing VLMs in scenarios requiring visual-only input and complex action output. We define 12 tasks within the game, with 75% focusing on combat, and incorporate several state-of-the-art VLMs into this benchmark. Additionally, we will release a human operation dataset containing recorded gameplay videos and operation logs, including mouse and keyboard actions. Moreover, we propose a novel VARP (Vision Action Role-Playing) agent framework, consisting of an action planning system and a visual trajectory system. Our framework demonstrates the ability to perform basic tasks and succeed in 90% of easy and medium-level combat scenarios. This research aims to provide new insights and directions for applying multimodal agents in complex action game environments. The code and datasets will be made available at https://varp-agent.github.io/.

Large Language models have achieved impressive performance in automated software engineering. Extensive efforts have been made to evaluate the abilities of code LLMs in various aspects, with an increasing number of benchmarks and evaluation frameworks proposed. Apart from the most sought-after capability of code generation, the capability of code comprehension is being granted growing attention. Nevertheless, existing works assessing the code comprehension capability of LLMs exhibit varied limitations. Evaluation frameworks like CRUXEval and REval usually focus on code reasoning tasks over a certain input case, leading to a limited range of execution traces covered, resulting in a loss in code semantics examined and the inability to assess the comprehensive understanding of LLMs concerning the target program. To tackle the challenges above, we propose SpecEval, a novel black-box evaluation framework to evaluate code comprehension in LLMs via program specifications. Inspired by the idea that specifications can comprehensively articulate program behaviors concerning all possible execution traces, we employ formal specifications to represent program semantics and perform thorough evaluations. In particular, four specification-related tasks are designed to assess the capability of LLMs from basic to advanced levels. Moreover, counterfactual analysis is conducted to study the performance variance of LLMs under semantics-preserving perturbations, and progressive consistency analysis is performed to study the performance consistency of LLMs over a series of tasks with sequential dependence. Systematic experiments are conducted on six state-of-the-art LLMs. Experimental results present a below-satisfactory performance of LLMs on specification-related tasks, revealing the limitations of existing LLMs in articulating program semantics, underscoring future directions for enhancement.

Band gap energy of an organic molecule can be reduced by intermolecular interaction enhancement, and thus, certain polycyclic aromatic hydrocarbons (PAHs), which are insulators with wide band gaps, are expected to undergo insulator-metal transitions by simple compression. Such a pressure-induced electronic transition can be exploited to transform non-metallic organic materials into states featuring intriguing electronic characteristics such as high-temperature superconductivity. Numerous attempts have been made to metalize various small PAHs, but so far only pressure-induced amorphization well below the megabar region was observed. The wide band gap energy of the small PAHs and low chemical stability under simple compression are the bottlenecks. We have investigated the band gap energy evolution and the crystal structural compression of the large PAH molecules, where the band gap energy is significantly reduced by increasing the number of {\pi}-electrons and improved chemical stability with fully benzenoid molecular structure. Herein, we present a pressure-induced transition in dicoronylene, C48H20, an insulator at ambient conditions that transforms into a semi-metallic state above 23.0 GPa with a three-order-of-magnitude reduction in resistivity. In-situ UV-visible absorption, transport property measurement, Raman spectroscopy, X-ray diffraction and density functional theory calculations were performed to provide tentative explanations to the alterations in its electronic structure at high pressure. The discovery of an electronic transition at pressures well below the megabar is a promising step towards realization of a single component purely hydrocarbon molecular metal in the near future., Comment: 22 pages, 5 figures

Despite significant progress in robotics and embodied AI in recent years, deploying robots for long-horizon tasks remains a great challenge. Majority of prior arts adhere to an open-loop philosophy and lack real-time feedback, leading to error accumulation and undesirable robustness. A handful of approaches have endeavored to establish feedback mechanisms leveraging pixel-level differences or pre-trained visual representations, yet their efficacy and adaptability have been found to be constrained. Inspired by classic closed-loop control systems, we propose CLOVER, a closed-loop visuomotor control framework that incorporates feedback mechanisms to improve adaptive robotic control. CLOVER consists of a text-conditioned video diffusion model for generating visual plans as reference inputs, a measurable embedding space for accurate error quantification, and a feedback-driven controller that refines actions from feedback and initiates replans as needed. Our framework exhibits notable advancement in real-world robotic tasks and achieves state-of-the-art on CALVIN benchmark, improving by 8% over previous open-loop counterparts. Code and checkpoints are maintained at https://github.com/OpenDriveLab/CLOVER., Comment: Accepted at NeurIPS 2024. Code and models: https://github.com/OpenDriveLab/CLOVER

Quantifying uncertainty is crucial for robust and reliable predictions. However, existing spatiotemporal deep learning mostly focuses on deterministic prediction, overlooking the inherent uncertainty in such prediction. Particularly, highly-granular spatiotemporal datasets are often sparse, posing extra challenges in prediction and uncertainty quantification. To address these issues, this paper introduces a novel post-hoc Sparsity-awar Uncertainty Calibration (SAUC) framework, which calibrates uncertainty in both zero and non-zero values. To develop SAUC, we firstly modify the state-of-the-art deterministic spatiotemporal Graph Neural Networks (ST-GNNs) to probabilistic ones in the pre-calibration phase. Then we calibrate the probabilistic ST-GNNs for zero and non-zero values using quantile approaches.Through extensive experiments, we demonstrate that SAUC can effectively fit the variance of sparse data and generalize across two real-world spatiotemporal datasets at various granularities. Specifically, our empirical experiments show a 20\% reduction in calibration errors in zero entries on the sparse traffic accident and urban crime prediction. Overall, this work demonstrates the theoretical and empirical values of the SAUC framework, thus bridging a significant gap between uncertainty quantification and spatiotemporal prediction., Comment: Paper accepted by ACM SIGSPATIAL 2024

Strong self-interference due to the co-located transmitter is the bottleneck for implementing an in-band full-duplex (IBFD) system. If not adequately mitigated, the strong interference can saturate the receiver's analog-digital converters (ADCs) and hence void the digital processing. This paper considers utilizing a reconfigurable intelligent surface (RIS), together with a receiving (Rx) phase shifter network (PSN), to mitigate the strong self-interference through jointly optimizing their phases. This method, named self-interference mitigation using RIS and PSN (SIMRP), can suppress self-interference to avoid ADC saturation effectively and therefore improve the sum rate performance of communication systems, as verified by the simulation studies., Comment: 6 pages, 4 figures, accepted by IEEE WCSP 2024

We define fermionic convolution and demonstrate its utility in characterizing fermionic non-Gaussian components, which are essential to the computational advantage of fermionic systems. Using fermionic convolution, we propose an efficient protocol that tests the fermionic Gaussianity of pure states using three copies of the input state. We also introduce "Non-Gaussian Entropy," an experimentally accessible resource measure that quantifies fermionic non-Gaussianity. These results provide new insights into the study of fermionic quantum computation., Comment: 6 +21 pages

Soft manipulators are known for their superiority in coping with high-safety-demanding interaction tasks, e.g., robot-assisted surgeries, elderly caring, etc. Yet the challenges residing in real-time contact feedback have hindered further applications in precise manipulation. This paper proposes an end-to-end network to estimate the 3D contact force of the soft robot, with the aim of enhancing its capabilities in interactive tasks. The presented method features directly utilizing monocular images fused with multidimensional actuation information as the network inputs. This approach simplifies the preprocessing of raw data compared to related studies that utilize 3D shape information for network inputs, consequently reducing configuration reconstruction errors. The unified feature representation module is devised to elevate low-dimensional features from the system's actuation signals to the same level as image features, facilitating smoother integration of multimodal information. The proposed method has been experimentally validated in the soft robot testbed, achieving satisfying accuracy in 3D force estimation (with a mean relative error of 0.84% compared to the best-reported result of 2.2% in the related works).

End-to-end architectures in autonomous driving (AD) face a significant challenge in interpretability, impeding human-AI trust. Human-friendly natural language has been explored for tasks such as driving explanation and 3D captioning. However, previous works primarily focused on the paradigm of declarative interpretability, where the natural language interpretations are not grounded in the intermediate outputs of AD systems, making the interpretations only declarative. In contrast, aligned interpretability establishes a connection between language and the intermediate outputs of AD systems. Here we introduce Hint-AD, an integrated AD-language system that generates language aligned with the holistic perception-prediction-planning outputs of the AD model. By incorporating the intermediate outputs and a holistic token mixer sub-network for effective feature adaptation, Hint-AD achieves desirable accuracy, achieving state-of-the-art results in driving language tasks including driving explanation, 3D dense captioning, and command prediction. To facilitate further study on driving explanation task on nuScenes, we also introduce a human-labeled dataset, Nu-X. Codes, dataset, and models will be publicly available., Comment: CoRL 2024, Project Page: https://air-discover.github.io/Hint-AD/

Audio Event Recognition (AER) traditionally focuses on detecting and identifying audio events. Most existing AER models tend to detect all potential events without considering their varying significance across different contexts. This makes the AER results detected by existing models often have a large discrepancy with human auditory perception. Although this is a critical and significant issue, it has not been extensively studied by the Detection and Classification of Sound Scenes and Events (DCASE) community because solving it is time-consuming and labour-intensive. To address this issue, this paper introduces the concept of semantic importance in AER, focusing on exploring the differences between human perception and model inference. This paper constructs a Multi-Annotated Foreground Audio Event Recognition (MAFAR) dataset, which comprises audio recordings labelled by 10 professional annotators. Through labelling frequency and variance, the MAFAR dataset facilitates the quantification of semantic importance and analysis of human perception. By comparing human annotations with the predictions of ensemble pre-trained models, this paper uncovers a significant gap between human perception and model inference in both semantic identification and existence detection of audio events. Experimental results reveal that human perception tends to ignore subtle or trivial events in the event semantic identification, while model inference is easily affected by events with noises. Meanwhile, in event existence detection, models are usually more sensitive than humans., Comment: Dataset homepage: https://github.com/Voltmeter00/MAFAR

The core challenge in applying abstract interpretation lies in the configuration of abstraction and analysis strategies encoded by a large number of external parameters of static analysis tools. To attain low false-positive rates (i.e., accuracy) while preserving analysis efficiency, tuning the parameters heavily relies on expert knowledge and is thus difficult to automate. In this paper, we present a fully automated framework called Parf to adaptively tune the external parameters of abstract interpretation-based static analyzers. Parf models various types of parameters as random variables subject to probability distributions over latticed parameter spaces. It incrementally refines the probability distributions based on accumulated intermediate results generated by repeatedly sampling and analyzing, thereby ultimately yielding a set of highly accurate parameter settings within a given time budget. We have implemented Parf on top of Frama-C/Eva - an off-the-shelf open-source static analyzer for C programs - and compared it against the expert refinement strategy and Frama-C/Eva's official configurations over the Frama-C OSCS benchmark. Experimental results indicate that Parf achieves the lowest number of false positives on 34/37 (91.9%) program repositories with exclusively best results on 12/37 (32.4%) cases. In particular, Parf exhibits promising performance for analyzing complex, large-scale real-world programs.

The StutteringSpeech Challenge focuses on advancing speech technologies for people who stutter, specifically targeting Stuttering Event Detection (SED) and Automatic Speech Recognition (ASR) in Mandarin. The challenge comprises three tracks: (1) SED, which aims to develop systems for detection of stuttering events; (2) ASR, which focuses on creating robust systems for recognizing stuttered speech; and (3) Research track for innovative approaches utilizing the provided dataset. We utilizes an open-source Mandarin stuttering dataset AS-70, which has been split into new training and test sets for the challenge. This paper presents the dataset, details the challenge tracks, and analyzes the performance of the top systems, highlighting improvements in detection accuracy and reductions in recognition error rates. Our findings underscore the potential of specialized models and augmentation strategies in developing stuttered speech technologies., Comment: 8 pages, 2 figures, accepted by SLT 2024

Spiking Neural Networks (SNNs) have emerged as a promising substitute for Artificial Neural Networks (ANNs) due to their advantages of fast inference and low power consumption. However, the lack of efficient training algorithms has hindered their widespread adoption. Existing supervised learning algorithms for SNNs require significantly more memory and time than their ANN counterparts. Even commonly used ANN-SNN conversion methods necessitate re-training of ANNs to enhance conversion efficiency, incurring additional computational costs. To address these challenges, we propose a novel training-free ANN-SNN conversion pipeline. Our approach directly converts pre-trained ANN models into high-performance SNNs without additional training. The conversion pipeline includes a local-learning-based threshold balancing algorithm, which enables efficient calculation of the optimal thresholds and fine-grained adjustment of threshold value by channel-wise scaling. We demonstrate the scalability of our framework across three typical computer vision tasks: image classification, semantic segmentation, and object detection. This showcases its applicability to both classification and regression tasks. Moreover, we have evaluated the energy consumption of the converted SNNs, demonstrating their superior low-power advantage compared to conventional ANNs. Our training-free algorithm outperforms existing methods, highlighting its practical applicability and efficiency. This approach simplifies the deployment of SNNs by leveraging open-source pre-trained ANN models and neuromorphic hardware, enabling fast, low-power inference with negligible performance reduction.

The potential for higher-resolution image generation using pretrained diffusion models is immense, yet these models often struggle with issues of object repetition and structural artifacts especially when scaling to 4K resolution and higher. We figure out that the problem is caused by that, a single prompt for the generation of multiple scales provides insufficient efficacy. In response, we propose HiPrompt, a new tuning-free solution that tackles the above problems by introducing hierarchical prompts. The hierarchical prompts offer both global and local guidance. Specifically, the global guidance comes from the user input that describes the overall content, while the local guidance utilizes patch-wise descriptions from MLLMs to elaborately guide the regional structure and texture generation. Furthermore, during the inverse denoising process, the generated noise is decomposed into low- and high-frequency spatial components. These components are conditioned on multiple prompt levels, including detailed patch-wise descriptions and broader image-level prompts, facilitating prompt-guided denoising under hierarchical semantic guidance. It further allows the generation to focus more on local spatial regions and ensures the generated images maintain coherent local and global semantics, structures, and textures with high definition. Extensive experiments demonstrate that HiPrompt outperforms state-of-the-art works in higher-resolution image generation, significantly reducing object repetition and enhancing structural quality., Comment: https://liuxinyv.github.io/HiPrompt/

We describe the physical mechanisms of launching and acceleration of wind from an active galactic nucleus (AGN) accretion disk. We focus on the radiation line force driven and magnetic driven wind models, which operate on the accretion disk scale. We review the investigation histories of the two mechanisms and the most new results obtained recently. The ultra-fast outflows (UFOs) found in the hard X-ray bands are believed to directly originate from AGN accretion disks. We review the theoretical works applying the two mechanisms of wind to explain the UFOs. We briefly introduce the propagation of winds on a large scale which is important for AGN wind feedback study. Finally, the roles of wind in AGN feedback are briefly reviewed., Comment: 21 pages, 2 figures, invited review chapter for the book "New Frontiers in GRMHD Simulations"

An integral domain $R$ is called atomic if every nonzero nonunit of $R$ factors into irreducibles, while $R$ satisfies the ascending chain condition on principal ideals if every ascending chain of principal ideals of $R$ stabilizes. It is well known and not hard to verify that if an integral domain satisfies the ACCP, then it must be atomic. The converse does not hold in general, but examples are hard to come by and most of them are the result of crafty and technical constructions. Sporadic constructions of such atomic domains have appeared in the literature in the last five decades, including the first example of a finite-dimensional atomic monoid algebra not satisfying the ACCP recently constructed by the second and third authors. Here we construct the first known one-dimensional monoid algebras satisfying the almost ACCP but not the ACCP (the almost ACCP is a notion weaker than the ACCP but still stronger than atomicity). Although the two constructions we provide here are rather technical, the corresponding monoid algebras are perhaps the most elementary known examples of atomic domains not satisfying the ACCP., Comment: 31 pages

Identifying structures in common forms the basis for networked systems design and optimization. However, real structures represented by graphs are often of varying sizes, leading to the low accuracy of traditional graph classification methods. These graphs are called cross-scale graphs. To overcome this limitation, in this study, we propose GSpect, an advanced spectral graph filtering model for cross-scale graph classification tasks. Compared with other methods, we use graph wavelet neural networks for the convolution layer of the model, which aggregates multi-scale messages to generate graph representations. We design a spectral-pooling layer which aggregates nodes to one node to reduce the cross-scale graphs to the same size. We collect and construct the cross-scale benchmark data set, MSG (Multi Scale Graphs). Experiments reveal that, on open data sets, GSpect improves the performance of classification accuracy by 1.62% on average, and for a maximum of 3.33% on PROTEINS. On MSG, GSpect improves the performance of classification accuracy by 15.55% on average. GSpect fills the gap in cross-scale graph classification studies and has potential to provide assistance in application research like diagnosis of brain disease by predicting the brain network's label and developing new drugs with molecular structures learned from their counterparts in other systems.

The limit of energy saving in the control of small systems has recently attracted much interest due to the concept refinement of the Maxwell demon. Inspired by a newly proposed set of fluctuation theorems, we report the first experimental verification of these equalities and inequalities in a ultracold 40Ca ion system, confirming the intrinsic nonequilibrium in the system due to involvement of the demon. Based on elaborately designed demon-involved control protocols, such as the Szilard engine protocol, we provide experimentally quantitative evidence of the dissipative information, and observe tighter bounds of both the extracted work and the demon's efficacy than the limits predicted by the Sagawa-Ueda theorem. Our results substantiate a close connection between the physical nature of information and nonequilibrium processes at the microscale, which help further understanding the thermodynamic characteristics of information and the optimal design of nanoscale and smaller systems.

Recent advancements in deep learning techniques have sparked performance boosts in various real-world applications including disease diagnosis based on multi-modal medical data. Cough sound data-based respiratory disease (e.g., COVID-19 and Chronic Obstructive Pulmonary Disease) diagnosis has also attracted much attention. However, existing works usually utilise traditional machine learning or deep models of moderate scales. On the other hand, the developed approaches are trained and evaluated on small-scale data due to the difficulty of curating and annotating clinical data on scale. To address these issues in prior works, we create a unified framework to evaluate various deep models from lightweight Convolutional Neural Networks (e.g., ResNet18) to modern vision transformers and compare their performance in respiratory disease classification. Based on the observations from such an extensive empirical study, we propose a novel approach to cough-based disease classification based on both self-supervised and supervised learning on a large-scale cough data set. Experimental results demonstrate our proposed approach outperforms prior arts consistently on two benchmark datasets for COVID-19 diagnosis and a proprietary dataset for COPD/non-COPD classification with an AUROC of 92.5%.

In the formalism of the non-supersymmetric modular invariance approach to the flavour problem the elements of the Yukawa coupling and fermion mass matrices are expressed in terms of polyharmonic Maa{\ss} modular forms of level $N$ in addition to the standard modula forms of the same level and a small number of constant parameters. Non-trivial polyharmonic Maa{\ss} forms exist for zero, negative and positive integer modular weights. Employing the finite modula group $S_4$ as a flavour symmetry group and assuming that the three left-handed lepton doublets furnish a triplet irreducible representation of $S_4$, we construct all possible 7- and 8-parameter lepton flavour models in which the neutrino masses are generated either by the Weinberg effective operator or by the type I seesaw mechanism. We identify the phenomenologically viable models and obtain predictions for each of these models for the neutrino mass ordering, the absolute neutrino mass scale, the Dirac and Majorana CP-violation phases and, correspondingly, for the sum of neutrino masses and the neutrinoless double beta decay effective Majorana mass. We comment on how these models can be tested and conclude that they are all falsifiable. Detailed analyses are presented in the case of three representative benchmark lepton flavour scenarios., Comment: 59 pages, 7 figures

The singly charged $SU(2)_L$ singlet scalar is one of the very interesting new particles, as it can generate neutrino masses at loop level, produce contributions to various flavour observables. We study the possibility of detecting this kind of scalar predicted by the singly-charged scalar model at ILC via the lepton flavour violation (LFV) process $e^+e^-\rightarrow S^+S^-\rightarrow \mu e + {E\mkern-10.5 mu/}$. Considering the constraints on the free parameters, we obtain the expected sensitivities of the ILC with the center of mass energy $\sqrt{s}=1~\mathrm{TeV}$ and the integrated luminosity $\mathcal{L}=$ $1.5~\mathrm{ab}^{-1}$ to the parameter space of the singly-charged scalar model. The prospective excluded mass range at $95\%$ C.L. is $M_S \gtrsim 470~\mathrm{GeV}$, $410~\mathrm{GeV}$ for the branching ratio $\mathcal{B}_{\mu e}$ = $100\%$ , $50\%$, respectively, while the scalar with $M_S \gtrsim 300~\mathrm{GeV}$ is excluded at $95\%$ C.L. for $\mathcal{B}_{\mu e}$ = $30\%$.

Privacy is a growing concern in modern deep-learning systems and applications. Differentially private (DP) training prevents the leakage of sensitive information in the collected training data from the trained machine learning models. DP optimizers, including DP stochastic gradient descent (DPSGD) and its variants, privatize the training procedure by gradient clipping and DP noise injection. However, in practice, DP models trained using DPSGD and its variants often suffer from significant model performance degradation. Such degradation prevents the application of DP optimization in many key tasks, such as foundation model pretraining. In this paper, we provide a novel signal processing perspective to the design and analysis of DP optimizers. We show that a ``frequency domain'' operation called low-pass filtering can be used to effectively reduce the impact of DP noise. More specifically, by defining the ``frequency domain'' for both the gradient and differential privacy (DP) noise, we have developed a new component, called DOPPLER. This component is designed for DP algorithms and works by effectively amplifying the gradient while suppressing DP noise within this frequency domain. As a result, it maintains privacy guarantees and enhances the quality of the DP-protected model. Our experiments show that the proposed DP optimizers with a low-pass filter outperform their counterparts without the filter by 3%-10% in test accuracy on various models and datasets. Both theoretical and practical evidence suggest that the DOPPLER is effective in closing the gap between DP and non-DP training.

Optimizing the size and depth of CNOT circuits is an active area of research in quantum computing and is particularly relevant for circuits synthesized from the Clifford + T universal gate set. Although many techniques exist for finding short syntheses, it is difficult to assess how close to optimal these syntheses are without an exponential brute-force search. We use a novel method of categorizing CNOT gates in a synthesis to obtain a strict lower bound computable in $O(n^{\omega})$ time on the minimum number of gates needed to synthesize a given CNOT circuit, where $\omega$ denotes the matrix multiplication constant and $n$ is the number of qubits involved. Applying our framework, we prove that $3(n-1)$ gate syntheses of the $n$-cycle circuit are optimal and provide insight into their structure. We also generalize this result to permutation circuits. For linear reversible circuits with $ n = 3, 4, 5$ qubits, our lower bound is optimal for 100%, 67.7%, and 23.1% of circuits and is accurate to within one CNOT gate in 100%, 99.5%, and 83.0% of circuits respectively. We also introduce an algorithm that efficiently determines whether certain circuits can be synthesized with fewer than $n$ CNOT gates., Comment: 14 pages, 12 figures