Search

Our work tackles the challenge of securing user inputs in cloud-based large language model (LLM) services while ensuring output consistency, model confidentiality, and compute efficiency. We introduce Secure Multi-party Decoding (SMD), which leverages confidential computing to confine user prompts to a trusted execution environment, namely a confidential virtual machine (CVM), while allowing service providers to generate tokens efficiently. We also introduce a novel cryptographic method, Prompt Obfuscation (PO), to ensure robustness against reconstruction attacks on SMD. We demonstrate that our approach preserves both prompt confidentiality and LLM serving efficiency. Our solution can enable privacy-preserving cloud LLM services that handle sensitive prompts, such as clinical records, financial data, and personal information.

Text-to-image synthesis has become highly popular for generating realistic and stylized images, often requiring fine-tuning generative models with domain-specific datasets for specialized tasks. However, these valuable datasets face risks of unauthorized usage and unapproved sharing, compromising the rights of the owners. In this paper, we address the issue of dataset abuse during the fine-tuning of Stable Diffusion models for text-to-image synthesis. We present a dataset watermarking framework designed to detect unauthorized usage and trace data leaks. The framework employs two key strategies across multiple watermarking schemes and is effective for large-scale dataset authorization. Extensive experiments demonstrate the framework's effectiveness, minimal impact on the dataset (only 2% of the data required to be modified for high detection accuracy), and ability to trace data leaks. Our results also highlight the robustness and transferability of the framework, proving its practical applicability in detecting dataset abuse.

The medium-temperature $T$ dependence of the jet transport coefficient $\hat q$ was studied via the nuclear modification factor $R_{AA}(p_{\rm T})$ and elliptical flow parameter $v_2(p_{\rm T})$ for large transverse momentum $p_{\rm T}$ hadrons in high-energy nucleus-nucleus collisions. Within a next-to-leading-order perturbative QCD parton model for hard scatterings with modified fragmentation functions due to jet quenching controlled by $\hat q$, we check the suppression and azimuthal anisotropy for large $p_{\rm T}$ hadrons, and extract $\hat q$ by global fits to $R_{AA}(p_{\rm T})$ and $v_2(p_{\rm T})$ data in A + A collisions at RHIC and LHC, respectively. The numerical results from the best fits show that $\hat q/T^3$ goes down with local medium temperature $T$ in the parton jet trajectory. Compared with the case of a constant $\hat{q}/T^3$, the going-down $T$ dependence of $\hat{q}/T^3$ makes a hard parton jet to lose more energy near $T_{\rm c}$ and therefore strengthens the azimuthal anisotropy for large $p_{\rm T}$ hadrons. As a result, $v_2(p_{\rm T})$ for large $p_{\rm T}$ hadrons was enhanced by approximately 10% to better fit the data at RHIC/LHC. Considering the first-order phase transition from QGP to the hadron phase and the additional energy loss in the hadron phase, $v_2(p_{\rm T})$ is again enhanced by 5%-10% at RHIC/LHC.

In everyday life, frequently used objects like cups often have unfixed positions and multiple instances within the same category, and their carriers frequently change as well. As a result, it becomes challenging for a robot to efficiently navigate to a specific instance. To tackle this challenge, the robot must capture and update scene changes and plans continuously. However, current object navigation approaches primarily focus on semantic-level and lack the ability to dynamically update scene representation. This paper captures the relationships between frequently used objects and their static carriers. It constructs an open-vocabulary Carrier-Relationship Scene Graph (CRSG) and updates the carrying status during robot navigation to reflect the dynamic changes of the scene. Based on the CRSG, we further propose an instance navigation strategy that models the navigation process as a Markov Decision Process. At each step, decisions are informed by Large Language Model's commonsense knowledge and visual-language feature similarity. We designed a series of long-sequence navigation tasks for frequently used everyday items in the Habitat simulator. The results demonstrate that by updating the CRSG, the robot can efficiently navigate to moved targets. Additionally, we deployed our algorithm on a real robot and validated its practical effectiveness., Comment: Project website: https://openobject-nav.github.io/

Artificial intelligence is gradually demonstrating its immense potential, and increasing attention is being given to how AI can be harnessed to advance scientific research. In this vision paper, we present our perspectives on how AI can better assist scientific inquiry and explore corresponding technical approach. We have proposed and open-sourced a large model of our KALE-LM model series, Llama3-KALE-LM-Chem-8B, which has achieved outstanding performance in tasks related to the field of chemistry. We hope that our work serves as a strong starting point, helping to realize more intelligent AI and promoting the advancement of human science and technology, as well as societal development.

Quantum annealers offer a promising approach to solve Quadratic Unconstrained Binary Optimization (QUBO) problems, which have a wide range of applications. However, when a user submits its QUBO problem to a third-party quantum annealer, the problem itself may disclose the user's private information to the quantum annealing service provider. To mitigate this risk, we introduce a privacy-preserving QUBO framework and propose a novel solution method. Our approach employs a combination of digit-wise splitting and matrix permutation to obfuscate the QUBO problem's model matrix $Q$, effectively concealing the matrix elements. In addition, based on the solution to the obfuscated version of the QUBO problem, we can reconstruct the solution to the original problem with high accuracy. Theoretical analysis and empirical tests confirm the efficacy and efficiency of our proposed technique, demonstrating its potential for preserving user privacy in quantum annealing services., Comment: 8 pages, 3 figures, QCE 2024

This comprehensive study evaluates the performance of OpenAI's o1-preview large language model across a diverse array of complex reasoning tasks, spanning multiple domains, including computer science, mathematics, natural sciences, medicine, linguistics, and social sciences. Through rigorous testing, o1-preview demonstrated remarkable capabilities, often achieving human-level or superior performance in areas ranging from coding challenges to scientific reasoning and from language processing to creative problem-solving. Key findings include: -83.3% success rate in solving complex competitive programming problems, surpassing many human experts. -Superior ability in generating coherent and accurate radiology reports, outperforming other evaluated models. -100% accuracy in high school-level mathematical reasoning tasks, providing detailed step-by-step solutions. -Advanced natural language inference capabilities across general and specialized domains like medicine. -Impressive performance in chip design tasks, outperforming specialized models in areas such as EDA script generation and bug analysis. -Remarkable proficiency in anthropology and geology, demonstrating deep understanding and reasoning in these specialized fields. -Strong capabilities in quantitative investing. O1 has comprehensive financial knowledge and statistical modeling skills. -Effective performance in social media analysis, including sentiment analysis and emotion recognition. The model excelled particularly in tasks requiring intricate reasoning and knowledge integration across various fields. While some limitations were observed, including occasional errors on simpler problems and challenges with certain highly specialized concepts, the overall results indicate significant progress towards artificial general intelligence.

The stable periodic patterns present in time series data serve as the foundation for conducting long-horizon forecasts. In this paper, we pioneer the exploration of explicitly modeling this periodicity to enhance the performance of models in long-term time series forecasting (LTSF) tasks. Specifically, we introduce the Residual Cycle Forecasting (RCF) technique, which utilizes learnable recurrent cycles to model the inherent periodic patterns within sequences, and then performs predictions on the residual components of the modeled cycles. Combining RCF with a Linear layer or a shallow MLP forms the simple yet powerful method proposed in this paper, called CycleNet. CycleNet achieves state-of-the-art prediction accuracy in multiple domains including electricity, weather, and energy, while offering significant efficiency advantages by reducing over 90% of the required parameter quantity. Furthermore, as a novel plug-and-play technique, the RCF can also significantly improve the prediction accuracy of existing models, including PatchTST and iTransformer. The source code is available at: https://github.com/ACAT-SCUT/CycleNet.

A two-dimensional (2D) Dirac semimetal with concomitant superconductivity has been long sought but rarely reported. It is believed that light-element materials have the potential to realize this goal owing to their intrinsic lightweight and metallicity. Here, based on the recently synthesized $\beta_{12}$ hydrogenated borophene [Science 371, 1143 (2021)], we investigate its counterpart named $\beta_{12}$-$ \rm {B_5H_3}$. Our first-principles calculations suggest it has good stability. $\beta_{12}$-$ \rm {B_5H_3}$ is a scarce Dirac semimetal demonstrating a strain-tunable phase transition from three Dirac cones to a single Dirac cone. Additionally, $\beta_{12}$-$ \rm {B_5H_3}$ is also a superior phonon-mediated superconductor with a superconducting critical temperature of 32.4 K and can be further boosted to 42 K under external strain. The concurrence of Dirac fermions and superconductivity, supplemented with dual tunabilities, reveals $\beta_{12}$-$ \rm {B_5H_3}$ is an attractive platform to study either quantum phase transition in 2D Dirac semimetal or the superconductivity or the exotic physics brought about by their interplay., Comment: 9 Pages, 5 Figures

We present a generative AI algorithm for addressing the challenging task of fast, accurate and robust statistical computation of three-dimensional turbulent fluid flows. Our algorithm, termed as GenCFD, is based on a conditional score-based diffusion model. Through extensive numerical experimentation with both incompressible and compressible fluid flows, we demonstrate that GenCFD provides very accurate approximation of statistical quantities of interest such as mean, variance, point pdfs, higher-order moments, while also generating high quality realistic samples of turbulent fluid flows and ensuring excellent spectral resolution. In contrast, ensembles of operator learning baselines which are trained to minimize mean (absolute) square errors regress to the mean flow. We present rigorous theoretical results uncovering the surprising mechanisms through which diffusion models accurately generate fluid flows. These mechanisms are illustrated with solvable toy models that exhibit the relevant features of turbulent fluid flows while being amenable to explicit analytical formulas., Comment: 71 pages, 30 figures

Thanks to the rapidly increasing time-domain facilities, we are entering a golden era of research on gamma-ray bursts (GRBs). In this Letter, we report our observations of GRB 240529A with the Burst Optical Observer and Transient Exploring System, the 1.5-meter telescope at Observatorio Sierra Nevada, the 2.5-meter Wide Field Survey Telescope of China, the Large Binocular Telescope, and the Telescopio Nazionale Galileo. The prompt emission of GRB 240529A shows two comparable energetic episodes separated by a quiescence time of roughly 400 s. Combining all available data on the GRB Coordinates Network, we reveal the simultaneous apparent X-ray plateau and optical re-brightening around $10^3-10^4$ s after the burst. Rather than the energy injection from the magnetar as widely invoked for similar GRBs, the multi-wavelength emissions could be better explained as two shocks launched from the central engine separately. The optical peak time and our numerical modeling suggest that the initial bulk Lorentz factor of the later shock is roughly 50, which indicates that the later jet should be accretion-driven and have a higher mass loading than a typical one. The quiescence time between the two prompt emission episodes may be caused by the transition between different accretion states of a central magnetar or black hole, or the fall-back accretion process. A sample of similar bursts with multiple emission episodes in the prompt phase and sufficient follow-up could help to probe the underlying physics of GRB central engines., Comment: Resubmitted to ApJL after addressing the referee's comments; comments are welcome

The Large Hadron Collider (LHC), the world's largest and most powerful particle accelerator, has been a pivotal tool in advancing our understanding of fundamental physics. By colliding heavy ions (such as lead ions), the LHC recreates conditions similar to those just after the Big Bang. This allows scientists to study the Quark-Gluon Plasma (QGP), a state of matter where quarks and gluons are not confined within protons and neutrons. These studies provide insights into the strong force and the early universe's behavior. In this paper, we provide a comprehensive overview of recent significant findings from A Large Ion Collider Experiment (ALICE) at LHC. The topics encompass measurements regarding to properties of the QGP, particle production, flow and correlations, dileptons, quarkonia and electromagnetic probes, heavy flavor, and jets. Additionally, we introduce future plans for detector upgrades of the ALICE experiment., Comment: 29 pages, 32 figures. This review is dedicated to Professor Wenqing Shen in honor of his leadership and significant impact on the Chinese heavy-ion physics community. All authors contributed equally to this work

We search for dark matter (DM) with a mass [3,12] $\mathrm{GeV} / c^2$ using an exposure of 3.51 $\mathrm{t} \times \mathrm{y}$ with the XENONnT experiment. We consider spin-independent, spin-dependent, momentum-dependent, mirror DM, and self-interacting DM with a light mediator coupling to Standard Model particles. Using a lowered energy threshold compared to the previous WIMP search, a blind analysis of [0.5, 5.0] $\mathrm{keV}$ nuclear recoil events reveals no significant signal excess over the background. XENONnT excludes spin-independent DM-nucleon cross sections $>2.5 \times 10^{-45} \mathrm{~cm}^2$ at $90 \%$ confidence level for 6 $\mathrm{GeV} / c^2$ DM. The solar ${ }^8 \mathrm{B}$ neutrino coherent elastic neutrino-nucleus scattering background accounts for approximately half of the background in the signal region. In the considered mass range, the DM sensitivity approaches the 'neutrino fog', the limitation where neutrinos produce a signal that is indistinguishable from that of light DM-xenon nucleus scattering.

Capturing different intensity and directions of light rays at the same scene Light field (LF) can encode the 3D scene cues into a 4D LF image which has a wide range of applications (i.e. post-capture refocusing and depth sensing). LF image super-resolution (SR) aims to improve the image resolution limited by the performance of LF camera sensor. Although existing methods have achieved promising results the practical application of these models is limited because they are not lightweight enough. In this paper we propose a lightweight model named LGFN which integrates the local and global features of different views and the features of different channels for LF image SR. Specifically owing to neighboring regions of the same pixel position in different sub-aperture images exhibit similar structural relationships we design a lightweight CNN-based feature extraction module (namely DGCE) to extract local features better through feature modulation. Meanwhile as the position beyond the boundaries in the LF image presents a large disparity we propose an efficient spatial attention module (namely ESAM) which uses decomposable large-kernel convolution to obtain an enlarged receptive field and an efficient channel attention module (namely ECAM). Compared with the existing LF image SR models with large parameter our model has a parameter of 0.45M and a FLOPs of 19.33G which has achieved a competitive effect. Extensive experiments with ablation studies demonstrate the effectiveness of our proposed method which ranked the second place in the Track 2 Fidelity & Efficiency of NTIRE2024 Light Field Super Resolution Challenge and the seventh place in the Track 1 Fidelity., Comment: 10 pages, 5 figures

Video causal reasoning aims to achieve a high-level understanding of video content from a causal perspective. However, current video reasoning tasks are limited in scope, primarily executed in a question-answering paradigm and focusing on short videos containing only a single event and simple causal relationships, lacking comprehensive and structured causality analysis for videos with multiple events. To fill this gap, we introduce a new task and dataset, Multi-Event Causal Discovery (MECD). It aims to uncover the causal relationships between events distributed chronologically across long videos. Given visual segments and textual descriptions of events, MECD requires identifying the causal associations between these events to derive a comprehensive, structured event-level video causal diagram explaining why and how the final result event occurred. To address MECD, we devise a novel framework inspired by the Granger Causality method, using an efficient mask-based event prediction model to perform an Event Granger Test, which estimates causality by comparing the predicted result event when premise events are masked versus unmasked. Furthermore, we integrate causal inference techniques such as front-door adjustment and counterfactual inference to address challenges in MECD like causality confounding and illusory causality. Experiments validate the effectiveness of our framework in providing causal relationships in multi-event videos, outperforming GPT-4o and VideoLLaVA by 5.7% and 4.1%, respectively., Comment: Accepted at NeurIPS 2024 as a spotlight paper

Semantic segmentation networks have achieved significant success under the assumption of independent and identically distributed data. However, these networks often struggle to detect anomalies from unknown semantic classes due to the limited set of visual concepts they are typically trained on. To address this issue, anomaly segmentation often involves fine-tuning on outlier samples, necessitating additional efforts for data collection, labeling, and model retraining. Seeking to avoid this cumbersome work, we take a different approach and propose to incorporate Vision-Language (VL) encoders into existing anomaly detectors to leverage the semantically broad VL pre-training for improved outlier awareness. Additionally, we propose a new scoring function that enables data- and training-free outlier supervision via textual prompts. The resulting VL4AD model, which includes max-logit prompt ensembling and a class-merging strategy, achieves competitive performance on widely used benchmark datasets, thereby demonstrating the potential of vision-language models for pixel-wise anomaly detection., Comment: 27 pages, 9 figures, to be published in ECCV 2024 2nd Workshop on Vision-Centric Autonomous Driving (VCAD)

We show that quantum advantage in distributed sensing can be achieved with noisy quantum networks. When using depolarized GHZ states as the probe, we derive a closed-form fidelity threshold to achieve advantage over the optimal local sensing strategy. The threshold indicates that while entanglement is needed for this quantum advantage, genuine multipartite entanglement is generally unnecessary. We further explore the impacts from imperfect local entanglement generation and local measurement constraint, and our results imply that the quantum advantage is more robust against quantum network imperfections than local operation errors. Finally, we demonstrate that the quantum advantage in distributed sensing can be achieved with a three-node quantum network using practical protocol stacks through simulations with SeQUeNCe, an open-source, customizable quantum network simulator., Comment: Updated references, added copyright statement, modified format

Data selection is of great significance in pre-training large language models, given the variation in quality within the large-scale available training corpora. To achieve this, researchers are currently investigating the use of data influence to measure the importance of data instances, $i.e.,$ a high influence score indicates that incorporating this instance to the training set is likely to enhance the model performance. Consequently, they select the top-$k$ instances with the highest scores. However, this approach has several limitations. (1) Computing the influence of all available data is time-consuming. (2) The selected data instances are not diverse enough, which may hinder the pre-trained model's ability to generalize effectively to various downstream tasks. In this paper, we introduce \texttt{Quad}, a data selection approach that considers both quality and diversity by using data influence to achieve state-of-the-art pre-training results. In particular, noting that attention layers capture extensive semantic details, we have adapted the accelerated $iHVP$ computation methods for attention layers, enhancing our ability to evaluate the influence of data, $i.e.,$ its quality. For the diversity, \texttt{Quad} clusters the dataset into similar data instances within each cluster and diverse instances across different clusters. For each cluster, if we opt to select data from it, we take some samples to evaluate the influence to prevent processing all instances. To determine which clusters to select, we utilize the classic Multi-Armed Bandit method, treating each cluster as an arm. This approach favors clusters with highly influential instances (ensuring high quality) or clusters that have been selected less frequently (ensuring diversity), thereby well balancing between quality and diversity.

The interplay between non-Hermiticity and disorder-free localization has recently become an intriguing and open question. In this work, we explore the impact of non-Hermiticity on pseudo mobility edges (MEs) and multi-states in disorder-free systems. We focus on a one-dimensional (1D) mosaic lattice with a finite-height Wannier-Stark potential under non-reciprocal non-Hermitian modulation. Using the transfer matrix techniques, we study how pseudo-MEs evolve under the influence of non-reciprocity. We then combine this with the perspective of similarity transformations to understand the changes in the emergent multi-states. Finally, we present the dynamical patterns of these multi-states. These findings expand the understanding of localization phenomena in non-Hermitian systems, offering new insights into the interplay between non-Hermiticity and disorder-free localization., Comment: Any comments are welcome!

Generating and inserting new objects into 3D content is a compelling approach for achieving versatile scene recreation. Existing methods, which rely on SDS optimization or single-view inpainting, often struggle to produce high-quality results. To address this, we propose a novel method for object insertion in 3D content represented by Gaussian Splatting. Our approach introduces a multi-view diffusion model, dubbed MVInpainter, which is built upon a pre-trained stable video diffusion model to facilitate view-consistent object inpainting. Within MVInpainter, we incorporate a ControlNet-based conditional injection module to enable controlled and more predictable multi-view generation. After generating the multi-view inpainted results, we further propose a mask-aware 3D reconstruction technique to refine Gaussian Splatting reconstruction from these sparse inpainted views. By leveraging these fabricate techniques, our approach yields diverse results, ensures view-consistent and harmonious insertions, and produces better object quality. Extensive experiments demonstrate that our approach outperforms existing methods., Comment: Project Page: https://github.com/JiuTongBro/MultiView_Inpaint

Recent advances on time series forecasting mainly focus on improving the forecasting models themselves. However, managing the length of the input data can also significantly enhance prediction performance. In this paper, we introduce a novel approach called Optimal Starting Point Time Series Forecast (OSP-TSP) to capture the intrinsic characteristics of time series data. By adjusting the sequence length via leveraging the XGBoost and LightGBM models, the proposed approach can determine optimal starting point (OSP) of the time series and thus enhance the prediction performances. The performances of the OSP-TSP approach are then evaluated across various frequencies on the M4 dataset and other real-world datasets. Empirical results indicate that predictions based on the OSP-TSP approach consistently outperform those using the complete dataset. Moreover, recognizing the necessity of sufficient data to effectively train models for OSP identification, we further propose targeted solutions to address the issue of data insufficiency.

Recent advances in fine-tuning Vision-Language Models (VLMs) have witnessed the success of prompt tuning and adapter tuning, while the classic model fine-tuning on inherent parameters seems to be overlooked. It is believed that fine-tuning the parameters of VLMs with few-shot samples corrupts the pre-trained knowledge since fine-tuning the CLIP model even degrades performance. In this paper, we revisit this viewpoint, and propose a new perspective: fine-tuning the specific parameters instead of all will uncover the power of classic model fine-tuning on VLMs. Through our meticulous study, we propose ClipFit, a simple yet effective method to fine-tune CLIP without introducing any overhead of extra parameters. We demonstrate that by only fine-tuning the specific bias terms and normalization layers, ClipFit can improve the performance of zero-shot CLIP by 7.27\% average harmonic mean accuracy. Lastly, to understand how fine-tuning in CLIPFit affects the pre-trained models, we conducted extensive experimental analyses w.r.t. changes in internal parameters and representations. We found that low-level text bias layers and the first layer normalization layer change much more than other layers. The code is available at \url{https://github.com/minglllli/CLIPFit}., Comment: EMNLP 2024 Main Conference

Scanning transmission electron microscopy (STEM) is a powerful tool to reveal the morphologies and structures of materials, thereby attracting intensive interests from the scientific and industrial communities. The outstanding spatial (atomic level) and temporal (ms level) resolutions of the STEM techniques generate fruitful amounts of high-definition data, thereby enabling the high-volume and high-speed analysis of materials. On the other hand, processing of the big dataset generated by STEM is time-consuming and beyond the capability of human-based manual work, which urgently calls for computer-based automation. In this work, we present a deep-learning mask region-based neural network (Mask R-CNN) for the recognition of nanoparticles imaged by STEM, as well as generating the associated dimensional analysis. The Mask R-CNN model was tested on simulated STEM-HAADF results with different Gaussian noises, particle shapes and particle sizes, and the results indicated that Gaussian noise has determining influence on the accuracy of recognition. By applying Gaussian and Non-Local Means filters on the noise-containing STEM-HAADF results, the influences of noises are largely mitigated, and recognition accuracy is significantly improved. This filtering-recognition approach was further applied to experimental STEM-HAADF results, which yields satisfying accuracy compared with the traditional threshold methods. The deep-learning-based method developed in this work has great potentials in analysis of the complicated structures and large data generated by STEM-HAADF.

SOWAHA is a protein-coding gene, also known as ANKRD43. Studies have indicated that SOWAHA can serve as a prognostic biomarker in colorectal cancer and pancreatic cancer. However, there are few reports about SOWAHA in other types of cancer and the specific mechanism of action of SOWAHA in cancer is also not clear. Based on National Center for Biotechnology Information (NCBI), The Cancer Genome Atlas (TCGA), Genotype-Tissue Expression Project (GTEx), cBioPortal, Human Protein Atlas (HPA), etc., we adopted bioinformatics methods to uncover the potential tumor genomic features of SOWAHA, including the correlation with prognosis, gene mutation, immune cell infiltration, and DNA methylation in different tumors and evaluated the association with tumor heterogeneity, stemness, chemokines chemokine receptors, and immunomodulators in pan-cancer. Besides, we knocked down SOWAHA in SW620 cells and performed RNA-seq analysis, then we conducted functional enrichment to uncover the biological significance of the gene set. SOWAHA has early diagnostic potential, and low expression of SOWAHA was associated with poor prognosis in was associated with poor prognosis in GBMLGG, PAAD, READ, etc. SOWAHA is associated with most tumor immune-infiltrating cells in pan-cancer. SOWAHA correlates with DNA methylation, tumor heterogeneity, and stemness in many epithelial carcinomas. Furthermore, SOWAHA is involved in many enzyme activity and metabolic pathways, mainly metabolic programming pathways in cancer. Additionally, we identified two potential transcription factors of SOWAHA, TBX4, and FOXP2, which are dysregulated in SW620 cells. Besides, the cell proliferation and viability in siSOWAHA groups are better than in siNC groups.SOWAHA, identified as a suppressor gene, and its role in the progression of colorectal cancer is primarily mediated through metabolic reprogramming mechanisms., Comment: 37 pages, 17 figures

The integration of large language models (LLMs) with pre-trained speech models has opened up new avenues in automatic speech recognition (ASR). While LLMs excel in multimodal understanding tasks, effectively leveraging their capabilities for ASR remains a significant challenge. This paper presents a novel training approach to enhance LLM performance in ASR tasks. We propose pre-training LLMs on Pinyin embedding sequences, which represent pronunciation features, to generate corresponding Chinese characters. This step enables the LLM to adapt to generating text from pronunciation features before encountering real speech data. Furthermore, we fine-tune the LoRA parameters to enhance the LLM's understanding of speech modality information. In AISHELL-1 corpus, our approach yields a 9.5% relative improvement in ASR tasks compared to the baseline without Pinyi-to-Character pre-training. Additionally, incorporating auxiliary text data for Pinyi-to-Character pre-training further boosts performance, achieving a 19.0% relative improvement., Comment: Accepted by ISCSLP2024-Special session-Speech Processing in LLM Era

Localization of the craniofacial landmarks from lateral cephalograms is a fundamental task in cephalometric analysis. The automation of the corresponding tasks has thus been the subject of intense research over the past decades. In this paper, we introduce the "Cephalometric Landmark Detection (CL-Detection)" dataset, which is the largest publicly available and comprehensive dataset for cephalometric landmark detection. This multi-center and multi-vendor dataset includes 600 lateral X-ray images with 38 landmarks acquired with different equipment from three medical centers. The overarching objective of this paper is to measure how far state-of-the-art deep learning methods can go for cephalometric landmark detection. Following the 2023 MICCAI CL-Detection Challenge, we report the results of the top ten research groups using deep learning methods. Results show that the best methods closely approximate the expert analysis, achieving a mean detection rate of 75.719% and a mean radial error of 1.518 mm. While there is room for improvement, these findings undeniably open the door to highly accurate and fully automatic location of craniofacial landmarks. We also identify scenarios for which deep learning methods are still failing. Both the dataset and detailed results are publicly available online, while the platform will remain open for the community to benchmark future algorithm developments at https://cl-detection2023.grand-challenge.org/., Comment: 16 pages, 7 figures

In recent years, research on aerial grasping, manipulation, and transportation of objects has garnered significant attention. These tasks often require UAVs to operate safely close to environments or objects and to efficiently grasp payloads. However, current widely adopted flying platforms pose safety hazards: unprotected high-speed rotating propellers can cause harm to the surroundings. Additionally, the space for carrying payloads on the fuselage is limited, and the restricted position of the payload also hinders efficient grasping. To address these issues, this paper presents a coaxial ducted fan UAV which is equipped with electromagnets mounted externally on the fuselage, enabling safe grasping and transfer of multiple loads in midair without complex additional actuators. It also has the capability to achieve direct human-UAV cargo transfer in the air. The forces acting on the loads during magnetic attachment and their influencing factors were analyzed. An ADRC controller is utilized to counteract disturbances during grasping and achieve attitude control. Finally, flight tests are conducted to verify the UAV's ability to directly grasp multiple loads from human hands in flight while maintaining attitude tracking., Comment: 8pages, 13figures,accepted by IROS2024 This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Intelligent Transportation Systems (ITS) are vital in modern traffic management and optimization, significantly enhancing traffic efficiency and safety. Recently, diffusion models have emerged as transformative tools for addressing complex challenges within ITS. In this paper, we present a comprehensive survey of diffusion models for ITS, covering both theoretical and practical aspects. First, we introduce the theoretical foundations of diffusion models and their key variants, including conditional diffusion models and latent diffusion models, highlighting their suitability for modeling complex, multi-modal traffic data and enabling controllable generation. Second, we outline the primary challenges in ITS and the corresponding advantages of diffusion models, providing readers with a deeper understanding of the intersection between ITS and diffusion models. Third, we offer a multi-perspective investigation of current applications of diffusion models in ITS domains, including autonomous driving, traffic simulation, trajectory prediction, and traffic safety. Finally, we discuss state-of-the-art diffusion model techniques and highlight key ITS research directions that warrant further investigation. Through this structured overview, we aim to provide researchers with a comprehensive understanding of diffusion models for ITS, thereby advancing their future applications in the transportation domain., Comment: 7 figures

The emergence of Large Language Models (LLMs) has significantly influenced various aspects of software development activities. Despite their benefits, LLMs also pose notable risks, including the potential to generate harmful content and being abused by malicious developers to create malicious code. Several previous studies have focused on the ability of LLMs to resist the generation of harmful content that violates human ethical standards, such as biased or offensive content. However, there is no research evaluating the ability of LLMs to resist malicious code generation. To fill this gap, we propose RMCBench, the first benchmark comprising 473 prompts designed to assess the ability of LLMs to resist malicious code generation. This benchmark employs two scenarios: a text-to-code scenario, where LLMs are prompted with descriptions to generate code, and a code-to-code scenario, where LLMs translate or complete existing malicious code. Based on RMCBench, we conduct an empirical study on 11 representative LLMs to assess their ability to resist malicious code generation. Our findings indicate that current LLMs have a limited ability to resist malicious code generation with an average refusal rate of 40.36% in text-to-code scenario and 11.52% in code-to-code scenario. The average refusal rate of all LLMs in RMCBench is only 28.71%; ChatGPT-4 has a refusal rate of only 35.73%. We also analyze the factors that affect LLMs' ability to resist malicious code generation and provide implications for developers to enhance model robustness., Comment: 12 pages, 6 figures, 5 tables, 39th IEEE/ACM International Conference on Automated Software Engineering (ASE '24)

The Large Vision-Language Model (LVLM) integrates computer vision and natural language processing techniques, offering substantial application potential. However, these models demand extensive resources during inference. Adaptive attention techniques can dynamically reduce computational redundancy and thus improve efficiency. Although current adaptive attention methods significantly reduce the memory requirements of Transformer-based language models, they are not tailored for LVLMs. We observe that LVLMs generate responses from both remote image tokens and local text tokens, and different modalities have different attention patterns. This observation inspires us to manage the attention for each modality separately. Specifically, for visual input, we store the cache of potentially useful information but only compute the most critical parts. For language input, we care more about local information. Based on our observation and analysis of vision-language attention patterns, we develop A-VL, a plug-and-play adaptive attention tailored for LVLM inference. Extensive evaluations on three vision-language tasks and five datasets show the effectiveness of our designs. Our approach A-VL outperforms existing adaptive attention methods in reducing memory usage and computational load without compromising performance.

We show that in a special type of two-dimensional dilaton-gravity-scalar model, where both the dilaton and the scalar matter fields have noncanonical kinetic terms, it is possible to construct kink solutions whose linear perturbation equation is a Schr\"odinger-like equation with Rosen-Morse potential. For this potential, eigenvalues and wave functions of the bound states, if had any, can be derived by using the standard shape invariance procedure. Depending on the values of the parameters, the stability potential can be reflective or reflectionless. There can be an arbitrary number of shape modes, but the zero mode is always absent., Comment: 8 pages, 1 figure

Artificial intelligence (AI) plays a crucial role in autonomous driving (AD) research, propelling its development towards intelligence and efficiency. Currently, the development of AD technology follows two main technical paths: modularization and end-to-end. Modularization decompose the driving task into modules such as perception, prediction, planning, and control, and train them separately. Due to the inconsistency of training objectives between modules, the integrated effect suffers from bias. End-to-end attempts to address this issue by utilizing a single model that directly maps from sensor data to control signals. This path has limited learning capabilities in a comprehensive set of features and struggles to handle unpredictable long-tail events and complex urban traffic scenarios. In the face of challenges encountered in both paths, many researchers believe that large language models (LLMs) with powerful reasoning capabilities and extensive knowledge understanding may be the solution, expecting LLMs to provide AD systems with deeper levels of understanding and decision-making capabilities. In light of the challenges faced by both paths, many researchers believe that LLMs, with their powerful reasoning abilities and extensive knowledge, could offer a solution. To understand if LLMs could enhance AD, this paper conducts a thorough analysis of the potential applications of LLMs in AD systems, including exploring their optimization strategies in both modular and end-to-end approaches, with a particular focus on how LLMs can tackle the problems and challenges present in current solutions. Furthermore, we discuss an important question: Can LLM-based artificial general intelligence (AGI) be a key to achieve high-level AD? We further analyze the potential limitations and challenges that LLMs may encounter in promoting the development of AD technology.

Diabetic foot neuropathy (DFN) is a critical factor leading to diabetic foot ulcers, which is one of the most common and severe complications of diabetes mellitus (DM) and is associated with high risks of amputation and mortality. Despite its significance, existing datasets do not directly derive from plantar data and lack continuous, long-term foot-specific information. To advance DFN research, we have collected a novel dataset comprising continuous plantar pressure data to recognize diabetic foot neuropathy. This dataset includes data from 94 DM patients with DFN and 41 DM patients without DFN. Moreover, traditional methods divide datasets by individuals, potentially leading to significant domain discrepancies in some feature spaces due to the absence of mid-domain data. In this paper, we propose an effective domain adaptation method to address this proplem. We split the dataset based on convolutional feature statistics and select appropriate sub-source domains to enhance efficiency and avoid negative transfer. We then align the distributions of each source and target domain pair in specific feature spaces to minimize the domain gap. Comprehensive results validate the effectiveness of our method on both the newly proposed dataset for DFN recognition and an existing dataset.

During the past decades, the numerical methods based on Navier-Stokes (N-S) equations and direct simulation Monte Carlo (DSMC) methods have been proved effective in simulating flows in the continuum and rarefied regimes, respectively. However, as single-scale methods, they face challenges in addressing common multi-scale problems, which are essential to simulate hypersonic flows around near-space vehicles and the flows in the micro-electro-mechanical systems. Hence, there is an urgent need for a method to predict multi-scale flows. In this work, a quantified model-competition (QMC) mechanism for diatomic multi-scale flows is derived from the integral solution of the Rykov model equations. This mechanism encapsulates both continuum and rarefied behaviors in a cell, weighted according to its local physical scale. By building upon the QMC mechanism, the N-S solver and DSMC solver are directly integrated within a cell to devise a simplified unified wave-particle (SUWP) method for diatomic gases. Specifically, the two-temperature equations considering the rotational energy are introduced into the kinetic inviscid flux (KIF) scheme and the N-S solver. As to the particle part, the collisionless DSMC solver is utilized to describe the non-equilibrium phenomenon. The proposed SUWP method for diatomic gases undergoes validation across a series of cases, including zero-dimensional homogeneous gas relaxation, one-dimensional normal shock structure, two-dimensional flow around the flat and the cylinder, and three-dimensional flows past the sphere and the blunt cone. Additionally, the implementation details of multi-scale wave-particle methods analysis and discussion are also undertaken in this work.

Understanding the dispersion process of supported catalysts is crucial for synthesizing atomic-level dispersed catalysts and precisely manipulating their chemical state. However, the underlying dispersion mechanism remains elusive due to the lack of atomic-level evidence during the dispersion process. Herein, by employing spherical aberration-corrected environmental scanning transmission electron microscopy (ESTEM), first-principles calculations, and a global optimization algorithm, we unraveled the pre-oxidation dispersion and direct dispersion mechanisms in the Pd/CeO2 (100) system, mediated by the thermodynamic and kinetic behaviors of single Pd species. We discovered that at lower temperatures, the Pd nanoparticles first undergo oxidation followed by the dispersion of PdO, while at higher temperatures, the entire dispersion process of Pd remains in a metallic state. The distinct dispersion mechanisms at different temperatures are driven by the thermodynamic and kinetic differences of environment-dependent single Pd species. The nonmobile Pd1O4 species stabilized at lower temperatures obstructs the direct dispersion of Pd nanoparticles, instead triggering a sequence of pre-oxidation followed by limited dispersion. In contrast, the highly mobile Pd1O2 species at higher temperatures facilitates the complete and direct dispersion of Pd nanoparticles. This research illuminates the essential physical mechanisms of oxidative dispersion from both thermodynamic and kinetic perspectives, potentially enabling strategies for precisely controlling the state of highly dispersed catalysts.

In a given randomized experiment, individuals are often volunteers and can differ in important ways from a population of interest. It is thus of interest to focus on the sample at hand. This paper focuses on inference about the sample local average treatment effect (LATE) in randomized experiments with non-compliance. We present a two-stage procedure that provides asymptotically correct coverage rate of the sample LATE in randomized experiments. The procedure uses a first-stage test to decide whether the instrument is strong or weak, and uses different confidence sets depending on the first-stage result. Proofs of the procedure is developed for the situation with and without regression adjustment and for two experimental designs (complete randomization and Mahalaonobis distance based rerandomization). Finite sample performance of the methods are studied using extensive Monte Carlo simulations and the methods are applied to data from a voter encouragement experiment.

Topological structures reveal the hidden secrets and beauty in nature, such as the double helix in DNA, whilst, the manipula-tion of which in physical fields, especially in ultrafast struc-tured light, draw booming attention. Here we introduce a new family of spatiotemporal light fields, i.e. helical pulses, carry-ing sophisticated double-helix singularities in its electromag-netic topological structures. The helical pulses were solved from Maxwell's equation as chiral extensions of toroidal light pulses but with controlled angular momentum dependence. We unveil that the double helix singularities can maintain their topological invariance during propagation and the field exhibits paired generation and annihilation of vortices and antivortices in ultrafast space-time, so as to be potential information carriers beating previous conventional vortex structured light.

Reinforcement Learning from Human Feedback (RLHF) enhances the alignment between LLMs and human preference. The workflow of RLHF typically involves several models and tasks in a series of distinct stages. Existing RLHF training systems view each task as the smallest execution unit thus overlooking the opportunities for subtask-level optimizations. Due to the intrinsic nature of RLHF training, i.e., the data skewness in the generation stage, and the pipeline bubbles in the training stage, existing RLHF systems suffer from low GPU utilization in production deployments. RLHFuse breaks the traditional view of RLHF workflow as a composition of individual tasks, splitting each task into finer-grained subtasks, and performing stage fusion to improve GPU utilization. RLHFuse contains two key ideas. First, for generation and inference tasks, RLHFuse splits them into sample-level subtasks, enabling efficient inter-stage fusion to mitigate the original generation bottleneck dominated by long-tailed samples. Second, for training tasks, RLHFuse breaks them into subtasks of micro-batches. By leveraging the intuition that pipeline execution can be essentially complemented by another pipeline, RLHFuse performs intra-stage fusion to concurrently execute these subtasks in the training stage with a fused pipeline schedule, resulting in fewer pipeline bubbles. In addition, RLHFuse incorporates a series of system optimizations tailored for each stage of RLHF, making it efficient and scalable for our internal product usage. We evaluate RLHFuse on various popular LLMs and the results show that RLHFuse increases the training throughput by up to 3.7x, compared to existing state-of-the-art systems.

Spin-space groups (SSGs), including the traditional space groups (SGs) and magnetic space groups (MSGs) as subsets, describe the complete symmetries of magnetic materials with weak spin-orbit coupling (SOC). In the present work, we systematically study the irreducible representations (irreps) of SSGs by focusing on the projective irreps of the little co-group $L(k)$ of any momentum point $\pmb k$. We analysis the factor systems of $L(k)$, and then reduce the projective regular representation of $L(k)$ into direct sum of irreps using the Hamiltonian approach. Especially, for collinear SSGs which contain continuous spin rotation operations, we adopt discrete subgroups to effectively capture their characteristics. Furthermore, we apply the representation theory of SSGs to study the band structure of electrons and magnons in magnetic materials. After identifying the SSG symmetry group, we extract relevant irreps and determine the $k\cdot p$ models. As an example, we illustrate how our approach works for the material \ch{Mn3Sn}. Degeneracies facilitated by SSG symmetry are observed, underscoring the effectiveness of application in material analysis. The SSG recognition and representation code is uploaded to GitHub, the information of irreps of all SSGs is also available in the online Database. Our work provides a practical toolkit for exploring the intricate symmetries of magnetic materials and paves the way for future advances in materials science.

We introduce PortraitGen, a powerful portrait video editing method that achieves consistent and expressive stylization with multimodal prompts. Traditional portrait video editing methods often struggle with 3D and temporal consistency, and typically lack in rendering quality and efficiency. To address these issues, we lift the portrait video frames to a unified dynamic 3D Gaussian field, which ensures structural and temporal coherence across frames. Furthermore, we design a novel Neural Gaussian Texture mechanism that not only enables sophisticated style editing but also achieves rendering speed over 100FPS. Our approach incorporates multimodal inputs through knowledge distilled from large-scale 2D generative models. Our system also incorporates expression similarity guidance and a face-aware portrait editing module, effectively mitigating degradation issues associated with iterative dataset updates. Extensive experiments demonstrate the temporal consistency, editing efficiency, and superior rendering quality of our method. The broad applicability of the proposed approach is demonstrated through various applications, including text-driven editing, image-driven editing, and relighting, highlighting its great potential to advance the field of video editing. Demo videos and released code are provided in our project page: https://ustc3dv.github.io/PortraitGen/, Comment: Accepted by SIGGRAPH Asia 2024. Project Page: https://ustc3dv.github.io/PortraitGen/

This paper is concerned with the detection, resilient and fault-tolerant control issues for cyber-physical systems. To this end, the impairment of system dynamics caused by the defined types of cyber-attacks and process faults is analyzed. Then, the relation of the system input and output signals with the residual subspaces spanned by both the process and the controller is studied. Considering the limit capacity of standard observer-based detection and feedback control schemes in detecting and handling the cyber-attacks, a modified configuration for cyber-physical systems is developed by transmitting the combinations of the input and output residuals instead of the input and output signals, which is facile for dealing with both the process faults and cyber-attacks. It is followed by the integrated design of fault and attack detection, resilient and fault-tolerant control schemes. To enhance the detectability of cyber-attacks, the potential stealthy attack mechanisms on deteriorating the tracking behavior and feedback control performance are developed from the attackers' point of view, and the associated detection schemes for such stealthy attacks are proposed from the defenders' point of view. A case study on the robotino system is utilized to demonstrate the proposed resilient cyber-physical configuration.

Artificial intelligence (AI) is revolutionizing numerous fields, with increasing applications in Global Navigation Satellite Systems (GNSS) positioning algorithms in intelligent transportation systems (ITS) via deep learning. However, a significant technological disparity exists as traditional GNSS algorithms are often developed in Fortran or C, contrasting with the Python-based implementation prevalent in deep learning tools. To address this discrepancy, this paper introduces pyrtklib, a Python binding for the widely utilized open-source GNSS tool, RTKLIB. This binding makes all RTKLIB functionalities accessible in Python, facilitating seamless integration. Moreover, we present a deep learning subsystem under pyrtklib, which is a novel deep learning framework that leverages pyrtklib to accurately predict weights and biases within the GNSS positioning process. The use of pyrtklib enables developers to easily and quickly prototype and implement deep learning-aided GNSS algorithms, showcasing its potential to enhance positioning accuracy significantly.

Symmetry protected topology has been studied extensively in the past twenty years, but the topology protected by spin symmetry has just begun to be studied. In this work, based on spin symmetry analysis, we propose that a class of Weyl nodal line semimetals is protected by the spin symmetry. Then, by the first-principles electronic structure calculations, we predict that both altermagnetic $\rm Nb_2FeB_2$ and $\rm Ta_2FeB_2$ are node-network semimetals protected by the spin symmetry. Moreover, both altermagnetic $\rm Nb_2FeB_2$ and $\rm Ta_2FeB_2$ have nodal rings protected by the mirror symmetry and Dirac points protected by nonsymmorphic spin symmetry. Furthermore, both altermagnetic $\rm Nb_2FeB_2$ and $\rm Ta_2FeB_2$ transform node-network semimetal phase into Weyl semimetal phase when considering spin-orbit coupling. Therefore, our work not only enriches the topological phases protected by spin symmetry, but also provides an excellent material platform to investigate the exotic physical properties of multiple altermagnetic topological semimetal phases in experiment., Comment: 6 pages, 4 figures

Language models (LMs) can produce errors that are hard to detect for humans, especially when the task is complex. RLHF, the most popular post-training method, may exacerbate this problem: to achieve higher rewards, LMs might get better at convincing humans that they are right even when they are wrong. We study this phenomenon under a standard RLHF pipeline, calling it "U-SOPHISTRY" since it is Unintended by model developers. Specifically, we ask time-constrained (e.g., 3-10 minutes) human subjects to evaluate the correctness of model outputs and calculate humans' accuracy against gold labels. On a question-answering task (QuALITY) and programming task (APPS), RLHF makes LMs better at convincing our subjects but not at completing the task correctly. RLHF also makes the model harder to evaluate: our subjects' false positive rate increases by 24.1% on QuALITY and 18.3% on APPS. Finally, we show that probing, a state-of-the-art approach for detecting Intended Sophistry (e.g. backdoored LMs), does not generalize to U-SOPHISTRY. Our results highlight an important failure mode of RLHF and call for more research in assisting humans to align them.

Quantum collision model provides a promising tool for investigating system-bath dynamics. Most of the studies on quantum collision models work in the resonant regime. In quantum dynamics, the off-resonant interaction often brings in exciting ffects. It is thereby attractive to investigate quantum collision models in the off-resonant regime. On the other hand, a bath with a negative absolute temperature is anticipated to be instrumental in developing thermal devices. The design of an effective bath with negative absolute temperature coupled to a qubit is significant for developing such thermal devices. We establish an effective negative-absolute-temperature bath coupled to a qubit with a quantum collision model in a far-off-resonant regime. We conduct a detailed and systematic investigation on the off-resonant collision model. There is an additional constraint on the collision duration resulting from the far-off resonant collision. The dynamics of the collision model in the far-off-resonant regime are different from the one beyond the far-off-resonant regime. Numerical simulations confirm the validity of the proposed approach., Comment: 7 pages, 3 figures

Objective: To establish an automated pipeline for post-processing of quantitative spin-lattice relaxation time constant in the rotating frame ($T_{1\rho}$) imaging of knee articular cartilage. Design: The proposed post-processing pipeline commences with an image standardisation procedure, followed by deep learning-based segmentation to generate cartilage masks. The articular cartilage is then automatically parcellated into 20 subregions, where $T_{1\rho}$ quantification is performed. The proposed pipeline was retrospectively validated on a dataset comprising knee $T_{1\rho}$ images of 10 healthy volunteers and 30 patients with knee osteoarthritis. Three experiments were conducted, namely an assessment of segmentation model performance (using Dice similarity coefficients, DSCs); an evaluation of the impact of standardisation; and a test of $T_{1\rho}$ quantification accuracy (using paired t-tests; root-mean-square deviations, RMSDs; and coefficients of variance of RMSDs, $CV_{RMSD}$). Statistical significance was set as p<0.05. Results: There was a substantial agreement between the subregional $T_{1\rho}$ quantification from the model-predicted masks and those from the manual segmentation labels. In patients, 17 of 20 subregions, and in healthy volunteers, 18 out of 20 subregions, demonstrated no significant difference between predicted and reference $T_{1\rho}$ quantifications. Average RMSDs were 0.79 ms for patients and 0.56 ms for healthy volunteers, while average $CV_{RMSD}$ were 1.97% and 1.38% for patients and healthy volunteers. Bland-Altman plots showed negligible bias across all subregions for patients and healthy volunteers. Conclusion: The proposed pipeline can perform automatic and reliable post-processing of quantitative $T_{1\rho}$ images of knee articular cartilage., Comment: Junru Zhong and Yongcheng Yao share the same contribution. Work was partially done when Yongcheng Yao and Siyue Li was with CUHK

Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biological interpretability. Their rich spatio-temporal information processing capability and event-driven nature make them ideally well-suited for neuromorphic datasets. However, current SNNs struggle to balance accuracy and latency in classifying these datasets. In this paper, we propose Hybrid Step-wise Distillation (HSD) method, tailored for neuromorphic datasets, to mitigate the notable decline in performance at lower time steps. Our work disentangles the dependency between the number of event frames and the time steps of SNNs, utilizing more event frames during the training stage to improve performance, while using fewer event frames during the inference stage to reduce latency. Nevertheless, the average output of SNNs across all time steps is susceptible to individual time step with abnormal outputs, particularly at extremely low time steps. To tackle this issue, we implement Step-wise Knowledge Distillation (SKD) module that considers variations in the output distribution of SNNs at each time step. Empirical evidence demonstrates that our method yields competitive performance in classification tasks on neuromorphic datasets, especially at lower time steps. Our code will be available at: {https://github.com/hsw0929/HSD}.

On-road air pollution exhibits substantial variability over short distances due to emission sources, dilution, and physicochemical processes. Integrating mobile monitoring data with street view images (SVIs) holds promise for predicting local air pollution. However, algorithms, sampling strategies, and image quality introduce extra errors due to a lack of reliable references that quantify their effects. To bridge this gap, we employed 314 taxis to monitor NO, NO2, PM2.5 and PM10 dynamically and sampled corresponding SVIs, aiming to develop a reliable strategy. We extracted SVI features from ~ 382,000 streetscape images, which were collected at various angles (0{\deg}, 90{\deg}, 180{\deg}, 270{\deg}) and ranges (buffers with radii of 100m, 200m, 300m, 400m, 500m). Also, three machine learning algorithms alongside the linear land-used regression (LUR) model were experimented with to explore the influences of different algorithms. Four typical image quality issues were identified and discussed. Generally, machine learning methods outperform linear LUR for estimating the four pollutants, with the ranking: random forest > XGBoost > neural network > LUR. Compared to single-angle sampling, the averaging strategy is an effective method to avoid bias of insufficient feature capture. Therefore, the optimal sampling strategy is to obtain SVIs at a 100m radius buffer and extract features using the averaging strategy. This approach achieved estimation results for each aggregation location with absolute errors almost less than 2.5 {\mu}g/m^2 or ppb. Overexposure, blur, and underexposure led to image misjudgments and incorrect identifications, causing an overestimation of road features and underestimation of human-activity features, contributing to inaccurate NO, NO2, PM2.5 and PM10 estimation.

We introduce a novel Recursive Fusion model, dubbed ReFu, designed to integrate point clouds and meshes for exemplar-free 3D Class-Incremental Learning, where the model learns new 3D classes while retaining knowledge of previously learned ones. Unlike existing methods that either rely on storing historical data to mitigate forgetting or focus on single data modalities, ReFu eliminates the need for exemplar storage while utilizing the complementary strengths of both point clouds and meshes. To achieve this, we introduce a recursive method which continuously accumulates knowledge by updating the regularized auto-correlation matrix. Furthermore, we propose a fusion module, featuring a Pointcloud-guided Mesh Attention Layer that learns correlations between the two modalities. This mechanism effectively integrates point cloud and mesh features, leading to more robust and stable continual learning. Experiments across various datasets demonstrate that our proposed framework outperforms existing methods in 3D class-incremental learning. Project Page: https://arlo397.github.io/ReFu/

Human-in-the-loop reinforcement learning integrates human expertise to accelerate agent learning and provide critical guidance and feedback in complex fields. However, many existing approaches focus on single-agent tasks and require continuous human involvement during the training process, significantly increasing the human workload and limiting scalability. In this paper, we propose HARP (Human-Assisted Regrouping with Permutation Invariant Critic), a multi-agent reinforcement learning framework designed for group-oriented tasks. HARP integrates automatic agent regrouping with strategic human assistance during deployment, enabling and allowing non-experts to offer effective guidance with minimal intervention. During training, agents dynamically adjust their groupings to optimize collaborative task completion. When deployed, they actively seek human assistance and utilize the Permutation Invariant Group Critic to evaluate and refine human-proposed groupings, allowing non-expert users to contribute valuable suggestions. In multiple collaboration scenarios, our approach is able to leverage limited guidance from non-experts and enhance performance. The project can be found at https://github.com/huawen-hu/HARP., Comment: 7 pages, 6 figures