Search

Ship detection needs to identify ship locations from remote sensing (RS) scenes. Due to different imaging payloads, various appearances of ships, and complicated background interference from the bird's eye view, it is difficult to set up a unified paradigm for achieving multi-source ship detection. To address this challenge, in this article, leveraging the large language models (LLMs)'s powerful generalization ability, a unified visual-language model called Popeye is proposed for multi-source ship detection from RS imagery. Specifically, to bridge the interpretation gap between the multi-source images for ship detection, a novel unified labeling paradigm is designed to integrate different visual modalities and the various ship detection ways, i.e., horizontal bounding box (HBB) and oriented bounding box (OBB). Subsequently, the hybrid experts encoder is designed to refine multi-scale visual features, thereby enhancing visual perception. Then, a visual-language alignment method is developed for Popeye to enhance interactive comprehension ability between visual and language content. Furthermore, an instruction adaption mechanism is proposed for transferring the pre-trained visual-language knowledge from the nature scene into the RS domain for multi-source ship detection. In addition, the segment anything model (SAM) is also seamlessly integrated into the proposed Popeye to achieve pixel-level ship segmentation without additional training costs. Finally, extensive experiments are conducted on the newly constructed ship instruction dataset named MMShip, and the results indicate that the proposed Popeye outperforms current specialist, open-vocabulary, and other visual-language models for zero-shot multi-source ship detection.

The counterflow superfluidity (CSF) was predicted two decades ago. Counterintuitively, while both components in the CSF have fluidity, their correlated counterflow currents cancel out leading the overall system to an incompressible Mott insulator. However, realizing and identifying the CSF remain challenging due to the request on extreme experimental capabilities in a single setup. Here, we observe the CSF in a binary Bose mixture in optical lattices. We prepare a low-entropy spin-Mott state by conveying and merging two spin-1/2 bosonic atoms at every site and drive it adiabatically to the CSF at $\sim$ 1 nK. Antipair correlations of the CSF are probed though a site- and spin-resolved quantum gas microscope in both real and momentum spaces. These techniques and observations provide accessibility to the symmetry-protected topological quantum matters., Comment: 13 pages, 10 figures

The restricted isometry property (RIP) is essential for the linear map to guarantee the successful recovery of low-rank matrices. The existing works show that the linear map generated by the measurement matrices with independent and identically distributed (i.i.d.) entries satisfies RIP with high probability. However, when dealing with non-i.i.d. measurement matrices, such as the rank-one measurements, the RIP compliance may not be guaranteed. In this paper, we show that the RIP can still be achieved with high probability, when the rank-one measurement matrix is constructed by the random unit-modulus vectors. Compared to the existing works, we first address the challenge of establishing RIP for the linear map in non-i.i.d. scenarios. As validated in the experiments, this linear map is memory-efficient, and not only satisfies the RIP but also exhibits similar recovery performance of the low-rank matrices to that of conventional i.i.d. measurement matrices., Comment: AISTATS 2024

Instead of making behavioral decisions directly from the exponentially expanding joint observational-action space, subtask-based multi-agent reinforcement learning (MARL) methods enable agents to learn how to tackle different subtasks. Most existing subtask-based MARL methods are based on hierarchical reinforcement learning (HRL). However, these approaches often limit the number of subtasks, perform subtask recognition periodically, and can only identify and execute a specific subtask within the predefined fixed time period, which makes them inflexible and not suitable for diverse and dynamic scenarios with constantly changing subtasks. To break through above restrictions, a \textbf{S}liding \textbf{M}ultidimensional t\textbf{A}sk window based m\textbf{U}ti-agent reinforcement learnin\textbf{G} framework (SMAUG) is proposed for adaptive real-time subtask recognition. It leverages a sliding multidimensional task window to extract essential information of subtasks from trajectory segments concatenated based on observed and predicted trajectories in varying lengths. An inference network is designed to iteratively predict future trajectories with the subtask-oriented policy network. Furthermore, intrinsic motivation rewards are defined to promote subtask exploration and behavior diversity. SMAUG can be integrated with any Q-learning-based approach. Experiments on StarCraft II show that SMAUG not only demonstrates performance superiority in comparison with all baselines but also presents a more prominent and swift rise in rewards during the initial training stage.

The remarkable frequency stability of resonant systems in the optical domain (optical cavities and atomic transitions) can be harnessed at frequency scales accessible by electronics using optical frequency division. This capability is revolutionizing technologies spanning time keeping to high-performance electrical signal sources. A version of the technique called 2-point optical frequency division (2P-OFD) is proving advantageous for application to high-performance signal sources. In 2P-OFD, an optical cavity anchors two spectral endpoints defined by lines of a frequency comb. The comb need not be self-referenced, which greatly simplifies the system architecture and reduces power requirements. Here, a 2P-OFD microwave signal source is demonstrated with record-low phase noise using a microcomb. Key to this advance is a spectral endpoint defined by a frequency agile single-mode dispersive wave that is emitted by the microcomb soliton. Moreover, the system frequency reference is a compact all-solid-state optical cavity with a record Q-factor. The results advance integrable microcomb-based signal sources into the performance realm of much larger microwave sources.

Large language models (LLMs) hold great potential for many natural language applications but risk generating incorrect or toxic content. To probe when an LLM generates unwanted content, the current paradigm is to recruit a \textit{red team} of human testers to design input prompts (i.e., test cases) that elicit undesirable responses from LLMs. However, relying solely on human testers is expensive and time-consuming. Recent works automate red teaming by training a separate red team LLM with reinforcement learning (RL) to generate test cases that maximize the chance of eliciting undesirable responses from the target LLM. However, current RL methods are only able to generate a small number of effective test cases resulting in a low coverage of the span of prompts that elicit undesirable responses from the target LLM. To overcome this limitation, we draw a connection between the problem of increasing the coverage of generated test cases and the well-studied approach of curiosity-driven exploration that optimizes for novelty. Our method of curiosity-driven red teaming (CRT) achieves greater coverage of test cases while mantaining or increasing their effectiveness compared to existing methods. Our method, CRT successfully provokes toxic responses from LLaMA2 model that has been heavily fine-tuned using human preferences to avoid toxic outputs. Code is available at \url{https://github.com/Improbable-AI/curiosity_redteam}, Comment: Published at ICLR 2024

This paper explores providing explainability for session-based recommendation (SR) by path reasoning. Current SR models emphasize accuracy but lack explainability, while traditional path reasoning prioritizes knowledge graph exploration, ignoring sequential patterns present in the session history. Therefore, we propose a generalized hierarchical reinforcement learning framework for SR, which improves the explainability of existing SR models via Path Reasoning, namely PR4SR. Considering the different importance of items to the session, we design the session-level agent to select the items in the session as the starting point for path reasoning and the path-level agent to perform path reasoning. In particular, we design a multi-target reward mechanism to adapt to the skip behaviors of sequential patterns in SR, and introduce path midpoint reward to enhance the exploration efficiency in knowledge graphs. To improve the completeness of the knowledge graph and to diversify the paths of explanation, we incorporate extracted feature information from images into the knowledge graph. We instantiate PR4SR in five state-of-the-art SR models (i.e., GRU4REC, NARM, GCSAN, SR-GNN, SASRec) and compare it with other explainable SR frameworks, to demonstrate the effectiveness of PR4SR for recommendation and explanation tasks through extensive experiments with these approaches on four datasets.

Logs produced by extensive software systems are integral to monitoring system behaviors. Advanced log analysis facilitates the detection, alerting, and diagnosis of system faults. Log parsing, which entails transforming raw log messages into structured templates, constitutes a critical phase in the automation of log analytics. Existing log parsers fail to identify the correct templates due to reliance on human-made rules. Besides, These methods focus on statistical features while ignoring semantic information in log messages. To address these challenges, we introduce a cutting-edge \textbf{L}og parsing framework with \textbf{E}ntropy sampling and Chain-of-Thought \textbf{M}erging (Lemur). Specifically, to discard the tedious manual rules. We propose a novel sampling method inspired by information entropy, which efficiently clusters typical logs. Furthermore, to enhance the merging of log templates, we design a chain-of-thought method for large language models (LLMs). LLMs exhibit exceptional semantic comprehension, deftly distinguishing between parameters and invariant tokens. We have conducted experiments on large-scale public datasets. Extensive evaluation demonstrates that Lemur achieves the state-of-the-art performance and impressive efficiency., Comment: 7 pages

We survey recent developments on generalizing the Gross--Zagier formula to high dimensional Shimura varieties, with an emphasis on the Arithmetic Gan--Gross--Prasad conjecture and the relative trace formula approach., Comment: Lectures at the IHES 2022 Summer School on the Langlands Program, Comments welcome

The Silicon Charge Detector (SCD) is a subdetector of the High Energy Cosmic Radiation Detection payload. The dynamic range of the silicon microstrip detector can be extended by the capacitive coupling effect, which is related to the interstrip capacitance and the coupling capacitance. A detector prototype with several sets of parameters was designed and tested in the ion beams at the CERN Super Proton Synchrotron. The capacitive coupling fractions with readout strip and floating strip incidences were studied using the beam test data and SPICE simulation.

Many tools exist for extracting structural and physiochemical descriptors from linear peptides to predict their properties, but similar tools for hydrocarbon-stapled peptides are lacking.Here, we present StaPep, a Python-based toolkit designed for generating 2D/3D structures and calculating 21 distinct features for hydrocarbon-stapled peptides.The current version supports hydrocarbon-stapled peptides containing 2 non-standard amino acids (norleucine and 2-aminoisobutyric acid) and 6 nonnatural anchoring residues (S3, S5, S8, R3, R5 and R8).Then we established a hand-curated dataset of 201 hydrocarbon-stapled peptides and 384 linear peptides with sequence information and experimental membrane permeability, to showcase StaPep's application in artificial intelligence projects.A machine learning-based predictor utilizing above calculated features was developed with AUC of 0.85, for identifying cell-penetrating hydrocarbon-stapled peptides.StaPep's pipeline spans data retrieval, cleaning, structure generation, molecular feature calculation, and machine learning model construction for hydrocarbon-stapled peptides.The source codes and dataset are freely available on Github: https://github.com/dahuilangda/stapep_package., Comment: 26 pages, 6 figures

While Reinforcement Learning from Human Feedback (RLHF) significantly enhances the generation quality of Large Language Models (LLMs), recent studies have raised concerns regarding the complexity and instability associated with the Proximal Policy Optimization (PPO) algorithm, proposing a series of order-based calibration methods as viable alternatives. This paper delves further into current order-based methods, examining their inefficiencies in utilizing reward values and addressing misalignment issues. Building upon these findings, we propose a novel \textbf{V}alue-based \textbf{C}ali\textbf{B}ration (VCB) method to better align LLMs with human preferences. Experimental results demonstrate that VCB surpasses existing alignment methods on AI assistant and summarization datasets, providing impressive generalizability, robustness, and stability in diverse settings., Comment: 19 pages, Under review

Developing hybrid materials and structures for electromagnetic wave engineering has been a promising route towards novel functionalities and tunabilities in many modern applications and perspectives in new quantum technologies. Despite its established success in engineering optical light and terahertz waves, the implementation of meta-resonators operating at the microwave band is still emerging, especially those that allow for on-chip integration and size miniaturization, which has turned out crucial to developing hybrid quantum systems at the microwave band. In this work, we present a microwave meta-resonator consisting of split-ring and and spiral resonators, and implement it to the investigation of photon-magnon coupling for hybrid magnonic applications. We observe broadened bandwidth to the split ring modes augmented by the additional spiral resonator, and, by coupling the modes to a magnetic sample, the resultant photon-magnon coupling can be significantly enhanced to more than ten-fold. Our work suggests that combinatorial, hybrid microwave resonators may be a promising approach towards future development and implementation of photon-magnon coupling in hybrid magnonic systems., Comment: 11 pages, 7 figures

Reference features from a template or historical frames are crucial for visual object tracking. Prior works utilize all features from a fixed template or memory for visual object tracking. However, due to the dynamic nature of videos, the required reference historical information for different search regions at different time steps is also inconsistent. Therefore, using all features in the template and memory can lead to redundancy and impair tracking performance. To alleviate this issue, we propose a novel tracking paradigm, consisting of a relevance attention mechanism and a global representation memory, which can adaptively assist the search region in selecting the most relevant historical information from reference features. Specifically, the proposed relevance attention mechanism in this work differs from previous approaches in that it can dynamically choose and build the optimal global representation memory for the current frame by accessing cross-frame information globally. Moreover, it can flexibly read the relevant historical information from the constructed memory to reduce redundancy and counteract the negative effects of harmful information. Extensive experiments validate the effectiveness of the proposed method, achieving competitive performance on five challenging datasets with 71 FPS., Comment: 9pages,5 figures, accepted by the Thirty-seventh Conference on Neural Information Processing Systems(Neurips 2023)

Deep neural networks have revolutionized many fields, but their black-box nature also occasionally prevents their wider adoption in fields such as healthcare and finance, where interpretable and explainable models are required. The recent development of Neural Additive Models (NAMs) is a significant step in the direction of interpretable deep learning for tabular datasets. In this paper, we propose a new subclass of NAMs that use a single-layer neural network construction of the Gaussian process via random Fourier features, which we call Gaussian Process Neural Additive Models (GP-NAM). GP-NAMs have the advantage of a convex objective function and number of trainable parameters that grows linearly with feature dimensionality. It suffers no loss in performance compared to deeper NAM approaches because GPs are well-suited for learning complex non-parametric univariate functions. We demonstrate the performance of GP-NAM on several tabular datasets, showing that it achieves comparable or better performance in both classification and regression tasks with a large reduction in the number of parameters., Comment: Appears at AAAI 2024

Over the past few years, the abilities of large language models (LLMs) have received extensive attention, which have performed exceptionally well in complicated scenarios such as logical reasoning and symbolic inference. A significant factor contributing to this progress is the benefit of in-context learning and few-shot prompting. However, the reasons behind the success of such models using contextual reasoning have not been fully explored. Do LLMs have understand logical rules to draw inferences, or do they ``guess'' the answers by learning a type of probabilistic mapping through context? This paper investigates the reasoning capabilities of LLMs on two logical reasoning datasets by using counterfactual methods to replace context text and modify logical concepts. Based on our analysis, it is found that LLMs do not truly understand logical rules; rather, in-context learning has simply enhanced the likelihood of these models arriving at the correct answers. If one alters certain words in the context text or changes the concepts of logical terms, the outputs of LLMs can be significantly disrupted, leading to counter-intuitive responses. This work provides critical insights into the limitations of LLMs, underscoring the need for more robust mechanisms to ensure reliable logical reasoning in LLMs.

A transparent decision-making process is essential for developing reliable and trustworthy recommender systems. For sequential recommendation, it means that the model can identify critical items asthe justifications for its recommendation results. However, achieving both model transparency and recommendation performance simultaneously is challenging, especially for models that take the entire sequence of items as input without screening. In this paper,we propose an interpretable framework (named PTSR) that enables a pattern-wise transparent decision-making process. It breaks the sequence of items into multi-level patterns that serve as atomic units for the entire recommendation process. The contribution of each pattern to the outcome is quantified in the probability space. With a carefully designed pattern weighting correction, the pattern contribution can be learned in the absence of ground-truth critical patterns. The final recommended items are those items that most critical patterns strongly endorse. Extensive experiments on four public datasets demonstrate remarkable recommendation performance, while case studies validate the model transparency. Our code is available at https://anonymous.4open.science/r/PTSR-2237.

Spatial-temporal forecasting systems play a crucial role in addressing numerous real-world challenges. In this paper, we investigate the potential of addressing spatial-temporal forecasting problems using general time series forecasting models, i.e., models that do not leverage the spatial relationships among the nodes. We propose a all-Multi-Layer Perceptron (all-MLP) time series forecasting architecture called RPMixer. The all-MLP architecture was chosen due to its recent success in time series forecasting benchmarks. Furthermore, our method capitalizes on the ensemble-like behavior of deep neural networks, where each individual block within the network behaves like a base learner in an ensemble model, particularly when identity mapping residual connections are incorporated. By integrating random projection layers into our model, we increase the diversity among the blocks' outputs, thereby improving the overall performance of the network. Extensive experiments conducted on the largest spatial-temporal forecasting benchmark datasets demonstrate that the proposed method outperforms alternative methods, including both spatial-temporal graph models and general forecasting models.

The extraction of road network is essential for the generation of high-definition maps since it enables the precise localization of road landmarks and their interconnections. However, generating road network poses a significant challenge due to the conflicting underlying combination of Euclidean (e.g., road landmarks location) and non-Euclidean (e.g., road topological connectivity) structures. Existing methods struggle to merge the two types of data domains effectively, but few of them address it properly. Instead, our work establishes a unified representation of both types of data domain by projecting both Euclidean and non-Euclidean data into an integer series called RoadNet Sequence. Further than modeling an auto-regressive sequence-to-sequence Transformer model to understand RoadNet Sequence, we decouple the dependency of RoadNet Sequence into a mixture of auto-regressive and non-autoregressive dependency. Building on this, our proposed non-autoregressive sequence-to-sequence approach leverages non-autoregressive dependencies while fixing the gap towards auto-regressive dependencies, resulting in success on both efficiency and accuracy. We further identify two main bottlenecks in the current RoadNetTransformer on a non-overfitting split of the dataset: poor landmark detection limited by the BEV Encoder and error propagation to topology reasoning. Therefore, we propose Topology-Inherited Training to inherit better topology knowledge into RoadNetTransformer. Additionally, we collect SD-Maps from open-source map datasets and use this prior information to significantly improve landmark detection and reachability. Extensive experiments on nuScenes dataset demonstrate the superiority of RoadNet Sequence representation and the non-autoregressive approach compared to existing state-of-the-art alternatives., Comment: V1 is the ICCV 2023 conference version, and V2 is the extended version

Large language models (LLMs) have gained much attention in the recommendation community; some studies have observed that LLMs, fine-tuned by the cross-entropy loss with a full softmax, could achieve state-of-the-art performance already. However, these claims are drawn from unobjective and unfair comparisons. In view of the substantial quantity of items in reality, conventional recommenders typically adopt a pointwise/pairwise loss function instead for training. This substitute however causes severe performance degradation, leading to under-estimation of conventional methods and over-confidence in the ranking capability of LLMs. In this work, we theoretically justify the superiority of cross-entropy, and showcase that it can be adequately replaced by some elementary approximations with certain necessary modifications. The remarkable results across three public datasets corroborate that even in a practical sense, existing LLM-based methods are not as effective as claimed for next-item recommendation. We hope that these theoretical understandings in conjunction with the empirical results will facilitate an objective evaluation of LLM-based recommendation in the future., Comment: 16 pages

We propose a major upgrade to the existing PandaX-4T experiment in the China Jinping Underground Laboratory. The new experiment, PandaX-xT, will be a multi-ten-tonne liquid xenon, ultra-low background, and general-purpose observatory. The full-scaled PandaX-xT contains a 43-tonne liquid xenon active target. Such an experiment will significantly advance our fundamental understanding of particle physics and astrophysics. The sensitivity of dark matter direct detection will be improved by nearly two orders of magnitude compared to the current best limits, approaching the so-called "neutrino floor" for a dark matter mass above 10 GeV/$c^2$, providing a decisive test to the Weakly Interacting Massive Particle paradigm. By searching for the neutrinoless double beta decay of $^{136}$Xe isotope in the detector, the effective Majorana neutrino mass can be measured to a [10 -- 41] meV/$c^2$ sensitivity, providing a key test to the Dirac/Majorana nature of neutrino s. Astrophysical neutrinos and other ultra-rare interactions can also be measured and searched for with an unprecedented background level, opening up new windows of discovery. Depending on the findings, PandaX-xT will seek the next stage upgrade utilizing isotopic separation on natural xenon.

The mass table in the deformed relativistic Hartree-Bogoliubov theory in continuum (DRHBc) with the PC-PK1 density functional has been established for even-$Z$ nuclei with $8\le Z\le120$, extended from the previous work for even-even nuclei [Zhang $\it{et.~al.}$ (DRHBc Mass Table Collaboration), At. Data Nucl. Data Tables 144, 101488 (2022)]. The calculated binding energies, two-nucleon and one-neutron separation energies, root-mean-square (rms) radii of neutron, proton, matter, and charge distributions, quadrupole deformations, and neutron and proton Fermi surfaces are tabulated and compared with available experimental data. A total of 4829 even-$Z$ nuclei are predicted to be bound, with an rms deviation of 1.477 MeV from the 1244 mass data. Good agreement with the available experimental odd-even mass differences, $\alpha$ decay energies, and charge radii is also achieved. The description accuracy for nuclear masses and nucleon separation energies as well as the prediction for drip lines is compared with the results obtained from other relativistic and nonrelativistic density functional. The comparison shows that the DRHBc theory with PC-PK1 provides an excellent microscopic description for the masses of even-$Z$ nuclei. The systematics of the nucleon separation energies, odd-even mass differences, pairing energies, two-nucleon gaps, $\alpha$ decay energies, rms radii, quadrupole deformations, potential energy curves, neutron density distributions, and neutron mean-field potentials are discussed., Comment: 394 pages, 17 figures, 2 tables, published in Atomic Data and Nuclear Data Tables, data file in the TXT form is available for download under "Ancillary files"

Numerous applications at the photon-starved regime require a free-space coupling singlephoton detector with a large active area, low dark count rate (DCR), and superior time resolutions. Here,we developed a superconducting microstrip single-photon detector (SMSPD), with a large active area of 260 um in diameter, a DCR of ~5 kcps, and a low time jitter of ~171 ps, operated at near-infrared of 1550 nm. As a demonstration, we applied the detector to a single-pixel galvanometer scanning system and successfully reconstructed object information in depth and intensity using a time-correlated photon counting technology., Comment: 12 pages, 6 figures

Distractors are important in learning evaluation. This paper surveys distractor generation tasks using English multiple-choice question datasets for textual and multimodal contexts. In particular, this paper presents a thorough literature review of the recent studies on distractor generation tasks, discusses multiple choice components and their characteristics, analyzes the related datasets, and summarizes the evaluation metrics of distractor generation. Our investigation reveals that more than half of datasets are human-generated from educational sources in specific domains such as Science and English, which are largely text-based, with a lack of open domain and multimodal datasets.

The existence and dynamics of stable quantized vortices is an important subject of quantum many-body physics. Spin-orbital-angular-momentum coupling (SOAMC), a special type of spin-orbit coupling, has been experimentally achieved to create vortices in atomic Bose-Einstein condensates (BEC). Here, we generalize the concept of SOAMC to a two-component polariton BEC and analyze the emergence and configuration of vortices under a finite-size circular pumping beam. We find that the regular configuration of vortex lattices induced by a finite-size circular pump is significantly distorted by the spatially dependent Raman coupling of SOAMC, even in the presence of a repulsive polariton interaction which can assist the forming of stable vortex configuration. Meanwhile, a pair of vortices induced by SOAMC located at the center of polariton cloud remains stable. When the Raman coupling is sufficiently strong and interaction is weak, the vortices spiraling in from the edge of polariton cloud will disrupt the polariton BEC.

Understanding road structures is crucial for autonomous driving. Intricate road structures are often depicted using lane graphs, which include centerline curves and connections forming a Directed Acyclic Graph (DAG). Accurate extraction of lane graphs relies on precisely estimating vertex and edge information within the DAG. Recent research highlights Transformer-based language models' impressive sequence prediction abilities, making them effective for learning graph representations when graph data are encoded as sequences. However, existing studies focus mainly on modeling vertices explicitly, leaving edge information simply embedded in the network. Consequently, these approaches fall short in the task of lane graph extraction. To address this, we introduce LaneGraph2Seq, a novel approach for lane graph extraction. It leverages a language model with vertex-edge encoding and connectivity enhancement. Our serialization strategy includes a vertex-centric depth-first traversal and a concise edge-based partition sequence. Additionally, we use classifier-free guidance combined with nucleus sampling to improve lane connectivity. We validate our method on prominent datasets, nuScenes and Argoverse 2, showcasing consistent and compelling results. Our LaneGraph2Seq approach demonstrates superior performance compared to state-of-the-art techniques in lane graph extraction., Comment: AAAI 2024

Multi-modal large language models (MLLMs) have demonstrated remarkable success in vision and visual-language tasks within the natural image domain. Owing to the significant diversities between the natural and remote sensing (RS) images, the development of MLLMs in the RS domain is still in the infant stage. To fill the gap, a pioneer MLLM named EarthGPT integrating various multi-sensor RS interpretation tasks uniformly is proposed in this paper for universal RS image comprehension. In EarthGPT, three key techniques are developed including a visual-enhanced perception mechanism, a cross-modal mutual comprehension approach, and a unified instruction tuning method for multi-sensor multi-task in the RS domain. More importantly, a dataset named MMRS-1M featuring large-scale multi-sensor multi-modal RS instruction-following is constructed, comprising over 1M image-text pairs based on 34 existing diverse RS datasets and including multi-sensor images such as optical, synthetic aperture radar (SAR), and infrared. The MMRS-1M dataset addresses the drawback of MLLMs on RS expert knowledge and stimulates the development of MLLMs in the RS domain. Extensive experiments are conducted, demonstrating the EarthGPT's superior performance in various RS visual interpretation tasks compared with the other specialist models and MLLMs, proving the effectiveness of the proposed EarthGPT and offering a versatile paradigm for open-set reasoning tasks.

Topological photonics provides a new degree of freedom to robustly control electromagnetic fields. To date, most of established topological states in photonics have been employed in Euclidean space. Motivated by unique properties of hyperbolic lattices, which are regular tessellations in non-Euclidean space with a constant negative curvature, the boundarydominated hyperbolic topological states have been proposed. However, limited by highly crowded boundary resonators and complicated site couplings, the hyperbolic topological insulator has only been experimentally constructed in electric circuits. How to achieve hyperbolic photonic topological insulators is still an open question. Here, we report the experimental realization of hyperbolic photonic topological insulators using coupled ring resonators on silicon chips. Boundary-dominated one-way edge states with pseudospindependent propagation directions have been observed. Furthermore, the robustness of edge states in hyperbolic photonic topological insulators is also verified. Our findings have potential applications in the field of designing high-efficient topological photonic devices with enhanced boundary responses.

Recent advancements in large language models (LLMs) have significantly enhanced capabilities in natural language processing and artificial intelligence. These models, including GPT-3.5 and LLaMA-2, have revolutionized text generation, translation, and question-answering tasks due to the transformative Transformer model. Despite their widespread use, LLMs present challenges such as ethical dilemmas when models are compelled to respond inappropriately, susceptibility to phishing attacks, and privacy violations. This paper addresses these challenges by introducing a multi-pronged approach that includes: 1) filtering sensitive vocabulary from user input to prevent unethical responses; 2) detecting role-playing to halt interactions that could lead to 'prison break' scenarios; 3) implementing custom rule engines to restrict the generation of prohibited content; and 4) extending these methodologies to various LLM derivatives like Multi-Model Large Language Models (MLLMs). Our approach not only fortifies models against unethical manipulations and privacy breaches but also maintains their high performance across tasks. We demonstrate state-of-the-art performance under various attack prompts, without compromising the model's core functionalities. Furthermore, the introduction of differentiated security levels empowers users to control their personal data disclosure. Our methods contribute to reducing social risks and conflicts arising from technological abuse, enhance data protection, and promote social equity. Collectively, this research provides a framework for balancing the efficiency of question-answering systems with user privacy and ethical standards, ensuring a safer user experience and fostering trust in AI technology.

Leveraging the unique properties of quantum entanglement, quantum entanglement distribution networks support multiple quantum information applications and are essential to the development of quantum networks. However, its practical implementation poses significant challenges to network scalability and flexibility. In this work, we propose a novel reconfigurable entanglement distribution network based on tunable multi-pump excitation of a spontaneous four-wave mixing (SFWM) source and a time-sharing method. We characterize the two-photon correlation under different pump conditions to demonstrate the effect of pump degenerate and pump non-degenerate SFWM processes on the two-photon correlation, and its tunability. Then as a benchmark application, a 10-user fully-connected quantum key distribution (QKD) network is established in a time-sharing way with triple pump lights. Each user receives one frequency channel thus it shows a linear scaling between the number of frequency channels and the user number in despite of the network topology. Our results thus provide a promising networking scheme for large-scale entanglement distribution networks owing to its scalability, functionality, and reconfigurability., Comment: 15 pages, 6 figures

Scene Text Recognition (STR) is an important and challenging upstream task for building structured information databases, that involves recognizing text within images of natural scenes. Although current state-of-the-art (SOTA) models for STR exhibit high performance, they typically suffer from low inference efficiency due to their reliance on hybrid architectures comprised of visual encoders and sequence decoders. In this work, we propose a VIsion Permutable extractor for fast and efficient Scene Text Recognition (SVIPTR), which achieves an impressive balance between high performance and rapid inference speeds in the domain of STR. Specifically, SVIPTR leverages a visual-semantic extractor with a pyramid structure, characterized by the Permutation and combination of local and global self-attention layers. This design results in a lightweight and efficient model and its inference is insensitive to input length. Extensive experimental results on various standard datasets for both Chinese and English scene text recognition validate the superiority of SVIPTR. Notably, the SVIPTR-T (Tiny) variant delivers highly competitive accuracy on par with other lightweight models and achieves SOTA inference speeds. Meanwhile, the SVIPTR-L (Large) attains SOTA accuracy in single-encoder-type models, while maintaining a low parameter count and favorable inference speed. Our proposed method provides a compelling solution for the STR challenge, which greatly benefits real-world applications requiring fast and efficient STR. The code is publicly available at https://github.com/cxfyxl/VIPTR., Comment: 10 pages, 4 figures, 6 tables

With the development of artificial intelligence, personalized learning has attracted much attention as an integral part of intelligent education. China, the United States, the European Union, and others have put forward the importance of personalized learning in recent years, emphasizing the realization of the organic combination of large-scale education and personalized training. The development of a personalized learning system oriented to learners' preferences and suited to learners' needs should be accelerated. This review provides a comprehensive analysis of the current situation of personalized learning and its key role in education. It discusses the research on personalized learning from multiple perspectives, combining definitions, goals, and related educational theories to provide an in-depth understanding of personalized learning from an educational perspective, analyzing the implications of different theories on personalized learning, and highlighting the potential of personalized learning to meet the needs of individuals and to enhance their abilities. Data applications and assessment indicators in personalized learning are described in detail, providing a solid data foundation and evaluation system for subsequent research. Meanwhile, we start from both student modeling and recommendation algorithms and deeply analyze the cognitive and non-cognitive perspectives and the contribution of personalized recommendations to personalized learning. Finally, we explore the challenges and future trajectories of personalized learning. This review provides a multidimensional analysis of personalized learning through a more comprehensive study, providing academics and practitioners with cutting-edge explorations to promote continuous progress in the field of personalized learning., Comment: 82 pages,5 figures

In this work, we proposed and experimentally demonstrated a photonic simulator for XY models, which is a typical kind of classical spin models. By encoding the XY spins on the phase term of the input light field, the corresponding XY Hamiltonian could be performed on the output light intensities. The simulator is mainly based on a programmable and reconfigurable optical vector-matrix multiplication system, which can map arbitrary XY models within the dimensionality limit. Here, we demonstrated the Berezinskii-Kosterlitz-Thouless transition in a two-dimensional XY model, in which the expectation values of some observables are calculated and consistent with the theory. Besides, we performed the ground state search of two 25-spin XY models with different spin connections and coupling strengths. Our proposal paves a new way to investigate the XY spin system.

The proton-proton ($pp$) fusion chain dominates the neutrino production from the Sun. The uncertainty of the predicted $pp$ neutrino flux is at the sub-percent level, whereas that of the best measurement is $\mathcal{O}(10\%)$. In this paper, we present the first result to measure the solar $pp$ neutrinos in the electron recoil energy range from 24 to 144 keV, using the PandaX-4T commissioning data with 0.63 tonne$\times$year exposure. The $pp$ neutrino flux is determined to be $(8.0 \pm 3.9 \,{\rm{(stat)}} \pm 10.0 \,{\rm{(syst)}} )\times 10^{10}\, $$\rm{s}^{-1} \rm{cm}^{-2}$, consistent with Standard Solar Model and existing measurements, corresponding to a flux upper limit of $23.3\times 10^{10}\, $$\rm{s}^{-1} \rm{cm}^{-2}$ at 90\% C.L.., Comment: 6 pages, 5 figures

Huge embedding tables in modern Deep Learning Recommender Models (DLRM) require prohibitively large memory during training and inference. Aiming to reduce the memory footprint of training, this paper proposes FIne-grained In-Training Embedding Dimension optimization (FIITED). Given the observation that embedding vectors are not equally important, FIITED adjusts the dimension of each individual embedding vector continuously during training, assigning longer dimensions to more important embeddings while adapting to dynamic changes in data. A novel embedding storage system based on virtually-hashed physically-indexed hash tables is designed to efficiently implement the embedding dimension adjustment and effectively enable memory saving. Experiments on two industry models show that FIITED is able to reduce the size of embeddings by more than 65% while maintaining the trained model's quality, saving significantly more memory than a state-of-the-art in-training embedding pruning method. On public click-through rate prediction datasets, FIITED is able to prune up to 93.75%-99.75% embeddings without significant accuracy loss., Comment: 16 pages, 9 figures

Main-group layered binary semiconductors, in particular, the III-VI alloys in the binary Ga-Te system are attracting increasing interest for a range of practical applications. The III-VI semiconductor, monoclinic gallium monotelluride (m-GaTe), has been recently used in high-sensitivity photodetectors/phototransistors and electronic memory applications due to its anisotropic properties yielding superior optical and electrical performance. Despite these applications, the origin of such anisotropy, namely the complex structural and bonding environments in GaTe nanostructures remain to be fully understood. In the present work, we report a comprehensive atomic-scale characterization of m-GaTe by state-of-the-art element-resolved atomic-scale microscopy experiments, enabling a direct measure of the in-plane anisotropy at the sub-Angstrom level. We show that these experimental images compare well with the results of first-principles modeling. Quantum-chemical bonding analyses provide a detailed picture of the atomic neighbor interactions within the layers, revealing that vertical Ga-Ga homopolar bonds get stronger when they are distorted and rotated, inducing the strong in-plane anisotropy. Beyond GaTe, using a systematic screening over the Materials Project database, we identify four additional low-symmetric layered crystals with similar distorted tetrahedral patterns, namely, GeP, GeAs, SiP, and SiAs. We thereby confirm the homopolar-bond-induced anisotropy is a more generic feature in these layered van-der-Waals materials., Comment: Accepted in Small Science

With the explosive growth of video data in real-world applications, a comprehensive representation of videos becomes increasingly important. In this paper, we address the problem of video scene recognition, whose goal is to learn a high-level video representation to classify scenes in videos. Due to the diversity and complexity of video contents in realistic scenarios, this task remains a challenge. Most existing works identify scenes for videos only from visual or textual information in a temporal perspective, ignoring the valuable information hidden in single frames, while several earlier studies only recognize scenes for separate images in a non-temporal perspective. We argue that these two perspectives are both meaningful for this task and complementary to each other, meanwhile, externally introduced knowledge can also promote the comprehension of videos. We propose a novel two-stream framework to model video representations from multiple perspectives, i.e. temporal and non-temporal perspectives, and integrate the two perspectives in an end-to-end manner by self-distillation. Besides, we design a knowledge-enhanced feature fusion and label prediction method that contributes to naturally introducing knowledge into the task of video scene recognition. Experiments conducted on a real-world dataset demonstrate the effectiveness of our proposed method.

Ensemble systems, pervasive in diverse scientific and engineering domains, pose challenges to existing control methods due to their massive scale and underactuated nature. This paper presents a dynamic moment approach to addressing theoretical and computational challenges in systems-theoretic analysis and control design for linear ensemble systems. We introduce the Legendre-moments and Legendre-moment transform, which maps an ensemble system defined on the $L^2$-space to a Legendre-moment system defined on the $\ell^2$-space. We show that this pair of systems is of one-to-one correspondence and shares the same controllability property. This equivalence admits the control of an ensemble system through the control of the corresponding Legendre-moment system and inspires a unified control design scheme for linear ensemble systems using structured truncated moment systems. In particular, we develop a sampling-free ensemble control design algorithm, then conduct error analysis for control design using truncated moment systems and derive error bounds with respect to the truncation orders, which are illustrated with numerical examples.

The emergence of pre-trained models has significantly impacted Natural Language Processing (NLP) and Computer Vision to relational datasets. Traditionally, these models are assessed through fine-tuned downstream tasks. However, this raises the question of how to evaluate these models more efficiently and more effectively. In this study, we explore a novel approach where we leverage the meta-features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose using the consistency between these representations and the meta-features as a metric for evaluating pre-trained models. Our method's effectiveness is demonstrated across various domains, including models with relational datasets, large language models and image models.

Recently, anomalous Floquet topological phases without static counterparts have been observed in different systems, where periodically driven models are realized to support a winding number of 1 and a pair of edge modes in each quasienergy gap. Here, we focus on cold atomic gases in optical lattices and propose a novel driving scheme that breaks rotation symmetry but maintains inversion symmetry of the instantaneous Hamiltonian, and discover a novel type of anomalous Floquet topological phase with winding number larger than 1. By analyzing the condition of band touching under symmetry constraint, we map out the phase diagram exactly by varying the driving parameters and discuss the quasienergy spectra of typical topological phases, which can present multiple pairs of edge modes within a single gap. Finally, we suggest to characterize the topology of such phases by detecting the band inversion surfaces via quench dynamics., Comment: 7 pages, 5 figures

In human-centric content generation, the pre-trained text-to-image models struggle to produce user-wanted portrait images, which retain the identity of individuals while exhibiting diverse expressions. This paper introduces our efforts towards personalized face generation. To this end, we propose a novel multi-modal face generation framework, capable of simultaneous identity-expression control and more fine-grained expression synthesis. Our expression control is so sophisticated that it can be specialized by the fine-grained emotional vocabulary. We devise a novel diffusion model that can undertake the task of simultaneously face swapping and reenactment. Due to the entanglement of identity and expression, it's nontrivial to separately and precisely control them in one framework, thus has not been explored yet. To overcome this, we propose several innovative designs in the conditional diffusion model, including balancing identity and expression encoder, improved midpoint sampling, and explicitly background conditioning. Extensive experiments have demonstrated the controllability and scalability of the proposed framework, in comparison with state-of-the-art text-to-image, face swapping, and face reenactment methods.

The rise of multimodal large language models (MLLMs) has spurred interest in language-based driving tasks. However, existing research typically focuses on limited tasks and often omits key multi-view and temporal information which is crucial for robust autonomous driving. To bridge these gaps, we introduce NuInstruct, a novel dataset with 91K multi-view video-QA pairs across 17 subtasks, where each task demands holistic information (e.g., temporal, multi-view, and spatial), significantly elevating the challenge level. To obtain NuInstruct, we propose a novel SQL-based method to generate instruction-response pairs automatically, which is inspired by the driving logical progression of humans. We further present BEV-InMLLM, an end-to-end method for efficiently deriving instruction-aware Bird's-Eye-View (BEV) features, language-aligned for large language models. BEV-InMLLM integrates multi-view, spatial awareness, and temporal semantics to enhance MLLMs' capabilities on NuInstruct tasks. Moreover, our proposed BEV injection module is a plug-and-play method for existing MLLMs. Our experiments on NuInstruct demonstrate that BEV-InMLLM significantly outperforms existing MLLMs, e.g. around 9% improvement on various tasks. We plan to release our NuInstruct for future research development.