Search

Median fins of fish-like swimmers play a crucial role in linear acceleration and maneuvering processes. However, few research focused on untethered robotic fish experiments. Imitating the behaviour of real tuna, we developed a free-swimming bionic tuna with a foldable dorsal fin. The erection of dorsal fin, at proper conditions, can reduce head heave by 50%, enhance linear acceleration by 15.7%, increase turning angular velocity by 32.78%, and turning radius decreasing by 33.13%. Conversely, erecting the dorsal fin increases the wetted surface area, resulting in decreased maximum speed and efficiency during steady swimming phase. This finding partially explains why tuna erect their median fins during maneuvers or acceleration and fold them afterward to reduce drag. In addition, we verified that folding the median fins after acceleration does not significantly affect locomotion efficiency. This study supports the application of morphing median fins in undulating underwater robots and helps to further understand the impact of median fins on fish locomotion., Comment: 7 pages, 5 figures

In incremental object detection, knowledge distillation has been proven to be an effective way to alleviate catastrophic forgetting. However, previous works focused on preserving the knowledge of old models, ignoring that images could simultaneously contain categories from past, present, and future stages. The co-occurrence of objects makes the optimization objectives inconsistent across different stages since the definition for foreground objects differs across various stages, which limits the model's performance greatly. To overcome this problem, we propose a method called ``Bridge Past and Future'' (BPF), which aligns models across stages, ensuring consistent optimization directions. In addition, we propose a novel Distillation with Future (DwF) loss, fully leveraging the background probability to mitigate the forgetting of old classes while ensuring a high level of adaptability in learning new classes. Extensive experiments are conducted on both Pascal VOC and MS COCO benchmarks. Without memory, BPF outperforms current state-of-the-art methods under various settings. The code is available at https://github.com/iSEE-Laboratory/BPF., Comment: Accepted to ECCV 2024

Vision and language navigation is a task that requires an agent to navigate according to a natural language instruction. Recent methods predict sub-goals on constructed topology map at each step to enable long-term action planning. However, they suffer from high computational cost when attempting to support such high-level predictions with GCN-like models. In this work, we propose an alternative method that facilitates navigation planning by considering the alignment between instructions and directed fidelity trajectories, which refers to a path from the initial node to the candidate locations on a directed graph without detours. This planning strategy leads to an efficient model while achieving strong performance. Specifically, we introduce a directed graph to illustrate the explored area of the environment, emphasizing directionality. Then, we firstly define the trajectory representation as a sequence of directed edge features, which are extracted from the panorama based on the corresponding orientation. Ultimately, we assess and compare the alignment between instruction and different trajectories during navigation to determine the next navigation target. Our method outperforms previous SOTA method BEVBert on RxR dataset and is comparable on R2R dataset while largely reducing the computational cost. Code is available: https://github.com/iSEE-Laboratory/VLN-PRET.

We study the domain adaptation task for action recognition, namely domain adaptive action recognition, which aims to effectively transfer action recognition power from a label-sufficient source domain to a label-free target domain. Since actions are performed by humans, it is crucial to exploit human cues in videos when recognizing actions across domains. However, existing methods are prone to losing human cues but prefer to exploit the correlation between non-human contexts and associated actions for recognition, and the contexts of interest agnostic to actions would reduce recognition performance in the target domain. To overcome this problem, we focus on uncovering human-centric action cues for domain adaptive action recognition, and our conception is to investigate two aspects of human-centric action cues, namely human cues and human-context interaction cues. Accordingly, our proposed Human-Centric Transformer (HCTransformer) develops a decoupled human-centric learning paradigm to explicitly concentrate on human-centric action cues in domain-variant video feature learning. Our HCTransformer first conducts human-aware temporal modeling by a human encoder, aiming to avoid a loss of human cues during domain-invariant video feature learning. Then, by a Transformer-like architecture, HCTransformer exploits domain-invariant and action-correlated contexts by a context encoder, and further models domain-invariant interaction between humans and action-correlated contexts. We conduct extensive experiments on three benchmarks, namely UCF-HMDB, Kinetics-NecDrone and EPIC-Kitchens-UDA, and the state-of-the-art performance demonstrates the effectiveness of our proposed HCTransformer., Comment: Accepted by TPAMI

Robotic grasping in clutters is a fundamental task in robotic manipulation. In this work, we propose an economic framework for 6-DoF grasp detection, aiming to economize the resource cost in training and meanwhile maintain effective grasp performance. To begin with, we discover that the dense supervision is the bottleneck of current SOTA methods that severely encumbers the entire training overload, meanwhile making the training difficult to converge. To solve the above problem, we first propose an economic supervision paradigm for efficient and effective grasping. This paradigm includes a well-designed supervision selection strategy, selecting key labels basically without ambiguity, and an economic pipeline to enable the training after selection. Furthermore, benefit from the economic supervision, we can focus on a specific grasp, and thus we devise a focal representation module, which comprises an interactive grasp head and a composite score estimation to generate the specific grasp more accurately. Combining all together, the EconomicGrasp framework is proposed. Our extensive experiments show that EconomicGrasp surpasses the SOTA grasp method by about 3AP on average, and with extremely low resource cost, for about 1/4 training time cost, 1/8 memory cost and 1/30 storage cost. Our code is available at https://github.com/iSEE-Laboratory/EconomicGrasp., Comment: 19 pages, 7 figures. Accepted in ECCV 2024!

Few-shot class-incremental learning (FSCIL) aims to learn sequential classes with limited samples in a few-shot fashion. Inherited from the classical class-incremental learning setting, the popular benchmark of FSCIL uses averaged accuracy (aAcc) and last-task averaged accuracy (lAcc) as the evaluation metrics. However, we reveal that such evaluation metrics may not provide adequate emphasis on the novel class performance, and the continual learning ability of FSCIL methods could be ignored under this benchmark. In this work, as a complement to existing metrics, we offer a new metric called generalized average accuracy (gAcc) which is designed to provide an extra equitable evaluation by incorporating different perspectives of the performance under the guidance of a parameter $\alpha$. We also present an overall metric in the form of the area under the curve (AUC) along the $\alpha$. Under the guidance of gAcc, we release the potential of intermediate features of the vision transformers to boost the novel-class performance. Taking information from intermediate layers which are less class-specific and more generalizable, we manage to rectify the final features, leading to a more generalizable transformer-based FSCIL framework. Without complex network designs or cumbersome training procedures, our method outperforms existing FSCIL methods at aAcc and gAcc on three datasets. See codes at https://github.com/iSEE-Laboratory/Revisting_FSCIL, Comment: Accepted to ECCV 2024

Quantum simulation of synthetic dynamic gauge field has attracted much attentions in recent years. There are two traditional ways to simulate gauge theories. One is to directly simulate the full Hamiltonian of gauge theories with local gauge symmetries. And the other is to engineer the projected Hamiltonian in one gauge subsector. In this work, we provide the third way towards the simulation of gauge theories based on \emph{gauge fixing}. To demonstrate this concept, we fix the gauge of an Ising lattice gauge field coupled with spinless fermions on a ladder geometry. After the gauge fixing, this gauge theory is reduced to a simpler model, in which fermions hop on a ladder with a fluctuating dynamical $\mathbb{Z}_{2}$ flux. Then we shows that this model can be realized via Floquet engineering in ultracold atomic gases. By analytical and numerical studies of this dynamical flux model, we deduce that there is confinement to deconfinement phase transition in the original unfixed gauge theory. This work paves the way to quantum simulate lattice gauge theory using the concept of gauge fixing, relevant both for condensed matter and high energy physics., Comment: 12 pages, 9 figures

With the rapid advancement in machine learning, the recognition and analysis of brain activity based on EEG and eye movement signals have attained a high level of sophistication. Utilizing deep learning models for learning EEG and eye movement features proves effective in classifying brain activities. A focused state indicates intense concentration on a task or thought. Distinguishing focused and unfocused states can be achieved through eye movement behaviors, reflecting variations in brain activities. By calculating binocular focusing point disparity in eye movement signals and integrating relevant EEG features, we propose an annotation method for focused states. The resulting comprehensive dataset, derived from raw data processed through a bio-acquisition device, includes both EEG features and focused labels annotated by eye movements. Extensive training and testing on several deep learning models, particularly the Transformer, yielded a 90.16% accuracy on the subject-dependent experiments. The validity of this approach was demonstrated, with cross-subject experiments, key frequency band and brain region analyses confirming its generalizability and providing physiological explanations.

We present EgoExo-Fitness, a new full-body action understanding dataset, featuring fitness sequence videos recorded from synchronized egocentric and fixed exocentric (third-person) cameras. Compared with existing full-body action understanding datasets, EgoExo-Fitness not only contains videos from first-person perspectives, but also provides rich annotations. Specifically, two-level temporal boundaries are provided to localize single action videos along with sub-steps of each action. More importantly, EgoExo-Fitness introduces innovative annotations for interpretable action judgement--including technical keypoint verification, natural language comments on action execution, and action quality scores. Combining all of these, EgoExo-Fitness provides new resources to study egocentric and exocentric full-body action understanding across dimensions of "what", "when", and "how well". To facilitate research on egocentric and exocentric full-body action understanding, we construct benchmarks on a suite of tasks (i.e., action classification, action localization, cross-view sequence verification, cross-view skill determination, and a newly proposed task of guidance-based execution verification), together with detailed analysis. Code and data will be available at https://github.com/iSEE-Laboratory/EgoExo-Fitness/tree/main., Comment: Accepted by ECCV2024

Universality often emerges in low-energy equilibrium physics of quantum many-body systems, despite their microscopic complexity and variety. Recently, there has been a growing interest in studying far-from-equilibrium dynamics of quantum many-body systems. Such dynamics usually involves highly excited states beyond the traditional low-energy theory description. Whether universal behaviors can also emerge in such non-equilibrium dynamics is a central issue at the frontier of quantum dynamics. Here we report the experimental observation of universal dynamics by monitoring the spin depolarization process in a solid-state NMR system described by an ensemble of randomly interacting spins. The spin depolarization can be related to temporal spin-spin correlation functions at high temperatures. We discover a remarkable phenomenon that these correlation functions obey a universal functional form. This experimental fact helps us identify the dominant interacting processes in the spin depolarization dynamics that lead to this universality. Our observation demonstrates the existence of universality even in non-equilibrium dynamics at high temperatures, thereby complementing the well-established universality in low-energy physics., Comment: 10 pages, 4 figures; Supplementary Information 26 pages, 11 figures, 2 tables

Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used to test the ability of vision-language models to understand and reason on knowledge present in both visual and textual data. However, most of the current VQA models use datasets that are primarily focused on English and a few major world languages, with images that are typically Western-centric. While recent efforts have tried to increase the number of languages covered on VQA datasets, they still lack diversity in low-resource languages. More importantly, although these datasets often extend their linguistic range via translation or some other approaches, they usually keep images the same, resulting in narrow cultural representation. To address these limitations, we construct CVQA, a new Culturally-diverse multilingual Visual Question Answering benchmark, designed to cover a rich set of languages and cultures, where we engage native speakers and cultural experts in the data collection process. As a result, CVQA includes culturally-driven images and questions from across 28 countries on four continents, covering 26 languages with 11 scripts, providing a total of 9k questions. We then benchmark several Multimodal Large Language Models (MLLMs) on CVQA, and show that the dataset is challenging for the current state-of-the-art models. This benchmark can serve as a probing evaluation suite for assessing the cultural capability and bias of multimodal models and hopefully encourage more research efforts toward increasing cultural awareness and linguistic diversity in this field.

Fluorescence detection is a commonly used analytical method with the advantages of fast response, good selectivity and low destructiveness. However, fluorescence detection, a single-mode detection method, has some limitations, such as background interference that affects the accuracy of the fluorescence signal, lack of visualization of the detection results, and low sensitivity for detecting low-concentration samples. In order to overcome the shortcomings of fluorescence single-mode detection, we used the dual-mode method of fluorescence and colorimetry to detect ascorbic acid. The dual-mode detection of AA by fluorescence and colorimetry in the probe system enhances the specificity and accuracy of the detection. This bimodal detection method solved the problem of low detection sensitivity in the low concentration range of the analytes to be tested, and was linear in the lower (0-50 {\mu}M) and higher (50-350 {\mu}M) concentration ranges, respectively, and had a lower detection limit (0.034 {\mu}M). This glutathione-based gold cluster assay is characterized by simplicity, rapidity and accuracy, and provides a new way for the quantitative analysis of ascorbic acid. In addition, the method was validated during the determination of AA in beverages, which has the advantages of high sensitivity and fast response time.

This paper explores a novel task ""Dexterous Grasp as You Say"" (DexGYS), enabling robots to perform dexterous grasping based on human commands expressed in natural language. However, the development of this field is hindered by the lack of datasets with natural human guidance; thus, we propose a language-guided dexterous grasp dataset, named DexGYSNet, offering high-quality dexterous grasp annotations along with flexible and fine-grained human language guidance. Our dataset construction is cost-efficient, with the carefully-design hand-object interaction retargeting strategy, and the LLM-assisted language guidance annotation system. Equipped with this dataset, we introduce the DexGYSGrasp framework for generating dexterous grasps based on human language instructions, with the capability of producing grasps that are intent-aligned, high quality and diversity. To achieve this capability, our framework decomposes the complex learning process into two manageable progressive objectives and introduce two components to realize them. The first component learns the grasp distribution focusing on intention alignment and generation diversity. And the second component refines the grasp quality while maintaining intention consistency. Extensive experiments are conducted on DexGYSNet and real world environment for validation., Comment: 9 pages, 7 figures

Iteratively improving and repairing source code with large language models (LLMs), known as refinement, has emerged as a popular way of generating programs that would be too complex to construct in one shot. Given a bank of test cases, together with a candidate program, an LLM can improve that program by being prompted with failed test cases. But it remains an open question how to best iteratively refine code, with prior work employing simple greedy or breadth-first strategies. We show here that refinement exposes an explore-exploit tradeoff: exploit by refining the program that passes the most test cases, or explore by refining a lesser considered program. We frame this as an arm-acquiring bandit problem, which we solve with Thompson Sampling. The resulting LLM-based program synthesis algorithm is broadly applicable: Across loop invariant synthesis, visual reasoning puzzles, and competition programming problems, we find that our new method can solve more problems using fewer language model calls.

A recent experiment has reported a nitrogen-doped lutetium hydride acheving a remarkable Tc of 294 K at just 1 GPa, significantly reducing the required pressure for obtaining room temperature superconductivity. However, subsequent experimental and theoretical investigations have encountered difficulties in replicating these results, leaving the structure of this Lu-H-N compound shrouded in uncertainty. Here, we propose a stable structure for Lu$_4$H$_7$N employing first-principles calculations. Our calculations reveal that Lu$_4$H$_7$N has a Tc of 1.044 K, which can be substantially enhanced to 11.721 K at 150 GPa, due to the increasing electron-phonon coupling (EPC). Notably, we delve into the nontrivial Z2 band topology of Lu$_4$H$_7$N, featuring discernible surface states near the Fermi level, and we explore its spin Hall conductivity characteristics. Furthermore, we find that the electron doping can enhance the EPC strength and Tc of Lu$_4$H$_7$N, such as the Lu$_4$H$_7$O structure we predict simulating electron doping for Lu$_4$H$_7$N with an impressive Tc of 3.837 K. This work demonstrates the coexistence of superconducting and topological properties in a Lu-H-N system compound, which holds the promise of guiding the search for novel topological superconducting materials.

In this work, we propose a novel discriminative framework for dexterous grasp generation, named Dexterous Grasp TRansformer (DGTR), capable of predicting a diverse set of feasible grasp poses by processing the object point cloud with only one forward pass. We formulate dexterous grasp generation as a set prediction task and design a transformer-based grasping model for it. However, we identify that this set prediction paradigm encounters several optimization challenges in the field of dexterous grasping and results in restricted performance. To address these issues, we propose progressive strategies for both the training and testing phases. First, the dynamic-static matching training (DSMT) strategy is presented to enhance the optimization stability during the training phase. Second, we introduce the adversarial-balanced test-time adaptation (AB-TTA) with a pair of adversarial losses to improve grasping quality during the testing phase. Experimental results on the DexGraspNet dataset demonstrate the capability of DGTR to predict dexterous grasp poses with both high quality and diversity. Notably, while keeping high quality, the diversity of grasp poses predicted by DGTR significantly outperforms previous works in multiple metrics without any data pre-processing. Codes are available at https://github.com/iSEE-Laboratory/DGTR ., Comment: Accepted to CVPR 2024

Confinement/deconfinement, captivating attributes of high-energy elementary particles, have recently garnered wide attention in quantum simulations based on cold atoms. Yet, the partial confinement, an intermediate state between the confinement and deconfinement, remains underexplored. The partial confinement encapsulates the phenomenon that the confining behavior of charged particles is contingent upon their relative positions. In this paper, we demonstrate that the spin-1 quantum link model provides an excellent platform for exploring partial confinement. We conduct a comprehensive investigation of the physics emerging from partial confinement in both the context of equilibrium and non-equilibrium dynamics. Potential experimental setups using cold atoms are also discussed. Our work offers a simple and feasible routine for the study of confinement-related physics in the state-of-the-art artificial quantum systems subject to gauge symmetries.

Floquet engineering is a powerful manipulation method in modern quantum technology. However, unwanted heating is the main challenge of Floquet engineering, therefore the Floquet thermalization has attracting considerable attentions recently. In this work, we investigate thermalization of periodically driven many-body systems through the lens of Krylov complexity, and find a topological origin of different thermalization behaviors. We demonstrate that If the topology of the Krylov chain is nontrivial, a periodically driven system will reach a state with finite temperature. When the Krylov chain is topologically trivial, the system will be heated to infinite temperature. We further show that the prethermalization can be understood as the tunnelling process of a quasi-edge mode through the local gap on Krylov chain. This picture provides a systematically method to obtain the effective prethermal Hamiltonian., Comment: 6 pages, 5 figures

In this work, we explore a novel task of generating human grasps based on single-view scene point clouds, which more accurately mirrors the typical real-world situation of observing objects from a single viewpoint. Due to the incompleteness of object point clouds and the presence of numerous scene points, the generated hand is prone to penetrating into the invisible parts of the object and the model is easily affected by scene points. Thus, we introduce S2HGrasp, a framework composed of two key modules: the Global Perception module that globally perceives partial object point clouds, and the DiffuGrasp module designed to generate high-quality human grasps based on complex inputs that include scene points. Additionally, we introduce S2HGD dataset, which comprises approximately 99,000 single-object single-view scene point clouds of 1,668 unique objects, each annotated with one human grasp. Our extensive experiments demonstrate that S2HGrasp can not only generate natural human grasps regardless of scene points, but also effectively prevent penetration between the hand and invisible parts of the object. Moreover, our model showcases strong generalization capability when applied to unseen objects. Our code and dataset are available at https://github.com/iSEE-Laboratory/S2HGrasp.

Plasma disruption presents a significant challenge in tokamak fusion, where it can cause severe damage and economic losses. Current disruption predictors mainly rely on data-driven methods, requiring extensive discharge data for training. However, future tokamaks require disruption prediction from the first shot, posing challenges of data scarcity during the early operation period. In this period disruption prediction aims to support safe exploration of operation range and accumulate necessary data to develop advanced prediction models. Thus, predictors must adapt to evolving plasma environments during this exploration phase. To address these issues, this study proposes a cross-tokamak adaptive deployment method using the Enhanced Convolutional Autoencoder Anomaly Detection (E-CAAD) predictor, enabling disruption prediction from the first shot of new devices. Experimental results indicate the ability of E-CAAD model trained on existing devices to effectively differentiate between disruption precursors and non-disruption samples on new devices, proving the feasibility of model cross-device transfer. Building upon this, adaptive learning from scratch and threshold adaptive adjustment strategies are proposed to achieve model cross-device transfer. The adaptive learning from scratch strategy enables the predictor to use scarce data during the early operation of the new device while rapidly adapting to changes in operation environment. The threshold adaptive adjustment strategy addresses the challenge of selecting warning thresholds on new devices where validation set is lacking, ensuring that the warning thresholds adapt to changes in the operation environment. Finally, experiments transferring the model from J-TEXT to EAST exhibit comparable performance to EAST models trained with ample data, achieving a TPR of 85.88% and a FPR of 6.15%, with a 20ms reserved MGI system reaction time., Comment: 18 pages, 7 figures

The topological states of the two-leg and three-leg ladders formed by two trivial quantum wires with different lattice constants are theoretically investigated. For the symmetric nearest-neighbor intra-chain hopping two-leg ladder, the inversion symmetry topological insulator phase with two degenerate topological edge states appears. When the inversion symmetry is broken, the topological insulators with one or two topological edge states of different energies and topological metals with edge states embedded in the bulk states could emerge dependent on the filling factor. The topological origin of these topological states in the two-leg ladders is the topological properties of the Chern insulators and Chern metals. According to the arrangement of two trivial quantum wires, we construct two types of three-leg ladders. Each type of the three-leg ladder could be divided into one trivial subspace and one topological nontrivial subspace by unitary transformation. The topological nontrivial subspace corresponds to the effective two-leg ladder model. As the filling factor changes, the system could be in topological insulators or topological metals phases. These rich topological states in the two-leg and three-leg ladders could be confirmed by current experimental techniques.

Text-to-3D generation, which synthesizes 3D assets according to an overall text description, has significantly progressed. However, a challenge arises when the specific appearances need customizing at designated viewpoints but referring solely to the overall description for generating 3D objects. For instance, ambiguity easily occurs when producing a T-shirt with distinct patterns on its front and back using a single overall text guidance. In this work, we propose DreamView, a text-to-image approach enabling multi-view customization while maintaining overall consistency by adaptively injecting the view-specific and overall text guidance through a collaborative text guidance injection module, which can also be lifted to 3D generation via score distillation sampling. DreamView is trained with large-scale rendered multi-view images and their corresponding view-specific texts to learn to balance the separate content manipulation in each view and the global consistency of the overall object, resulting in a dual achievement of customization and consistency. Consequently, DreamView empowers artists to design 3D objects creatively, fostering the creation of more innovative and diverse 3D assets. Code and model will be released at https://github.com/iSEE-Laboratory/DreamView., Comment: Accepted to ECCV 2024, camera ready version

The origin of intermediate helium (He)-rich hot subdwarfs are still unclear. Previous studies have suggested that some surviving Type Ia supernovae (SNe Ia) companions from the white dwarf~+~main-sequence (WD+MS) channel may contribute to the intermediate He-rich hot subdwarfs. However, previous studies ignored the impact of atomic diffusion on the post-explosion evolution of surviving companion stars of SNe Ia, leading to that they could not explain the observed surface He abundance of intermediate He-rich hot subdwarfs. In this work, by taking the atomic diffusion and stellar wind into account, we trace the surviving companions of SNe Ia from the WD+MS channel using the one-dimensional stellar evolution code \textsc{MESA} until they evolve into hot subdwarfs. We find that the surface He-abundances of our surviving companion models during their core He-burning phases are in a range of $-1 \lesssim {\rm log}(N_{\rm He}/N_{\rm H}) \lesssim 0$, which are consistent with those observed in intermediate He-rich hot subdwarfs. This seems to further support that surviving companions of SNe Ia in the WD+MS channel are possible to form some intermediate He-rich hot subdwarfs., Comment: 10 pages, 5 figures

Weakly-supervised action segmentation is a task of learning to partition a long video into several action segments, where training videos are only accompanied by transcripts (ordered list of actions). Most of existing methods need to infer pseudo segmentation for training by serial alignment between all frames and the transcript, which is time-consuming and hard to be parallelized while training. In this work, we aim to escape from this inefficient alignment with massive but redundant frames, and instead to directly localize a few action transitions for pseudo segmentation generation, where a transition refers to the change from an action segment to its next adjacent one in the transcript. As the true transitions are submerged in noisy boundaries due to intra-segment visual variation, we propose a novel Action-Transition-Aware Boundary Alignment (ATBA) framework to efficiently and effectively filter out noisy boundaries and detect transitions. In addition, to boost the semantic learning in the case that noise is inevitably present in the pseudo segmentation, we also introduce video-level losses to utilize the trusted video-level supervision. Extensive experiments show the effectiveness of our approach on both performance and training speed., Comment: Accepted to CVPR 2024

This paper focuses on open-ended video question answering, which aims to find the correct answers from a large answer set in response to a video-related question. This is essentially a multi-label classification task, since a question may have multiple answers. However, due to annotation costs, the labels in existing benchmarks are always extremely insufficient, typically one answer per question. As a result, existing works tend to directly treat all the unlabeled answers as negative labels, leading to limited ability for generalization. In this work, we introduce a simple yet effective ranking distillation framework (RADI) to mitigate this problem without additional manual annotation. RADI employs a teacher model trained with incomplete labels to generate rankings for potential answers, which contain rich knowledge about label priority as well as label-associated visual cues, thereby enriching the insufficient labeling information. To avoid overconfidence in the imperfect teacher model, we further present two robust and parameter-free ranking distillation approaches: a pairwise approach which introduces adaptive soft margins to dynamically refine the optimization constraints on various pairwise rankings, and a listwise approach which adopts sampling-based partial listwise learning to resist the bias in teacher ranking. Extensive experiments on five popular benchmarks consistently show that both our pairwise and listwise RADIs outperform state-of-the-art methods. Further analysis demonstrates the effectiveness of our methods on the insufficient labeling problem., Comment: Accepted to CVPR 2024

As networked control systems continue to evolve, ensuring the privacy of sensitive data becomes an increasingly pressing concern, especially in situations where the controller is physically separated from the plant. In this paper, we propose a secure control scheme for computing linear quadratic control in a networked control system utilizing two networked controllers, a privacy encoder and a control restorer. Specifically, the encoder generates two state signals blurred with random noise and sends them to the controllers, while the restorer reconstructs the correct control signal. The proposed design effectively preserves the privacy of the control system's state without sacrificing the control performance. We theoretically quantify the privacy-preserving performance in terms of the state estimation error of the controllers and the disclosure probability. Additionally, the proposed privacy-preserving scheme is also proven to satisfy differential privacy. Moreover, we extend the proposed privacy-preserving scheme and evaluation method to cases where collusion between two controllers occurs. Finally, we verify the validity of our proposed scheme through simulations.

Video Paragraph Grounding (VPG) is an emerging task in video-language understanding, which aims at localizing multiple sentences with semantic relations and temporal order from an untrimmed video. However, existing VPG approaches are heavily reliant on a considerable number of temporal labels that are laborious and time-consuming to acquire. In this work, we introduce and explore Weakly-Supervised Video Paragraph Grounding (WSVPG) to eliminate the need of temporal annotations. Different from previous weakly-supervised grounding frameworks based on multiple instance learning or reconstruction learning for two-stage candidate ranking, we propose a novel siamese learning framework that jointly learns the cross-modal feature alignment and temporal coordinate regression without timestamp labels to achieve concise one-stage localization for WSVPG. Specifically, we devise a Siamese Grounding TRansformer (SiamGTR) consisting of two weight-sharing branches for learning complementary supervision. An Augmentation Branch is utilized for directly regressing the temporal boundaries of a complete paragraph within a pseudo video, and an Inference Branch is designed to capture the order-guided feature correspondence for localizing multiple sentences in a normal video. We demonstrate by extensive experiments that our paradigm has superior practicability and flexibility to achieve efficient weakly-supervised or semi-supervised learning, outperforming state-of-the-art methods trained with the same or stronger supervision., Comment: Accepted to CVPR 2024. v2: fix a typo in figure 1

La$_{3}$Ni$_{2}$O$_{7}$ has garnered widespread interest recently due to its high-temperature superconductivity under pressure, accompanied by charge density wave (CDW) ordering and metal-insulator (MI) transitions in the phase diagram. Here, we reveal with comprehensive calculations that La$_{3}$Ni$_{2}$O$_{7}$ possesses an antiferromagnetic ground state under both low and high pressures, with the strong Fermi surface nesting contributed by the flat band that leads to phonon softening and electronic instabilities. Several stable CDW orders with oxygen octahedral distortions are identified, which can trigger the MI transitions. The estimated CDW transition temperature ($\approx$120 K) at ambient pressure agrees nicely with experimental results. In the presence of apical oxygen vacancies, we identify two different phases, say, half distortion and full distortion phases, respectively, and their competition can lead to a pressure-induced MI transition, in good agreement with experimental observations. In addition, we find that the electron-phonon coupling is too small to contribute to superconductivity. These results appear to indicate an unconventional superconducting pairing mechanism mediated by antiferromagnetic fluctuations. A phase diagram that is consistent with the experimental results is given. The present results not only explain the origins of experimentally observed CDW and MI transitions, but also provide insight for deeply understanding the properties like superconductivity, CDW and the role of oxygen vacancies in pressurized La$_{3}$Ni$_{2}$O$_{7}$.

Universal image restoration is a practical and potential computer vision task for real-world applications. The main challenge of this task is handling the different degradation distributions at once. Existing methods mainly utilize task-specific conditions (e.g., prompt) to guide the model to learn different distributions separately, named multi-partite mapping. However, it is not suitable for universal model learning as it ignores the shared information between different tasks. In this work, we propose an advanced selective hourglass mapping strategy based on diffusion model, termed DiffUIR. Two novel considerations make our DiffUIR non-trivial. Firstly, we equip the model with strong condition guidance to obtain accurate generation direction of diffusion model (selective). More importantly, DiffUIR integrates a flexible shared distribution term (SDT) into the diffusion algorithm elegantly and naturally, which gradually maps different distributions into a shared one. In the reverse process, combined with SDT and strong condition guidance, DiffUIR iteratively guides the shared distribution to the task-specific distribution with high image quality (hourglass). Without bells and whistles, by only modifying the mapping strategy, we achieve state-of-the-art performance on five image restoration tasks, 22 benchmarks in the universal setting and zero-shot generalization setting. Surprisingly, by only using a lightweight model (only 0.89M), we could achieve outstanding performance. The source code and pre-trained models are available at https://github.com/iSEE-Laboratory/DiffUIR, Comment: Accepted to CVPR2024

Person Re-identification (ReID) has been extensively developed for a decade in order to learn the association of images of the same person across non-overlapping camera views. To overcome significant variations between images across camera views, mountains of variants of ReID models were developed for solving a number of challenges, such as resolution change, clothing change, occlusion, modality change, and so on. Despite the impressive performance of many ReID variants, these variants typically function distinctly and cannot be applied to other challenges. To our best knowledge, there is no versatile ReID model that can handle various ReID challenges at the same time. This work contributes to the first attempt at learning a versatile ReID model to solve such a problem. Our main idea is to form a two-stage prompt-based twin modeling framework called VersReID. Our VersReID firstly leverages the scene label to train a ReID Bank that contains abundant knowledge for handling various scenes, where several groups of scene-specific prompts are used to encode different scene-specific knowledge. In the second stage, we distill a V-Branch model with versatile prompts from the ReID Bank for adaptively solving the ReID of different scenes, eliminating the demand for scene labels during the inference stage. To facilitate training VersReID, we further introduce the multi-scene properties into self-supervised learning of ReID via a multi-scene prioris data augmentation (MPDA) strategy. Through extensive experiments, we demonstrate the success of learning an effective and versatile ReID model for handling ReID tasks under multi-scene conditions without manual assignment of scene labels in the inference stage, including general, low-resolution, clothing change, occlusion, and cross-modality scenes. Codes and models are available at https://github.com/iSEE-Laboratory/VersReID., Comment: To appear in TPAMI

Human avatar has become a novel type of 3D asset with various applications. Ideally, a human avatar should be fully customizable to accommodate different settings and environments. In this work, we introduce NECA, an approach capable of learning versatile human representation from monocular or sparse-view videos, enabling granular customization across aspects such as pose, shadow, shape, lighting and texture. The core of our approach is to represent humans in complementary dual spaces and predict disentangled neural fields of geometry, albedo, shadow, as well as an external lighting, from which we are able to derive realistic rendering with high-frequency details via volumetric rendering. Extensive experiments demonstrate the advantage of our method over the state-of-the-art methods in photorealistic rendering, as well as various editing tasks such as novel pose synthesis and relighting. The code is available at https://github.com/iSEE-Laboratory/NECA., Comment: Accepted to CVPR 2024

Upconversion persistent luminescence (UCPL) phosphors that can be directly charged by near-infrared (NIR) light have gained considerable attention due to their promising applications ranging from photonics to biomedicine. However, current lanthanide-based UCPL phosphors show small absorption cross-sections and low upconversion charging efficiency. The development of UCPL phosphors faces challenges of lacking flexible upconversion charging pathways and poor design flexibility. Herein, we discovered a new lattice defect-mediated broadband photon upconversion process and the accompanied NIR-to-NIR UCPL in Cr-doped zinc gallate nanoparticles. The zinc gallate nanoparticles can be directly activated by broadband NIR light in the 700-1000 nm range to produce persistent luminescence at about 700 nm, which is also readily enhanced by rationally tailoring the lattice defects in the phosphors. This proposed UCPL phosphors achieved a signal-to-background ratio of over 200 in bioimaging by efficiently avoiding interference from autofluorescence and light scattering. Our findings reported the lattice defect-mediated photon upconversion for the first time, which significantly expanded the horizons for the flexible design of NIR-to-NIR UCPL phosphors toward broad applications.

The phenomenon of quantum many-body scars has received widespread attention both in theoretical and experimental physics in recent years due to its unique physical properties. In this paper, based on the $su(2)$ algebraic relations, we propose a general method for constructing scar models by combining simple modules.This allows us to investigate many-body scar phenomena in high-spin systems. We numerically verify the thermalization and non-integrability of this model and demonstrate the dynamical properties of the scar states. We also provide a theoretical analysis of the properties of these scar states. For spin-$1$ case, we find that our 1D chain model reduces to the famous PXP model[C. J. Turner et al. Phys. Rev. B 98, 155134(2018)] under special parameter condition. In addition, due to the continuous tunability of the parameters, our model also enables us to investigate the transitions of QMBS from non-integrable to integrable system., Comment: 12 pages, 7 figures

Building upon the impressive success of CLIP (Contrastive Language-Image Pretraining), recent pioneer works have proposed to adapt the powerful CLIP to video data, leading to efficient and effective video learners for open-vocabulary action recognition. Inspired by that humans perform actions in diverse environments, our work delves into an intriguing question: Can CLIP-based video learners effectively generalize to video domains they have not encountered during training? To answer this, we establish a CROSS-domain Open-Vocabulary Action recognition benchmark named XOV-Action, and conduct a comprehensive evaluation of five state-of-the-art CLIP-based video learners under various types of domain gaps. The evaluation demonstrates that previous methods exhibit limited action recognition performance in unseen video domains, revealing potential challenges of the cross-domain open-vocabulary action recognition task. In this paper, we focus on one critical challenge of the task, namely scene bias, and accordingly contribute a novel scene-aware video-text alignment method. Our key idea is to distinguish video representations apart from scene-encoded text representations, aiming to learn scene-agnostic video representations for recognizing actions across domains. Extensive experiments demonstrate the effectiveness of our method. The benchmark and code will be available at https://github.com/KunyuLin/XOV-Action/.

Artificial monopoles have been engineered in various systems, yet there has been no systematic study of the singular vector potentials associated with the monopole field. We show that the Dirac string, the line singularity of the vector potential, can be engineered, manipulated, and made manifest in a spinor atomic condensate. We elucidate the connection among spin, orbital degrees of freedom, and the artificial gauge, and show that there exists a mapping between the vortex filament and the Dirac string. We also devise a proposal where preparing initial spin states with relevant symmetries can result in different vortex patterns, revealing an underlying correspondence between the internal spin states and the spherical vortex structures. Such a mapping also leads to a new way of constructing spherical Landau levels, and monopole harmonics. Our observation provides insights into the behavior of quantum matter possessing internal symmetries in curved spaces., Comment: 5 pages + 5 figures + 5 appendices. In comparison with the previous version, we have (1) added supplementary material as appendices, and (2) made some minor revisions in the main text. Also, a simulation video for creating two artificial Dirac Strings with a mixed spin-1 state, is available at https://youtu.be/PCQGMqd-DfQ

Mrk 501 is a prototypical high-synchrotron-peaked blazar (HBL) and serves as one of the primary targets for the {\it Imaging X-ray Polarimetry Explorer} ({\it IXPE}). In this study, we report X-ray polarization measurements of Mrk 501 based on six {\it IXPE} observations. The detection of X-ray polarization at a confidence level exceeding 99\% is achieved in four out of the six observations conducted across the entire energy range (2--8 keV) of {\it IXPE}. The maximum polarization degree ($\Pi_{\rm X}$) is measured to be $15.8\%\pm2.8\%$, accompanied by a polarization angle ($\psi_{\rm X}$) of $98.0\deg\pm5.1\deg$ at a confidence level of $5.6 \sigma$. During the remaining two observations, only an upper limit of $\Pi_{\rm X}<$12\% could be derived at the 99\% confidence level. No temporal variability in polarization is observed throughout all six {\it IXPE} observations for Mrk 501. A discernible trend of energy-dependent variation in the polarization degree is detected in optical spectropolarimetry; however, no analogous indication is observed in $\Pi_{\rm X}$. The chromatic behavior of $\Pi$ and the consistent values of $\psi$ across different frequencies from X-rays to radio waves, along with the agreement between $\psi$ and jet position angle, strongly support the interpretation of the energy-stratified model with shock-accelerated particles in the jet of Mrk 501. Additionally, the possibility of the presence of a global helical magnetic field in the jet of Mrk 501 is discussed., Comment: 20 pages, 8 figures, 4 tables, accepted for publication in ApJL

A good translation should be faithful to the source and should respect the norms of the target language. We address a theoretical puzzle about the relationship between these objectives. On one hand, intuition and some prior work suggest that accuracy and fluency should trade off against each other, and that capturing every detail of the source can only be achieved at the cost of fluency. On the other hand, quality assessment researchers often suggest that accuracy and fluency are highly correlated and difficult for human raters to distinguish (Callison-Burch et al., 2007). We show that the tension between these views is an instance of Simpson's paradox, and that accuracy and fluency are positively correlated at the level of the corpus but trade off at the level of individual source segments. We further suggest that the relationship between accuracy and fluency is best evaluated at the segment (or sentence) level, and that the trade off between these dimensions has implications both for assessing translation quality and developing improved MT systems.

With the development of mobile communication and industrial internet technologies, the demand for robust absolute time synchronization based on network for diverse scenarios is significantly growing. TAP is a novel network timing method that aims to achieve sub-microsecond synchronization over air interface. This paper investigates the improvement and end-to-end realization of TAP. This paper first analyzes the effectiveness and deficiencies of TAP by establishing an equivalent clock model which evaluates TAP from timing error composition and allan variance. Second, this paper proposes a detailed base station and terminal design and the corresponding improvement of TAP. Both hardware compensation and protocol software design are taken into account so as to minimize timing error and system cost while maximizing compatibility with 3GPP. Finally, this paper presents a TAP end-to-end 5G prototype system developed based on software defined radio base station and COTS baseband module. The field test results show that the proposed scheme effectively solves the problems of TAP in application and robustly achieves 200ns level timing accuracy in various situations. The average accuracy with long observations can reach 1 nanosecond. It is 2$\sim$3 orders of magnitude better than common network timing methods, including NTP, PTP and the original TAP.

Action quality assessment (AQA) is to assess how well an action is performed. Previous works perform modelling by only the use of visual information, ignoring audio information. We argue that although AQA is highly dependent on visual information, the audio is useful complementary information for improving the score regression accuracy, especially for sports with background music, such as figure skating and rhythmic gymnastics. To leverage multimodal information for AQA, i.e., RGB, optical flow and audio information, we propose a Progressive Adaptive Multimodal Fusion Network (PAMFN) that separately models modality-specific information and mixed-modality information. Our model consists of with three modality-specific branches that independently explore modality-specific information and a mixed-modality branch that progressively aggregates the modality-specific information from the modality-specific branches. To build the bridge between modality-specific branches and the mixed-modality branch, three novel modules are proposed. First, a Modality-specific Feature Decoder module is designed to selectively transfer modality-specific information to the mixed-modality branch. Second, when exploring the interaction between modality-specific information, we argue that using an invariant multimodal fusion policy may lead to suboptimal results, so as to take the potential diversity in different parts of an action into consideration. Therefore, an Adaptive Fusion Module is proposed to learn adaptive multimodal fusion policies in different parts of an action. This module consists of several FusionNets for exploring different multimodal fusion strategies and a PolicyNet for deciding which FusionNets are enabled. Third, a module called Cross-modal Feature Decoder is designed to transfer cross-modal features generated by Adaptive Fusion Module to the mixed-modality branch., Comment: IEEE Transactions on Image Processing 2024

Zero-shot action recognition (ZSAR) aims to learn an alignment model between videos and class descriptions of seen actions that is transferable to unseen actions. The text queries (class descriptions) used in existing ZSAR works, however, are often short action names that fail to capture the rich semantics in the videos, leading to misalignment. With the intuition that video content descriptions (e.g., video captions) can provide rich contextual information of visual concepts in videos, we propose to utilize human annotated video descriptions to enrich the semantics of the class descriptions of each action. However, all existing action video description datasets are limited in terms of the number of actions, the semantics of video descriptions, etc. To this end, we collect a large-scale action video descriptions dataset named ActionHub, which covers a total of 1,211 common actions and provides 3.6 million action video descriptions. With the proposed ActionHub dataset, we further propose a novel Cross-modality and Cross-action Modeling (CoCo) framework for ZSAR, which consists of a Dual Cross-modality Alignment module and a Cross-action Invariance Mining module. Specifically, the Dual Cross-modality Alignment module utilizes both action labels and video descriptions from ActionHub to obtain rich class semantic features for feature alignment. The Cross-action Invariance Mining module exploits a cycle-reconstruction process between the class semantic feature spaces of seen actions and unseen actions, aiming to guide the model to learn cross-action invariant representations. Extensive experimental results demonstrate that our CoCo framework significantly outperforms the state-of-the-art on three popular ZSAR benchmarks (i.e., Kinetics-ZSAR, UCF101 and HMDB51) under two different learning protocols in ZSAR. We will release our code, models, and the proposed ActionHub dataset.

Human translators linger on some words and phrases more than others, and predicting this variation is a step towards explaining the underlying cognitive processes. Using data from the CRITT Translation Process Research Database, we evaluate the extent to which surprisal and attentional features derived from a Neural Machine Translation (NMT) model account for reading and production times of human translators. We find that surprisal and attention are complementary predictors of translation difficulty, and that surprisal derived from a NMT model is the single most successful predictor of production duration. Our analyses draw on data from hundreds of translators operating across 13 language pairs, and represent the most comprehensive investigation of human translation difficulty to date.

Text-to-image person re-identification (ReID) aims to retrieve images of a person based on a given textual description. The key challenge is to learn the relations between detailed information from visual and textual modalities. Existing works focus on learning a latent space to narrow the modality gap and further build local correspondences between two modalities. However, these methods assume that image-to-text and text-to-image associations are modality-agnostic, resulting in suboptimal associations. In this work, we show the discrepancy between image-to-text association and text-to-image association and propose CADA: Cross-Modal Adaptive Dual Association that finely builds bidirectional image-text detailed associations. Our approach features a decoder-based adaptive dual association module that enables full interaction between visual and textual modalities, allowing for bidirectional and adaptive cross-modal correspondence associations. Specifically, the paper proposes a bidirectional association mechanism: Association of text Tokens to image Patches (ATP) and Association of image Regions to text Attributes (ARA). We adaptively model the ATP based on the fact that aggregating cross-modal features based on mistaken associations will lead to feature distortion. For modeling the ARA, since the attributes are typically the first distinguishing cues of a person, we propose to explore the attribute-level association by predicting the masked text phrase using the related image region. Finally, we learn the dual associations between texts and images, and the experimental results demonstrate the superiority of our dual formulation. Codes will be made publicly available.