Search

Skeleton-based Temporal Action Segmentation involves the dense action classification of variable-length skeleton sequences. Current approaches primarily apply graph-based networks to extract framewise, whole-body-level motion representations, and use one-hot encoded labels for model optimization. However, whole-body motion representations do not capture fine-grained part-level motion representations and the one-hot encoded labels neglect the intrinsic semantic relationships within the language-based action definitions. To address these limitations, we propose a novel method named Language-assisted Human Part Motion Representation Learning (LPL), which contains a Disentangled Part Motion Encoder (DPE) to extract dual-level (i.e., part and whole-body) motion representations and a Language-assisted Distribution Alignment (LDA) strategy for optimizing spatial relations within representations. Specifically, after part-aware skeleton encoding via DPE, LDA generates dual-level action descriptions to construct a textual embedding space with the help of a large-scale language model. Then, LDA motivates the alignment of the embedding space between text descriptions and motions. This alignment allows LDA not only to enhance intra-class compactness but also to transfer the language-encoded semantic correlations among actions to skeleton-based motion learning. Moreover, we propose a simple yet efficient Semantic Offset Adapter to smooth the cross-domain misalignment. Our experiments indicate that LPL achieves state-of-the-art performance across various datasets (e.g., +4.4\% Accuracy, +5.6\% F1 on the PKU-MMD dataset). Moreover, LDA is compatible with existing methods and improves their performance (e.g., +4.8\% Accuracy, +4.3\% F1 on the LARa dataset) without additional inference costs., Comment: This work has been submitted to the IEEE for possible publication

Super-Resolution Ultrasound (SRUS) imaging through localising and tracking microbubbles, also known as Ultrasound Localisation Microscopy (ULM), has demonstrated significant potential for reconstructing microvasculature and flows with sub-diffraction resolution in clinical diagnostics. However, imaging organs with large tissue movements, such as those caused by respiration, presents substantial challenges. Existing methods often require breath holding to maintain accumulation accuracy, which limits data acquisition time and ULM image saturation. To improve image quality in the presence of large tissue movements, this study introduces an approach integrating high-frame-rate ultrasound with online precise robotic probe control. Tested on a microvasculature phantom with translation motions up to 20 mm, twice the aperture size of the matrix array used, our method achieved real-time tracking of the moving phantom and imaging volume rate at 85 Hz, keeping majority of the target volume in the imaging field of view. ULM images of the moving cross channels in the phantom were successfully reconstructed in post-processing, demonstrating the feasibility of super-resolution imaging under large tissue motions. This represents a significant step towards ULM imaging of organs with large motion.

Microgrid serves as a promising solution to integrate and manage distributed renewable energy resources. In this paper, we establish a stochastic multi-objective sizing optimization (SMOSO) model for microgrid planning, which fully captures the battery degradation characteristics and the total carbon emissions. The microgrid operator aims to simultaneously maximize the economic benefits and minimize carbon emissions, and the degradation of the battery energy storage system (BESS) is modeled as a nonlinear function of power throughput. A self-adaptive multi-objective genetic algorithm (SAMOGA) is proposed to solve the SMOSO model, and this algorithm is enhanced by pre-grouped hierarchical selection and self-adaptive probabilities of crossover and mutation. Several case studies are conducted to determine the microgrid size by analyzing Pareto frontiers, and the simulation results validate that the proposed method has superior performance over other algorithms on the solution quality of optimum and diversity., Comment: Accepted by IEEE Transactions on Industry Applications

We have identified a potential method for unifying first-order optimizers through the use of variable Second-Moment Exponential Scaling(SMES). We begin with back propagation, addressing classic phenomena such as gradient vanishing and explosion, as well as issues related to dataset sparsity, and introduce the theory of balance in optimization. Through this theory, we suggest that SGD and adaptive optimizers can be unified under a broader inference, employing variable moving exponential scaling to achieve a balanced approach within a generalized formula for first-order optimizers. We conducted tests on some classic datasets and networks to confirm the impact of different balance coefficients on the overall training process.

In this paper we solve a long standing problem about the bilinear $T1$ theorem to characterize the (weighted) compactness of bilinear Calder\'{o}n-Zygmund operators. Let $T$ be a bilinear operator associated with a standard bilinear Calder\'{o}n-Zygmund kernel. We prove that $T$ can be extended to a compact bilinear operator from $L^{p_1}(w_1^{p_1}) \times L^{p_2}(w_2^{p_2})$ to $L^p(w^p)$ for all exponents $\frac1p = \frac{1}{p_1} + \frac{1}{p_2}>0$ with $p_1, p_2 \in (1, \infty]$ and for all weights $(w_1, w_2) \in A_{(p_1, p_2)}$ if and only if the following hypotheses hold: (H1) $T$ is associated with a compact bilinear Calder\'{o}n-Zygmund kernel, (H2) $T$ satisfies the weak compactness property, and (H3) $T(1,1), T^{*1}(1,1), T^{*2}(1,1) \in \mathrm{CMO}(\mathbb{R}^n)$. This is also equivalent to the endpoint compactness: (1) $T$ is compact from $L^1(w_1) \times L^1(w_2)$ to $L^{\frac12, \infty}(w^{\frac12})$ for all $(w_1, w_2) \in A_{(1, 1)}$, or (2) $T$ is compact from $L^{\infty}(w_1^{\infty}) \times L^{\infty}(w_2^{\infty})$ to $\mathrm{CMO}_{\lambda}(w^{\infty})$ for all $(w_1, w_2) \in A_{(\infty, \infty)}$. Besides, any of these properties is equivalent to the fact that $T$ admits a compact bilinear dyadic representation. Our main approaches consist of the following new ingredients: (i) a resulting representation of a compact bilinear Calder\'{o}n-Zygmund operator as an average of some compact bilinear dyadic shifts and paraproducts; (ii) extrapolation of endpoint compactness for bilinear operators; and (iii) compactness criterion in weighted Lorentz spaces. Finally, to illustrate the applicability of our result, we demonstrate the hypotheses (H1)-(H3) through examples including bilinear continuous/dyadic paraproducts, bilinear pseudo-differential operators, and bilinear commutators.

Human-Object Interaction (HOI) detection plays a vital role in scene understanding, which aims to predict the HOI triplet in the form of . Existing methods mainly extract multi-modal features (e.g., appearance, object semantics, human pose) and then fuse them together to directly predict HOI triplets. However, most of these methods focus on seeking for self-triplet aggregation, but ignore the potential cross-triplet dependencies, resulting in ambiguity of action prediction. In this work, we propose to explore Self- and Cross-Triplet Correlations (SCTC) for HOI detection. Specifically, we regard each triplet proposal as a graph where Human, Object represent nodes and Action indicates edge, to aggregate self-triplet correlation. Also, we try to explore cross-triplet dependencies by jointly considering instance-level, semantic-level, and layout-level relations. Besides, we leverage the CLIP model to assist our SCTC obtain interaction-aware feature by knowledge distillation, which provides useful action clues for HOI detection. Extensive experiments on HICO-DET and V-COCO datasets verify the effectiveness of our proposed SCTC.

Through theoretical and experimental validation, unlike all existing adaptive methods like Adam which penalize frequently-changing parameters and are only applicable to sparse gradients, we propose the simplest SGD enhanced method, Loss-Controlled Asymmetric Momentum(LCAM). By averaging the loss, we divide training process into different loss phases and using different momentum. It not only can accelerates slow-changing parameters for sparse gradients, similar to adaptive optimizers, but also can choose to accelerates frequently-changing parameters for non-sparse gradients, thus being adaptable to all types of datasets. We reinterpret the machine learning training process through the concepts of weight coupling and weight traction, and experimentally validate that weights have directional specificity, which are correlated with the specificity of the dataset. Thus interestingly, we observe that in non-sparse gradients, frequently-changing parameters should actually be accelerated, which is completely opposite to traditional adaptive perspectives. Compared to traditional SGD with momentum, this algorithm separates the weights without additional computational costs. It is noteworthy that this method relies on the network's ability to extract complex features. We primarily use Wide Residual Networks for our research, employing the classic datasets Cifar10 and Cifar100 to test the ability for feature separation and conclude phenomena that are much more important than just accuracy rates. Finally, compared to classic SGD tuning methods, while using WRN on these two datasets and with nearly half the training epochs, we achieve equal or better test accuracy.

Given an untrimmed video, repetitive actions counting aims to estimate the number of repetitions of class-agnostic actions. To handle the various length of videos and repetitive actions, also optimization challenges in end-to-end video model training, down-sampling is commonly utilized in recent state-of-the-art methods, leading to ignorance of several repetitive samples. In this paper, we attempt to understand repetitive actions from a full temporal resolution view, by combining offline feature extraction and temporal convolution networks. The former step enables us to train repetition counting network without down-sampling while preserving all repetition regardless of the video length and action frequency, and the later network models all frames in a flexible and dynamically expanding temporal receptive field to retrieve all repetitions with a global aspect. We experimentally demonstrate that our method achieves better or comparable performance in three public datasets, i.e., TransRAC, UCFRep and QUVA. We expect this work will encourage our community to think about the importance of full temporal resolution., Comment: 12 pages and 4 figures and 17 conferences

With the rapid advancements in autonomous driving and robot navigation, there is a growing demand for lifelong learning models capable of estimating metric (absolute) depth. Lifelong learning approaches potentially offer significant cost savings in terms of model training, data storage, and collection. However, the quality of RGB images and depth maps is sensor-dependent, and depth maps in the real world exhibit domain-specific characteristics, leading to variations in depth ranges. These challenges limit existing methods to lifelong learning scenarios with small domain gaps and relative depth map estimation. To facilitate lifelong metric depth learning, we identify three crucial technical challenges that require attention: i) developing a model capable of addressing the depth scale variation through scale-aware depth learning, ii) devising an effective learning strategy to handle significant domain gaps, and iii) creating an automated solution for domain-aware depth inference in practical applications. Based on the aforementioned considerations, in this paper, we present i) a lightweight multi-head framework that effectively tackles the depth scale imbalance, ii) an uncertainty-aware lifelong learning solution that adeptly handles significant domain gaps, and iii) an online domain-specific predictor selection method for real-time inference. Through extensive numerical studies, we show that the proposed method can achieve good efficiency, stability, and plasticity, leading the benchmarks by 8% to 15%.

The recent trend in 2D multiple object tracking (MOT) is jointly solving detection and tracking, where object detection and appearance feature (or motion) are learned simultaneously. Despite competitive performance, in crowded scenes, joint detection and tracking usually fail to find accurate object associations due to missed or false detections. In this paper, we jointly model counting, detection and re-identification in an end-to-end framework, named CountingMOT, tailored for crowded scenes. By imposing mutual object-count constraints between detection and counting, the CountingMOT tries to find a balance between object detection and crowd density map estimation, which can help it to recover missed detections or reject false detections. Our approach is an attempt to bridge the gap of object detection, counting, and re-Identification. This is in contrast to prior MOT methods that either ignore the crowd density and thus are prone to failure in crowded scenes,or depend on local correlations to build a graphical relationship for matching targets. The proposed MOT tracker can perform online and real-time tracking, and achieves the state-of-the-art results on public benchmarks MOT16 (MOTA of 79.7), MOT17 (MOTA of 81.3%) and MOT20 (MOTA of 78.9%).

In recent years, sharp or quantitative weighted inequalities have attracted considerable attention on account of $A_2$ conjecture solved by Hyt\"{o}nen. Advances have greatly improved conceptual understanding of classical objects such as Calder\'{o}n-Zygmund operators. However, plenty of operators do not fit into the class of Calder\'{o}n-Zygmund operators and fail to be bounded on all $L^p(w)$ spaces for $p \in (1, \infty)$ and $w \in A_p$. In this paper we develop Rubio de Francia extrapolation with quantitative bounds to investigate quantitative weighted inequalities for operators beyond the (multilinear) Calder\'{o}n-Zygmund theory. We mainly establish a quantitative multilinear limited range extrapolation in terms of exponents $p_i \in (\mathfrak{p}_i^-, \mathfrak{p}_i^+)$ and weights $w_i^{p_i} \in A_{p_i/\mathfrak{p}_i^-} \cap RH_{(\mathfrak{p}_i^+/p_i)'}$, $i=1, \ldots, m$, which refines a result of Cruz-Uribe and Martell. We also present an extrapolation from multilinear operators to the corresponding commutators. Additionally, our result is quantitative and allows us to extend special quantitative estimates in the Banach space setting to the quasi-Banach space setting. Our proof is based on an off-diagonal extrapolation result with quantitative bounds. Finally, we present various applications to illustrate the utility of extrapolation by concentrating on quantitative weighted estimates for some typical multilinear operators such as bilinear Bochner-Riesz means, bilinear rough singular integrals, and multilinear Fourier multipliers. In the linear case, based on the Littlewood-Paley theory, we include weighted jump and variational inequalities for rough singular integrals.

Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e.g., LiDARs. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Recent successes on the task have been demonstrated and dominated by deep learning based solutions. In this article, for the first time, we provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances. We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods. Besides, we present a quantitative comparison of model performance on three widely used benchmarks, including indoor and outdoor datasets. Finally, we discuss the challenges of prior works and provide readers with some insights for future research directions.