Search

Robotic ultrasound (US)-guided surgery is a common clinical workflow for the diagnosis and treatment of many diseases. Tissue deformation is an important factor that affects surgical accuracy. Intraoperative real-time deformable registration is therefore critical to ensure surgical precision. In this study, a real-time US image nonrigid registration method was proposed. First, the analytical derivatives and corresponding cascaded convolution operations were derived, facilitating the fast calculation of the deformation field and gradients. Second, an optimizer fusing the trust-region and limited-memory Broyden-Fletcher–Goldfarb-Shanno (L-BFGS) method was proposed, which accelerates the optimization by adopting vectorized operations instead of large-scale matrix multiplication. The proposed method achieved real-time nonrigid registration with an average registration time of 30 ms for 2-D images with a resolution of $256^{2}$ and 500 ms for 3-D images with a resolution of $112^{3}$ , respectively, on a desktop CPU computing platform. It satisfied clinical real-time requirements without prior dataset construction and training process. The proposed method was validated on public datasets, clinical cases, and robotic phantom intervention experiments. Results demonstrated that our method realized great performance in both registration speed and accuracy. This method is expected to improve the US guidance accuracy of robots, promoting the precision and automation of related surgeries.

Image-guided depth completion (IGDC) is a multimodal computer vision task for acquiring high-precision dense depth maps. It reasonably predicts values around accurate sparse depth measurements by relying on the details of simultaneous dense RGB images. To achieve this goal, spatial propagation networks (SPNs) elaborate context-aware meta cells that connect pixels to their neighbors and fuse sparse and dense modalities by linear propagation. However, static affinity matrices and fixed neighborhood connections limit the representation of the networks. In this paper, our proposed dynamic SPN (DySPN) uses a nonlinear propagation model (NLPM), which processes the propagation more finely by adjusting the affinity weights, diffusion paths, and the number of neighbors. Specifically, we first generate adaptive weighting (AW) matrices by decoupling the neighborhood into parts with respect to different distances. Independent attention maps are recursively applied to refine the weight value. Furthermore, a dynamic path (DP) strategy is adopted to unfreeze the links of the neighborhood for learning variable connections. The solution space of the paths is also constrained by a propagation decay loss to keep the results stable. Finally, we introduce a diffusion suppression (DS) operation, which preserves the edge of dense depth maps by manipulating the AW and DP strategies to decelerate and terminate the propagation. In our experiment, the proposed method requires fewer iterations and neighbors than other SPNs while yielding better results. DySPN outperforms state-of-the-art (SoTA) methods on the KITTI DC, NYU Depth v2, and VOID datasets. Our code is available at: https://github.com/Kyakaka/DySPN.

Industrial edge computing devices typically have limited computational performance. Current high-precision grasping pose detection models are often over-parameterized, resulting in computational inefficiency and making it difficult to deploy on industrial robots. Existing efficient network design methods rely on constructing, training, and validating a large number of new networks to obtain the final target network, which also requires significant computing resources. To address this issue, the network convergence indicator (NCI) is proposed to realize the prediction of network accuracy before training. NCI can evaluate convergence based on the evasion of flat regions and saddle points theory. It is used to control the width and depth of the network model and optimize its inner layers. Based on these principles, NCIGrasp is proposed, which is a high-precision lightweight grasping network achieving outstanding performance on the Cornell and Jacquard datasets with only 10% of the parameters of previous advanced models. NCIGrasp has a small size of 0.69 MB and fast inference speed on the server (~8 ms), making it suitable for edge computing. Experimental results show that NCIGrasp has real-time grasping pose detection ability (~21 ms) for unknown objects on an ordinary laptop with limited computation capability.

Flexible smart sensing skin is a key enabling technology for the future “Fly-by-Feel” control of morphing aircraft. It represents the next-generation skin of aircraft that can exhibit a more powerful sensing function than a conventional one and could be mounted on arbitrary curvilinear surfaces, especially for advanced autonomic, morphing aircraft. Recent significant technical advances in flexible electronics have overcome many historic drawbacks of conventional smart skin, e.g., only a limited number of discrete block sensors can be integrated due to the inevitable structural damage and heavy guidelines. Herein, we review the key developments in flexible sensors technology and highlight both the state-of-the-art devices and the potential applications for the measurement of aircraft. We begin with the importance of flexible smart skin for morphing aircraft and then expand to the latest progress in various types of flexible sensors. Then we highlight flexible sensors as smart skin to measure aerodynamic parameters and monitor the structural health, and further to achieve the Fly-by-Feel control. Finally, the challenges and opportunities on flexible smart sensing skin are discussed, from the functional design to practical applications.

Image anomaly detection is extremely challenging in industrial manufacturing processes due to unforeseen and diversified anomalies. Recently, unsupervised anomaly detection methods based on knowledge distillation have been developed and have shown remarkable potential. While most existing methods are devoted to knowledge generalization, they are inadequate for the fine-grained detection task. To address this issue, we propose a novel adaptive context-aware distillation (ACAD) paradigm that gives due consideration to distillation component dependencies and knowledge transfer optimization. Technically, a novel adaptive distillation module (ADM) is proposed for optimal context-aware knowledge transfer, which consists of contrastive decoupling distillation (CDD) and masked perceiving distillation (MPD). The proposed CDD helps to constrain the distribution of different semantic patterns and strengthen the discriminative capability. Vanilla methods treat every pixel as an equal contribution and fail to focus on critical information. To this end, the MPD is proposed to weigh different contextual knowledge adaptively. Extensive experiments with mainstream anomaly detection datasets show that ACAD outperforms the state-of-the-art competitors in accuracy and efficiency. In addition, the experimental results with a real-world inkjet printing organic electroluminescence display (OLED) panel dataset further demonstrate the effectiveness of our method.

The wheel-legged robot inherits the merit of both the wheeled robot and the legged robot, which can not only adapt to the complex terrain but also maintain the driving efficiency on the flat road. This article presents an optimization-based approach that leverage ideas from computational geometric mechanics to generate safe and high-quality wheel-leg hybrid motions among obstacles. The formulation of the proposed motion optimization problem incorporates the Lagrange–d’Alembert principle as the robot’s dynamic constraints and an efficient closed-form formulation of collision-free constraints. By discretizing the variational mechanics principle directly, rather than its corresponding forced Euler–Lagrange equation, the continuous trajectory optimization problem is transformed into a nonlinear programming (NLP) problem. Numerical simulations and several real-world experiments are conducted on a wheel-legged robot to demonstrate the effectiveness of the proposed trajectory generation approach.