Search

Medical image segmentation provides a crucial foundation for cancer diagnosis. Transformers are adept at understanding global context and complex dependencies. CNNs, meanwhile, are efficient for local feature extraction and hierarchical learning but struggle with long-range dependencies. In this paper, we combine the benefits of both methodologies. We propose DMFC-UFormer, an advanced fusion of Depthwise Multi-Scale Factorized Convolution-based transformers (DMFC-Transformer) with UNet. The DMFC-Transformer integrates two sub-transformer blocks. The first employs Multi-Scale Factorized Feature Extraction (MFFE) to enhance diversity in feature representation across different levels. The second uses Depthwise Multi-Scale Factorized Convolution (DMFC) to capture a broader range of patterns and variations. An Enhanced Contextual Feature Integration (ECFI) block is incorporated after each transition. This emulates contextual features and facilitates segmentation at each phase. The Spatial-Channel Partitioned Feature Attention (SCPFA) bottleneck module replaces stacking modules. This expands the receptive field and augments feature diversity. An Attention-based Feature Stabilization (AFS) module is integrated into skip connections. It ensures global interaction and highlights important semantic features from the encoder to the decoder. To assess the versatility of the network, we evaluated DMFC-UFormer across a range of medical image segmentation datasets. These include diverse imaging modalities such as wireless capsule endoscopy (WCE), colonoscopy, and dermoscopic images. DMFC-UFormer achieves Dice coefficients (DCs) of 92.14%, 89.99%, 90.47%, and 82.39% on the MICCAI 2017 (Red Lesion), PH2, CVC-ClinicalDB, and ISIC 2017 datasets, respectively. It outperforms the second-ranked methods by margins of 0.83%, 1.22%, 0.57%, and 0.13% in DC on the respective datasets. • We propose a DMFC-UFormer architecture for Medical Image segmentation. • DMFC-UFormer captures both local and global context for precise segmentation. • SCPFA module expands the receptive field, boosting feature diversity in segmentation. • ECFI integrates after transitions, effectively capturing contextual features. • AFS enhances global interaction and highlights semantic features in skip connections. [ABSTRACT FROM AUTHOR]

This paper introduces a novel approach to measure the correspondence between objects, and exploit it for object and image classification tasks, using the proposed Partial Dominant Orientation Descriptor (PDOD). In particular, the object is represented by a set of stable and informative key locations sampled using Difference of Gaussian. The proposed PDOD at each extracted key location takes into account the position and partially computes the dominant orientation of other key locations relative to it, thus, offering a global distinctive and discriminative characterization. This allows us to learn features that are largely invariant to common image transformations, including changes in object colors and textures. The correspondence in-between two objects is performed by finding for each key location in one object the key location in the other object that has the most similar descriptor. Object classification proceeds by assigning the most relevant category that has maximally similar stored prototype objects to the query object using k -Nearest Neighbors algorithm with Adaptive Object Distance. For efficiency, we further investigate PDOD for image classification by developing powerful image representations based on the popular Bag-of-Words model. The extensive experiments demonstrate that the proposed approach greatly improves the matching and classification results, while achieving the state-of-the-art performances on several challenging benchmark datasets. The obtained results suggest also broader applicability to other classification modalities. [ABSTRACT FROM AUTHOR]

• We propose a multi-dimensional deep learning-based method for gesture recognition. • Learning spatiotemporal features from RGB-D videos and their motion representation. • We investigate different fusion strategies to boost the recognition performance. • We show efficiency to other video classification tasks (i.e. activity recognition). • The proposed method achieves the state-of-the-art results on several datasets. Human gesture recognition has become a pillar of today's intelligent Human-Computer Interfaces as it typically provides more comfortable and ubiquitous interaction. Such expert system has a promising prospect in various applications, including smart houses, gaming, healthcare, and robotics. However, recognizing human gestures in videos is one of the most challenging topics in computer vision, because of some irrelevant environmental factors like complex background, occlusion, lighting conditions, and so on. With the recent development of deep learning, many researchers have addressed this problem by building single deep networks to learn spatiotemporal features from video data. However, the performance is still unsatisfactory due to the limitation that the single deep networks are incapable of handling these challenges simultaneously. Hence, the extracted features cannot efficiently capture both relevant shape information and detailed spatiotemporal variation of the gestures. One solution to overcome the aforementioned drawbacks is to fuse multiple features from different models learned on multiple vision cues. Aiming at this objective, we present in this paper an effective multi-dimensional feature learning approach, termed as MultiD-CNN, for human gesture recognition in RGB-D videos. The key to our design is to learn high-level gesture representations by taking advantages from Convolutional Residual Networks (ResNets) for training extremely deep models and Convolutional Long Short-Term Memory Networks (ConvLSTM) for dealing with time-series connections. More specifically, we first construct an architecture to simultaneously learn the spatiotemporal features from RGB and depth sequences through 3D ResNets which are then linked to a ConvLSTM to capture the temporal dependencies between them, and we show that they better combine appearance and motion information effectively. Second, to alleviate distractions from background and other variations, we propose a method that encodes the temporal information into a motion representation, while a two-stream architecture based on 2D-ResNets is then employed to extract deep features from this representation. Third, we investigate different fusion strategies at different levels for blending the classification results, and we show that integrating multiple ways of encoding the spatial and temporal information leads to a robust and stable spatiotemporal feature learning with better generalization capability. Finally, we perform different experiments to evaluate the performance of the investigated architectures on four kinds of challenging datasets, demonstrating that our approach is particularly impressive where it outperforms prior arts in both accuracy and efficiency. The obtained results affirm also the importance of embedding the proposed approach in other intelligent systems application areas. [ABSTRACT FROM AUTHOR]