Search

Few-shot Semantic Segmentation (FSS) was proposed to segment unseen classes in a query image, referring to only a few annotated examples named support images. One of the characteristics of FSS is spatial inconsistency between query and support targets, e.g., texture or appearance. This greatly challenges the generalization ability of methods for FSS, which requires to effectively exploit the dependency of the query image and the support examples. Most existing methods abstracted support features into prototype vectors and implemented the interaction with query features using cosine similarity or feature concatenation. However, this simple interaction may not capture spatial details in query features. To address this limitation, some methods utilized pixel-level support information by computing pixel-level correlations between paired query and support features implemented with the attention mechanism of Transformer. Nevertheless, these approaches suffer from heavy computation due to dot-product attention between all pixels of support and query features. In this paper, we propose a novel framework, termed ProtoFormer, built upon the Transformer architecture, to fully capture spatial details in query features. ProtoFormer treats the abstracted prototype of the target class in support features as the Query and the query features as Key and Value embeddings, which are input to the Transformer decoder. This approach enables better capture of spatial details and focuses on the semantic features of the target class in the query image. The output of the Transformer-based module can be interpreted as semantic-aware dynamic kernels that filter the segmentation mask from the enriched query features. Extensive experiments conducted on PASCAL- 5 i and COCO- 20 i datasets demonstrate that ProtoFormer significantly outperforms the state-of-the-art methods in FSS. [ABSTRACT FROM AUTHOR]

Objective. The accurate automatic segmentation of tumors from computed tomography (CT) volumes facilitates early diagnosis and treatment of patients. A significant challenge in tumor segmentation is the integration of the spatial correlations among multiple parts of a CT volume and the context relationship across multiple channels. Approach. We proposed a mutually enhanced multi-view information model (MEMI) to propagate and fuse the spatial correlations and the context relationship and then apply it to lung tumor CT segmentation. First, a feature map was obtained from segmentation backbone encoder, which contained many image region nodes. An attention mechanism from the region node perspective was presented to determine the impact of all the other nodes on a specific node and enhance the node attribute embedding. A gated convolution-based strategy was also designed to integrate the enhanced attributes and the original node features. Second, transformer across multiple channels was constructed to integrate the channel context relationship. Finally, since the encoded node attributes from the gated convolution view and those from the channel transformer view were complementary, an interaction attention mechanism was proposed to propagate the mutual information among the multiple views. Main results. The segmentation performance was evaluated on both public lung tumor dataset and private dataset collected from a hospital. The experimental results demonstrated that MEMI was superior to other compared segmentation methods. Ablation studies showed the contributions of node correlation learning, channel context relationship learning, and mutual information interaction across multiple views to the improved segmentation performance. Utilizing MEMI on multiple segmentation backbones also demonstrated MEMI's generalization ability. Significance. Our model improved the lung tumor segmentation performance by learning the correlations among multiple region nodes, integrating the channel context relationship, and mutual information enhancement from multiple views. [ABSTRACT FROM AUTHOR]

Machine vision plays a great role in localizing strawberries in a complex orchard or greenhouse for picking robots. Due to the variety of each strawberry (shape, size, and color) and occlusions of strawberries by leaves and stems, precisely locating each strawberry brings a great challenge to the vision system of picking robots. Several methods have been developed for localizing strawberries, based on the well-known Mask R-CNN network, which, however, are not efficient running on the picking robots. In this paper, we propose a simple and highly efficient framework for strawberry instance segmentation running on low-power devices for picking robots, termed StrawSeg. Instead of using the common paradigm of "detection-then-segment", we directly segment each strawberry in a single-shot manner without relying on object detection. In our model, we design a novel feature aggregation network to merge features with different scales, which employs a pixel shuffle operation to increase the resolution and reduce the channels of features. Experiments on the open-source dataset StrawDI_Db1 demonstrate that our model can achieve a good trade-off between accuracy and inference speed on a low-power device. [ABSTRACT FROM AUTHOR]

Segmentation of retinal vessels in fundus images plays a very important role in diagnosing relevant diseases. In this paper, we have constructed automated segmentation models for the retinal vessel segmentation task based on convolutional neural networks. Since some typical deep convolutional neural networks need to be fed by high-resolution patches, small retinal patches should be interpolated to the specific resolution. The interpolated patches sometimes would introduce additional noises. Thus, we modify some typical deep architectures by inserting a set of convolutional layers. In this way, our models have the ability to adapt to different resolutions. Overall, five models are analyzed and compared in our studies including LeNet, M-AlexNet (modified AlexNet), M-ZF-Net (Modified ZF-Net), M-VGG (Modified VGG) and Deformable-ConvNet. Deformable-ConvNet captures the vascular structure and is used to do the retinal vessel segmentation task for the first time. We train the models from scratch and compare their ability to discriminate vessels/non-vessel pixels on two retinal fundus image datasets, DRIVE and STARE. Results are analyzed and compared in our studies. We obtain the highest accuracy of 0.9628/0.9690, lowest loss of 0.1045/0.0968, and highest AUC of 0.9764/0.9844 on DRIVE/STARE respectively. We also compare the CNN models with other segmentation methods. The results demonstrate the high effectiveness of the CNN-based approaches. [ABSTRACT FROM AUTHOR]