Search

Self-distillation method guides the model learning via transferring knowledge of the model itself, which has shown the advantages in object segmentation. However, it has been proved that uncertain pixels with predicted probability close to 0.5 will restrict the model performance. The existing self-distillation methods cannot guide the model to enhance its learning ability for uncertain pixels, so the improvement is limited. To boost the student model's learning ability for uncertain pixels, a novel self-distillation method is proposed. Firstly, the predicted probability in the current training sample and the ground truth label are fused to construct the teacher knowledge, as the current predicted information can express the performance of student models and represent the uncertainty of pixels more accurately. Secondly, a quadratic mapping function between the predicted probabilities of the teacher and student model is proposed. Theoretical analysis shows that the proposed method using the mapping function can guide the model to enhance the learning ability for uncertain pixels. Finally, the essential difference of utilizing the predicted probability of the student model in self-distillation is discussed in detail. Extensive experiments were conducted on models with convolutional neural networks and Transformer architectures as the backbone networks. The results on four public datasets demonstrate that the proposed method can effectively improve the student model performance. [ABSTRACT FROM AUTHOR]

Digitization in pathology and cytology labs is now widespread, a significant shift from a decade ago when few doctors used image processing tools. Despite unchanged scanning times due to excitation in fluorescent imaging, advancements in computing power and software have enabled more complex algorithms, yielding better-quality results. This study evaluates three nucleus segmentation algorithms for ploidy analysis using propidium iodide-stained digital WSI slides. Our goal was to improve segmentation accuracy to more closely match DNA histograms obtained via flow cytometry, with the ultimate aim of enhancing the calibration method we proposed in a previous study, which seeks to align image cytometry results with those from flow cytometry. We assessed these algorithms based on raw segmentation performance and DNA histogram similarity, using confusion-matrix-based metrics. Results indicate that modern algorithms perform better, with F1 scores exceeding 0.845, compared to our earlier solution's 0.807, and produce DNA histograms that more closely resemble those from the reference FCM method. [ABSTRACT FROM AUTHOR]

In digital agriculture, accurate crop detection is fundamental to developing automated systems for efficient plantation management. For oil palm, the main challenge lies in developing robust models that perform well in different environmental conditions. This study addresses the feasibility of using GAN augmentation methods to improve palm detection models. For this purpose, drone images of young palms (< 5 year-old) from eight different estates were collected, annotated, and used to build a baseline detection model based on DETR. StyleGAN2 was trained on the extracted palms and then used to generate a series of synthetic palms, which were then inserted into tiles representing different environments. CycleGAN networks were trained for bidirectional translation between synthetic and real tiles, subsequently utilized to augment the authenticity of synthetic tiles. Both synthetic and real tiles were used to train the GAN-based detection model. The baseline model achieved precision and recall values of 95.8% and 97.2%. The GAN-based model achieved comparable result, with precision and recall values of 98.5% and 98.6%. In the challenge dataset 1 consisting older palms (> 5 year-old), both models also achieved similar accuracies, with baseline model achieving precision and recall of 93.1% and 99.4%, and GAN-based model achieving 95.7% and 99.4%. As for the challenge dataset 2 consisting of storm affected palms, the baseline model achieved precision of 100% but recall was only 13%. The GAN-based model achieved a significantly better result, with a precision and recall values of 98.7% and 95.3%. This result demonstrates that images generated by GANs have the potential to enhance the accuracies of palm detection models. [ABSTRACT FROM AUTHOR]

Applying deep learning algorithms in the construction industry holds tremendous potential for enhancing site management, safety, and efficiency. The development of such algorithms necessitates a comprehensive and diverse image dataset. This study introduces the Aerial Image Dataset for Construction (AIDCON), a novel aerial image collection containing 9563 construction machines across nine categories annotated at the pixel level, carrying critical value for researchers and professionals seeking to develop and refine object detection and segmentation algorithms across various construction projects. The study highlights the benefits of utilizing UAV-captured images by evaluating the performance of five cutting-edge deep learning algorithms—Mask R-CNN, Cascade Mask R-CNN, Mask Scoring R-CNN, Hybrid Task Cascade, and Pointrend—on the AIDCON dataset. It underscores the significance of clustering strategies for generating reliable and robust outcomes. The AIDCON dataset's unique aerial perspective aids in reducing occlusions and provides comprehensive site overviews, facilitating better object positioning and segmentation. The findings presented in this paper have far-reaching implications for the construction industry, as they enhance construction site efficiency while setting the stage for future advancements in construction site monitoring and management utilizing remote sensing technologies. [ABSTRACT FROM AUTHOR]

In-process dimension measurement is critical to achieving higher productivity and realizing smart manufacturing goals during machining operations. Vision-based systems have significant potential to serve for in-process dimensions measurements, reduce human interventions, and achieve manufacturing-inspection integration. This paper presents early research on developing a vision-based system for in-process dimension measurement of machined cylindrical components utilizing image-processing techniques. The challenges with in-process dimension measurement are addressed by combining a deep learning-based object detection model, You Only Look Once version 2 (YOLOv2), and image processing algorithms for object localization, segmentation, and spatial pixel estimation. An automated image pixel calibration approach is incorporated to improve algorithm robustness. The image acquisition hardware and the real-time image processing framework are integrated to demonstrate the working of the proposed system by considering a case study of in-process stepped shaft diameter measurement. The system implementation on a manual lathe demonstrated robust utilities, eliminating the need for manual intermittent measurements, digitized in-process component dimensions, and improved machining productivity. [ABSTRACT FROM AUTHOR]

In this paper, we address an essential problem related to object detection and image processing: detecting objects potentially nested in other ones. This problem exists particularly in the beekeeping sector: detecting varroa parasites on bees. Indeed, beekeepers must ensure the level of infestation of their apiaries by the varroa parasite which settles on the backs of bees. As far as we know, there is no yet a published approach to deal with nested object detection using only one neural network trained on two different datasets. We propose an approach that fills this gap. Therefore, we improve the accuracy and the efficiency of bee and varroa detection task. Our work is based on deep learning, more precisely Mask R-CNN neural network. Instead of segmenting detected objects (bees), we segment internal objects (varroas). We add a branch to Faster R-CNN to segment internal objects. We extract relevant features for internal object segmentation and suggest efficient method for training the neural network on two different datasets. Our experiments are based on a set of images of bee frames, containing annotated bees and varroa mites. Due to differences in occurrence rates, two different sets were created. After carrying out experiments, we ended up with a single neural network capable of detecting two nested objects without decreasing accuracy compared to two separate neural networks. Our approach, compared to traditional separate neural networks, improves varroa detection accuracy by 1.9%, reduces infestation level prediction error by 0.22%, and reduces execution time by 28% and model memory by 23%. In our approach, we extract Res4 (a layer of the ResNet neural network) features for varroa segmentation, which improves detection accuracy by 11% compared to standard FPN extraction. Thus, we suggest a new approach that detects nested objects more accurately than two separate network approaches. [ABSTRACT FROM AUTHOR]

Cancer is one of the leading causes of death in the world, with radiotherapy as one of the treatment options. Radiotherapy planning starts with delineating the affected area from healthy organs, called organs at risk (OAR). A new approach to automatic OAR segmentation in the chest cavity in Computed Tomography (CT) images is presented. The proposed approach is based on the modified U‐Net architecture with the ResNet‐34 encoder, which is the baseline adopted in this work. The new two‐branch CS‐SA U‐Net architecture is proposed, which consists of two parallel U‐Net models in which self‐attention blocks with cosine similarity as query‐key similarity function (CS‐SA) blocks are inserted between the encoder and decoder, which enabled the use of consistency regularisation. The proposed solution demonstrates state‐of‐the‐art performance for the problem of OAR segmentation in CT images on the publicly available SegTHOR benchmark dataset in terms of a Dice coefficient (oesophagus—0.8714, heart—0.9516, trachea—0.9286, aorta—0.9510) and Hausdorff distance (oesophagus—0.2541, heart—0.1514, trachea—0.1722, aorta—0.1114) and significantly outperforms the baseline. The current approach is demonstrated to be viable for improving the quality of OAR segmentation for radiotherapy planning. [ABSTRACT FROM AUTHOR]

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This research aims at presenting a video-based system to detect fire in real time taking advantage of already existing surveillance systems for fire detection either inside or outside the building with different illumination and short or long distance surveillance scenes. Detection of fires with surveillance cameras is characterized by early detection and rapid performance. Information about the progress of the fire can be obtained through live video. Also vision-based system is capable of providing forensic evidence. The basic idea of the research is fire detection based on video as proposed Fourier descriptors were used to describe reddish moving objects. The proposed system idea is to detect reddish moving bodies in every frame and correlate the detections with the same reddish bodiest over time. Multi-threshold segmentation is used to divide the image. This method can be integrated with pretreatment and post-processing. The threshold is one of the most common ways to divide the image. The next stage after the segmentation is to obtain the reddish body features. The feature is created by obtaining the contour of the reddish body and estimating its normalized Fourier descriptors. If the reddish body contour's Fourier descriptors vary from frame to frame, one can predict fire. [ABSTRACT FROM AUTHOR]

Machine vision, an interdisciplinary field that aims to replicate human visual perception in computers, has experienced rapid progress and significant contributions. This paper traces the origins of machine vision, from early image processing algorithms to its convergence with computer science, mathematics, and robotics, resulting in a distinct branch of artificial intelligence. The integration of machine learning techniques, particularly deep learning, has driven its growth and adoption in everyday devices. This study focuses on the objectives of computer vision systems: replicating human visual capabilities including recognition, comprehension, and interpretation. Notably, image classification, object detection, and image segmentation are crucial tasks requiring robust mathematical foundations. Despite the advancements, challenges persist, such as clarifying terminology related to artificial intelligence, machine learning, and deep learning. Precise definitions and interpretations are vital for establishing a solid research foundation. The evolution of machine vision reflects an ambitious journey to emulate human visual perception. Interdisciplinary collaboration and the integration of deep learning techniques have propelled remarkable advancements in emulating human behavior and perception. Through this research, the field of machine vision continues to shape the future of computer systems and artificial intelligence applications. [ABSTRACT FROM AUTHOR]

Counting small pixel-sized vehicles and crowds in unmanned aerial vehicles (UAV) images is crucial across diverse fields, including geographic information collection, traffic monitoring, item delivery, communication network relay stations, as well as target segmentation, detection, and tracking. This task poses significant challenges due to factors such as varying view angles, non-fixed drone cameras, small object sizes, changing illumination, object occlusion, and image jitter. In this paper, we introduce a novel multi-data-augmentation and multi-deep-learning framework designed for counting small vehicles and crowds in UAV images. The framework harnesses the strengths of specific deep-learning detection models, coupled with the convolutional block attention module and data augmentation techniques. Additionally, we present a new method for detecting cars, motorcycles, and persons with small pixel sizes. Our proposed method undergoes evaluation on the test dataset v2 of the 2022 AI Cup competition, where we secured the first place on the private leaderboard by achieving the highest harmonic mean. Subsequent experimental results demonstrate that our framework outperforms the existing YOLOv7-E6E model. We also conducted comparative experiments using the publicly available VisDrone datasets, and the results show that our model outperforms the other models with the highest AP50 score of 52%. [ABSTRACT FROM AUTHOR]

Visual tracking is a classical research problem and recently tracking with mask prediction has been a popular task in tracking research. Many trackers add a pixel-wise segmentation subnetwork behind the original bounding box tracker to get the target's mask. These two-stage methods need to crop the target region after finding its location and extract deep features for segmentation redundantly. This paper proposes an anchor-free Siamese Refine Polar Mask (SiamRPM) prediction network for visual tracking, which can obtain the target's mask directly. Similar to bounding box regression, we use polar mask regression to get the target's convex hull mask. To further adjust the contour points, we propose to employ a cascaded refinement module. The mask contours are iteratively shifted using the offset outputs of the refinement module. Comprehensive experiments on visual tracking benchmark datasets illustrate that our SiamRPM can achieve competitive results with a real-time running speed. Our method provides an effective contour-based pipeline for the tracking and segmentation task. [ABSTRACT FROM AUTHOR]

Object segmentation aims to perfectly identify objects embedded in the surrounding environment and has a wide range of applications. Most previous methods of object segmentation only use RGB images and ignore geometric information from disparity images. Making full use of heterogeneous data from different devices has proved to be a very effective strategy for improving segmentation performance. The key challenge of the multimodal fusion-based object segmentation task lies in the learning, transformation, and fusion of multimodal information. In this article, we focus on the transformation of disparity images and the fusion of multimodal features. We develop a multimodal fusion object segmentation framework, termed the Hybrid Fusion Segmentation Network (HFSNet). Specifically, HFSNet contains three key components, i.e., disparity convolutional sparse coding (DCSC), asymmetric dense projection feature aggregation (ADPFA), and multimodal feature fusion (MFF). The DCSC is designed based on convolutional sparse coding. It not only has better interpretability but also preserves the key geometric information of the object. ADPFA is designed to enhance texture and geometric information to fully exploit nonadjacent features. MFF is used to perform multimodal feature fusion. Extensive experiments show that our HFSNet outperforms existing state-of-the-art models on two challenging datasets. [ABSTRACT FROM AUTHOR]

Orchard monitoring is a vital direction of scientific research and practical application for increasing fruit production in ecological conditions. Recently, due to the development of technology and the decrease in equipment cost, the use of unmanned aerial vehicles and artificial intelligence algorithms for image acquisition and processing has achieved tremendous progress in orchards monitoring. This paper highlights the new research trends in orchard monitoring, emphasizing neural networks, unmanned aerial vehicles (UAVs), and various concrete applications. For this purpose, papers on complex topics obtained by combining keywords from the field addressed were selected and analyzed. In particular, the review considered papers on the interval 2017-2022 on the use of neural networks (as an important exponent of artificial intelligence in image processing and understanding) and UAVs in orchard monitoring and production evaluation applications. Due to their complexity, the characteristics of UAV trajectories and flights in the orchard area were highlighted. The structure and implementations of the latest neural network systems used in such applications, the databases, the software, and the obtained performances are systematically analyzed. To recommend some suggestions for researchers and end users, the use of the new concepts and their implementations were surveyed in concrete applications, such as a) identification and segmentation of orchards, trees, and crowns; b) detection of tree diseases, harmful insects, and pests; c) evaluation of fruit production, and d) evaluation of development conditions. To show the necessity of this review, in the end, a comparison is made with review articles with a related theme. [ABSTRACT FROM AUTHOR]

Unsupervised defect detection methods have garnered substantial attention in industrial defect detection owing to their capacity to circumvent complex fault sample collection. However, these models grapple with establishing a robust boundary between normal and abnormal conditions in intricate scenarios, leading to a heightened frequency of false-positive predictions. Spurious alerts exacerbate the work of reconfirmation and impede the widespread adoption of unsupervised anomaly detection models in industrial applications. To this end, we delve into the sole available data source in unsupervised defect detection models, the unsupervised training dataset, to introduce a solution called the False Alarm Identification (FAI) method aimed at learning the distribution of potential false alarms using anomaly-free images. It exploits a multi-layer perceptron to capture the semantic information of potential false alarms from a detector trained on anomaly-free training images at the object level. During the testing phase, the FAI model operates as a post-processing module applied after the baseline detection algorithm. The FAI algorithm determines whether each positive patch predicted by the normalizing flow algorithm is a false alarm by its semantic features. When a positive prediction is identified as a false alarm, the corresponding pixel-wise predictions are set to negative. The effectiveness of the FAI method is demonstrated by two state-of-the-art normalizing flow algorithms on extensive industrial applications. [ABSTRACT FROM AUTHOR]

Interactive image segmentation (IIS) is an important technique for obtaining pixel-level annotations. In many cases, target objects share similar semantics. However, IIS methods neglect this connection and in particular the cues provided by representations of previously segmented objects, previous user interaction, and previous prediction masks, which can all provide suitable priors for the current annotation. In this paper, we formulate a sequential interactive image segmentation (SIIS) task for minimizing user interaction when segmenting sequences of related images, and we provide a practical approach to this task using two pertinent designs. The first is a novel interaction mode. When annotating a new sample, our method can automatically propose an initial click proposal based on previous annotation. This dramatically helps to reduce the interaction burden on the user. The second is an online optimization strategy, with the goal of providing semantic information when annotating specific targets, optimizing the model with dense supervision from previously labeled samples. Experiments demonstrate the effectiveness of regarding SIIS as a particular task, and our methods for addressing it. [ABSTRACT FROM AUTHOR]

Yield forecasting is of immeasurable value in modern viticulture to optimize harvest scheduling and quality management. Traditionally, this task is carried out through manual and destructive sampling of production components and their accurate assessment is expensive, time-consuming, and error-prone, resulting in erroneous projections. The number of inflorescences and flowers per vine is one of the main components and serves as an early predictor. The adoption of new non-invasive technologies can automate this task and drive viticulture yield forecasting to higher levels of accuracy. In this study, different Single Stage Instance Segmentation models from the state-of-the-art You Only Look Once (YOLO) family, such as YOLOv5 and YOLOv8, were benchmarked on a dataset of RGB images for grapevine inflorescence detection and segmentation, with the aim of validating and subsequently implementing the solution for counting the number of inflorescences and flowers. All models obtained promising results, with the YOLOv8s and the YOLOv5s models standing out with an F1-Score of 95.1% and 97.7% for the detection and segmentation tasks, respectively. Moreover, the low inference times obtained demonstrate the models' ability to be deployed in real-time applications, allowing for non-destructive predictions in uncontrolled environments. [ABSTRACT FROM AUTHOR]

Due to the numerous objects with regular structures in indoor environments, identifying and modeling the regular objects in scenes aids indoor robots in sensing unknown environments. Typically, point cloud preprocessing can obtain highly complete object segmentation results in scenes which can be utilized as the objects for geometric analysis and modeling, thus ensuring modeling accuracy and speed. However, due to the lack of a complete object model, it is not possible to recognize and model segmented objects through matching methods. To achieve a greater understanding of scene point clouds, this paper proposes a direct geometric modeling algorithm based on segmentation results, which focuses on extracting regular geometries in the scene, rather than objects with geometric details or combinations of multiple primitives. This paper suggests using simpler geometric models to describe the corresponding point cloud data. By fully utilizing the surface structure information of segmented objects, the paper analyzes the types of faces and their relationships to classify regular geometric objects into two categories: planar and curved. Different types of geometric objects are fitted using random sampling consistency algorithms with type classification results as prior knowledge, and segmented results are modeled through a combination of size information associated with directed bounding boxes. For indoor scenes with occlusion and stacking, utilizing a higher-level semantic expression can effectively simplify the scene, complete scene abstraction and structural modeling, and aid indoor robots' understanding and further operation in unknown environments. [ABSTRACT FROM AUTHOR]

Six-degree-of-freedom (6DoF) object pose estimation is a critical task for robot manipulation, autonomous vehicles, and augmented reality. Category-level 6DoF object pose estimation is trending because it can generalize to same-category unknown objects. However, existing mean shape based methods do not consider that predicting adjustment must model shape differences, which makes these methods still suffer from shape variations among same-category objects, limiting their accuracy. Also, existing methods overlook the importance of object segmentation to 6DoF pose estimation and use an RGB-based object segmentation method with low accuracy. To address these problems, we propose difference-aware shape adjustment and RGB-D feature fusion-based object segmentation for category-level 6DoF object pose estimation. The proposed method encodes shape differences, improving mean shape adjustment and alleviating same-category shape variations. Specifically, a difference-aware shape adjustment network (DASAN) is proposed to model shape differences between the object instance and mean shape by feature subtraction with an attention mechanism. We also propose an RGB-D feature fusion-based object segmentation method that uses a coarse-to-fine framework: a 2D detector and a novel RGB-D feature fusion-based binary classification network for coarse and fine segmentation. Experiments on two well-known datasets demonstrate the proposed method's state-of-the-art (SOTA) pose estimation accuracy. In addition, we construct comparative experiments on the latest dataset (Wild6D) and a self-collected dataset (OBJECTS) and achieve high accuracies, demonstrating the strong generalizability of the proposed method. Also, we apply the proposed method to unknown object grasping, thus demonstrating the practicability of the proposed method. [ABSTRACT FROM AUTHOR]

Side-scan sonar (SSS) is an important underwater imaging method that has high resolutions and is convenient to use. However, due to the restriction of conventional pulse compression technology, the side-scan sonar beam sidelobe in the range direction is relatively high, which affects the definition and contrast of images. When working in a shallow-water environment, image quality is especially influenced by strong bottom reverberation or other targets on the seabed. To solve this problem, a method for image-quality improvement based on deconvolution is proposed herein. In this method, to increase the range resolution and lower the sidelobe, a deconvolution algorithm is employed to improve the conventional pulse compression. In our simulation, the tolerance of the algorithm to different signal-to-noise ratios (SNRs) and the resolution ability of multi-target conditions were analyzed. Furthermore, the proposed method was applied to actual underwater data. The experimental results showed that the quality of underwater acoustic imaging could be effectively improved. The ratios of improvement for the SNR and contrast ratio (CR) were 32 and 12.5%, respectively. The target segmentation results based on this method are also shown. The accuracy of segmentation was effectively improved. [ABSTRACT FROM AUTHOR]

The self-distillation methods can transfer the knowledge within the network itself to enhance the generalization ability of the network. However, due to the lack of spatially refined knowledge representations, current self-distillation methods can hardly be directly applied to object segmentation tasks. In this paper, we propose a novel self-distillation framework via pyramid knowledge representation and transfer for the object segmentation task. Firstly, a lightweight inference network is built to perform pixel-wise prediction rapidly. Secondly, a novel self-distillation method is proposed. To derive refined pixel-wise knowledge representations, the auxiliary self-distillation network via multi-level pyramid representation branches is built and appended to the inference network. A synergy distillation loss, which utilizes the top-down and consistency knowledge transfer paths, is presented to force more discriminative knowledge to be distilled into the inference network. Consequently, the performance of the inference network is improved. Experimental results on five datasets of object segmentation demonstrate that the proposed self-distillation method helps our inference network perform better segmentation effectiveness and efficiency than nine recent object segmentation network. Furthermore, the proposed self-distillation method outperforms typical self-distillation methods. The source code is publicly available at https://github.com/xfflyer/SKDforSegmentation. [ABSTRACT FROM AUTHOR]

To minimize any impediments in real-time Internet of Things (IoT)-enabled robotics applications, this study demonstrated how to build and deploy a revolutionary framework using computer vision and deep learning. In contrast to robotic path planning algorithms based on geolocation. We focus on sensor-captured streams/images and geographical information to enable the Internet of Robotic Things (IoRT) to evolve. The application will collect real-time data from moving robotics at various situations and intervals and use it for research projects. The data collected in videos/image forms are delivered in the robotics application using visual sensor nodes. In this study, anticipating moving robot moves automatically early on can aid in issuing commands to monitor and regulate robots' future activities before they occur. To do so, we propose the framework using efficient computer vision techniques and a deep learning classifier. The computer vision methods are designed for frame quality improvement, object segmentation, and feature estimation. The Long-Term Short Memory (LSTM) classifier detects robot motions automatically from initial sequential features. We mainly designed the proposed model using an LSTM classifier to perform the earlier prediction from the initial sequential features of partial video frames and to overcome the problems of exploding and vanishing gradients. LSTM helps to reduce the prediction duration with higher accuracy. It also enables the central system of a certain robotic application to prevent collisions caused by impediments in the interior or outdoor situation. The simulation results utilizing publicly available research datasets demonstrate the proposed model's efficiency and robustness compared to state-of-the-art approaches. The overall accuracy of the proposed model has improved approximately by 5% and reduced computational complexity by 84% approximately. [ABSTRACT FROM AUTHOR]

The extraction of salient objects from a cluttered background without any prior knowledge is a challenging task in salient object detection and segmentation. A salient object can be detected from the uniqueness, rarity, or unproductivity of the salient regions in an image. However, an object with a similar color appearance may have a marginal visual divergence that is even difficult for the human eyes to recognize. In this paper, we propose a technique which compose and fuse the fast fuzzy c-mean (FFCM) clustering saliency maps to separate the salient object from the background in the image. To be specific, we first generate the maps using FFCM clustering, that contain specific parts of the salient region, which are composed later by using the Porter–Duff composition method. Outliers in the extracted salient regions are removed using a morphological technique in the post-processing step. To extract the final map from the initially constructed blended maps, we use a fused mask, which is the composite form of color prior, location prior, and frequency prior. Experiment results on six public data sets (MSRA, THUR-15000, MSRA-10K, HKU-IS, DUT-OMRON, and SED) clearly show the efficiency of the proposed method for images with a noisy background. [ABSTRACT FROM AUTHOR]

This article proposes a deep learning approach for automatic segmentation of low-contrast watermark defects on Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) surfaces. The task of surface defect detection has been tackled by machine vision or deep learning methods. Traditional machine vision methods require expert domain knowledge for feature extraction, and they are computationally intensive. Deep learning methods based on Convolutional Neural Network (CNN) are extremely fast, but can be affected by illumination variations and perform poorly for low-contrast images. In this study, a U-Net type of encoder-decoder architecture is proposed. The Convolutional operation is substituted with the proposed Cosine-Convolutional operation in the encoder layers of the U-Net model with the objective of minimizing the effects of uneven illumination and highlighting the defect region. To further reduce the computation complexity and improve defect segmentation, an attention-like mechanism is used in the semantic encoder-decoder network. Pixel-wise multiplication is applied in the skip connections between encoder and decoder layers of the U-Net to replace the classical concatenation, thus reducing the parameter count of the model. The proposed methods are computationally fast, with an evaluation time of less than 10-ms for an image of $384\times 256$ pixels on a single low-end GPU. Experimental results show that the proposed Cosine-Convolutional encoder-decoder architectures can effectively and accurately be applied for defect detection and segmentation of watermarks in low-contrast CIS images. [ABSTRACT FROM AUTHOR]

Most of the buildings that exist today were built based on 2D drawings. Building information models that represent design-stage product information have become prevalent in the second decade of the 21st century. Still, it will take many decades before such models become the norm for all existing buildings. In the meantime, the building industry lacks the tools to leverage the benefits of digital information management for construction, operation, and renovation. To this end, this paper reviews the state-of-the-art practice and research for constructing (generating) and maintaining (updating) geometric digital twins. This paper also highlights the key limitations preventing current research from being adopted in practice and derives a new geometry-based object class hierarchy that mainly focuses on the geometric properties of building objects, in contrast to widely used existing object categorisations that are mainly function-oriented. We argue that this new class hierarchy can serve as the main building block for prioritising the automation of the most frequently used object classes for geometric digital twin construction and maintenance. We also draw novel insights into the limitations of current methods and uncover further research directions to tackle these problems. Specifically, we believe that adapting deep learning methods can increase the robustness of object detection and segmentation of various types; involving design intents can achieve a high resolution of model construction and maintenance; using images as a complementary input can help to detect transparent and specular objects; and combining synthetic data for algorithm training can overcome the lack of real labelled datasets. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Data Acquisition & Processing / Shu Ju Cai Ji Yu Chu Li is the property of Editorial Department of Journal of Nanjing University of Aeronautics & Astronautics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Background: Delineating the lesion area is an important task in image-based diagnosis. Pixel-wise classification is a popular approach to segmenting the region of interest. However, at fuzzy boundaries, such methods usually result in glitches, discontinuity or disconnection, inconsistent with the fact that lesions are solid and smooth. Methods: To overcome these problems and to provide an efficient, accurate, robust and concise solution that simplifies the whole segmentation pipeline in AI-assisted applications, we propose the BézierSeg model which outputs Bézier curves encompassing the region of interest. Results: Directly modeling the contour with analytic equations ensures that the segmentation is connected and continuous, and that the boundary is smooth. In addition, it offers sub-pixel accuracy. Without loss of precision, the Bézier contour can be resampled and overlaid with images of any resolution. Moreover, clinicians can conveniently adjust the curve's control points to refine the result. Conclusions: Our experiments show that the proposed method runs in real time and achieves accuracy competitive with pixel-wise segmentation models. [ABSTRACT FROM AUTHOR]

Accurately segmenting an insect from its original ecological image is the core technology restricting the accuracy and efficiency of automatic recognition. However, the performance of existing segmentation methods is unsatisfactory in insect images shot in wild backgrounds on account of challenges: various sizes, similar colors or textures to the surroundings, transparent body parts and vague outlines. These challenges of image segmentation are accentuated when dealing with camouflaged insects. Here, we developed an insect image segmentation method based on deep learning termed the progressive refinement network (PRNet), especially for camouflaged insects. Unlike existing insect segmentation methods, PRNet captures the possible scale and location of insects by extracting the contextual information of the image, and fuses comprehensive features to suppress distractors, thereby clearly segmenting insect outlines. Experimental results based on 1900 camouflaged insect images demonstrated that PRNet could effectively segment the camouflaged insects and achieved superior detection performance, with a mean absolute error of 3.2%, pixel-matching degree of 89.7%, structural similarity of 83.6%, and precision and recall error of 72%, which achieved improvements of 8.1%, 25.9%, 19.5%, and 35.8%, respectively, when compared to the recent salient object detection methods. As a foundational technology for insect detection, PRNet provides new opportunities for understanding insect camouflage, and also has the potential to lead to a step progress in the accuracy of the intelligent identification of general insects, and even being an ultimate insect detector. [ABSTRACT FROM AUTHOR]

In interactive image segmentation methods, users can participate in and influence the segmentation process through their interactions, such as scribbles or bounding boxes. Similarly, the process of deep interactive segmentation utilizes users’ interactions to guide the network to learn the target of interest. This article mainly considers mouse clicking, which is the simplest interaction mode. Then, how to effectively characterize the click interaction (we call this “click encoding”) and fuse the click-related information with the network are the key issues in a deep interactive segmentation framework. However, the current click encoding method concentrates only on the spatial information of the clicks, so the region affected by each click is difficult to control, and the stability of the network is therefore reduced. Therefore, we propose a feature-interactive map that builds a close relationship between interaction information and target semantics. The affected region of the feature interactive map is determined by semantic information. Furthermore, we introduce an interactive nonlocal block by embedding a feature-interactive map into a nonlocal block, so that the long-range dependencies of the interaction information can be captured. Finally, based on the early fusion strategy, the features of the interactive nonlocal block are fused with the high-level features, thus amplifying the impacts of position and semantics on the final prediction results. Comprehensive experiments demonstrate that our click embedding approach significantly boosts the efficiency of the network and achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

The vision system provides an important way for construction robots to obtain the type and spatial location information of the object. The characteristics of the construction environment, construction object, and robot structure are jointly examined in this paper to propose an approach of object segmentation by spraying the robot based on multi-layer perceptron. Firstly, the hand-eye system experimental platform is built through establishing the mathematical model of the system and calibrating the parameters of the model. Secondly, effort is made to carry out research on image preprocessing algorithms and related experiments, and compare the effects of different binocular stereo-matching algorithms in the actual engineering environment. Finally, research and an experiment are conducted to identify the applicability and effect of the depth image object segmentation algorithm based on multi-layer perceptron. The experimental results prove that the application of multi-layer perceptron to object segmentation by spraying robots can meet the requirement on solution accuracy and is suitable for the object segmentation of complex projects in real life. This approach not only overcomes the shortcomings of the existing recognition methods that are poor in accuracy and difficult to be used widely, but also provides basic data for the subsequent three-dimensional reconstruction, thus making a significant contribution to the research of image processing by spraying robots. [ABSTRACT FROM AUTHOR]

Template-based discriminative trackers are currently the dominant tracking paradigm due to their robustness, but are restricted to bounding box tracking and a limited range of transformation models, which reduces their localization accuracy. We propose a discriminative single-shot segmentation tracker – D3S $_2$ 2 , which narrows the gap between visual object tracking and video object segmentation. A single-shot network applies two target models with complementary geometric properties, one invariant to a broad range of transformations, including non-rigid deformations, the other assuming a rigid object to simultaneously achieve robust online target segmentation. The overall tracking reliability is further increased by decoupling the object and feature scale estimation. Without per-dataset finetuning, and trained only for segmentation as the primary output, D3S $_2$ 2 outperforms all published trackers on the recent short-term tracking benchmark VOT2020 and performs very close to the state-of-the-art trackers on the GOT-10k, TrackingNet, OTB100 and LaSoT. D3S $_2$ 2 outperforms the leading segmentation tracker SiamMask on video object segmentation benchmarks and performs on par with top video object segmentation algorithms. [ABSTRACT FROM AUTHOR]

Template-based discriminative trackers are currently the dominant tracking paradigm due to their robustness, but are restricted to bounding box tracking and a limited range of transformation models, which reduces their localization accuracy. We propose a discriminative single-shot segmentation tracker – D3S $_2$ 2 , which narrows the gap between visual object tracking and video object segmentation. A single-shot network applies two target models with complementary geometric properties, one invariant to a broad range of transformations, including non-rigid deformations, the other assuming a rigid object to simultaneously achieve robust online target segmentation. The overall tracking reliability is further increased by decoupling the object and feature scale estimation. Without per-dataset finetuning, and trained only for segmentation as the primary output, D3S $_2$ 2 outperforms all published trackers on the recent short-term tracking benchmark VOT2020 and performs very close to the state-of-the-art trackers on the GOT-10k, TrackingNet, OTB100 and LaSoT. D3S $_2$ 2 outperforms the leading segmentation tracker SiamMask on video object segmentation benchmarks and performs on par with top video object segmentation algorithms. [ABSTRACT FROM AUTHOR]

Rain and snowfall will increase noise, change the resolution of objects in the point cloud and present great challenges to the accurate recognition of traffic objects. Accordingly, this article proposes an efficient real-time method for roadside 3-D light detection and ranging (LIDAR) background point cloud extraction and object segmentation under snowy weather. We first use a historical point cloud sequence to quickly construct a background model, extract the background point cloud from the current frame by using a background difference method and update the background model in real-time. Then, the noise caused by snowfall in the non-background point cloud is filtered based on the beam density difference of the object point clouds. Finally, the remaining object point cloud is accurately segmented based on the proposed hierarchical object clustering method. We use an intelligent roadside system equipped with 3-D roadside LIDAR to collect point cloud data in a snowfall environment and evaluate the proposed method qualitatively and quantitatively. Experimental results show that the proposed method effectively avoids the problem of under-segmentation and over-segmentation of object point clouds under snowy conditions, and the precision and recall rate of traffic object segmentation reached 96.41% and 95.02%, respectively, thereby indicating a significant improvement in the accuracy and reliability of traffic object detection using roadside LIDAR under snowy conditions. [ABSTRACT FROM AUTHOR]

Summary: The tracking of multiple objects has gained immense interest in several applications that ranges from video surveillance and robotics. The goal of tracking multiple objects is to discover correspondences and perform matching among various objects in frames. However, occlusion is a major issue in several classical techniques, which fail to track multiple moving objects. Here, the extraction of frames is done to obtain information about an action. Thereafter, the segmentation of objects is done using the entropy weighting K‐means (EWKM) algorithm to generate optimum segments from the frame. After that, the detection of the object is done using a deep convolutional neural network (DCNN) to obtain objects from segments. Proposed Jaya‐political optimizer (Jaya‐PO) is used to train DCNN, which is devised by collaborating political optimizer (PO) and Jaya optimization algorithm. After detecting the object, the tracking of multiple objects is done using the proposed Henry gas solubility optimized unscented Kalman filtering (HGSO‐based UKF), wherein the UKF is modified with HGSO. The proposed Jaya‐PO‐based DCNN and HGSO‐based UKF outperformed other methods with high accuracy of 92%, high multiple object tracking precision (MOTP) of 87.6%, high sensitivity of 91.7%, and high specificity of 92%. [ABSTRACT FROM AUTHOR]

We propose a dual system for unsupervised object segmentation in video, which brings together two modules with complementary properties: a space-time graph that discovers objects in videos and a deep network that learns powerful object features. The system uses an iterative knowledge exchange policy. A novel spectral space-time clustering process on the graph produces unsupervised segmentation masks passed to the network as pseudo-labels. The net learns to segment in single frames what the graph discovers in video and passes back to the graph strong image-level features that improve its node-level features in the next iteration. Knowledge is exchanged for several cycles until convergence. The graph has one node per each video pixel, but the object discovery is fast. It uses a novel power iteration algorithm computing the main space-time cluster as the principal eigenvector of a special Feature-Motion matrix without actually computing the matrix. The thorough experimental analysis validates our theoretical claims and proves the effectiveness of the cyclical knowledge exchange. We also perform experiments on the supervised scenario, incorporating features pretrained with human supervision. We achieve state-of-the-art level on unsupervised and supervised scenarios on four challenging datasets: DAVIS, SegTrack, YouTube-Objects, and DAVSOD. We will make our code publicly available. [ABSTRACT FROM AUTHOR]

Personalized Fixation-based Object Segmentation (PFOS) aims at segmenting the gazed objects in images conditioned on personalized fixations. However, the performances of existing PFOS methods are degraded when facing anomalous fixation maps (some fixations fall in the background) or enormous objects because of their poor localization ability. In this paper, we propose a novel Selective Intra-image Similarity Network (SISNet) that achieves significant performance by precisely localizing the gazed objects. First, we propose a Response Purifying Module (RPM) to eliminate the false response regions caused by anomalous fixations in the background. By suppressing these false responses, we can significantly reduce the negative impacts caused by anomalous fixations. Second, we propose an intra-image similarity module (ISM) to better localize large objects by integrating more long-range information. In addition, we propose a new Discriminative Intersection-over-Union metric that evaluates whether PFOS methods can produce distinctive predictions for varying fixations. Experiments on the PFOS and our proposed OSIE-CFPS-UN datasets prove that our network achieves remarkable improvements and outperforms existing state-of-the-art methods. Code has been published at https://www.github.com/moothes/SISNet. [ABSTRACT FROM AUTHOR]

Previous studies of multiple object tracking suggested that spatiotemporal features (e.g., speed, direction, location) and surface features (e.g., color, shape, size) can guide perceptual grouping. However, it is still unclear how target-distractor distinctiveness and target-background similarity affect grouping effects in attentive tracking. To address these two questions, three experiments have been carried out in the current study. In Experiment 1, we manipulated the target-distractor distinctiveness and found that tracking performance logarithmically improved when the target-distractor distinctiveness linearly increased. In Experiment 2a and 2b, we varied the target-background similarity and found that too high or too low target-background similarity damaged the tracking performance, while only the middle target-background similarity resulted in the best tracking performance. These findings reveal that not only target-distractor distinctiveness but also target-background similarity plays a vital role in guiding the attention of perceptual grouping in attentive tracking. The guidance induced by target-distractor distinctiveness is constrained by the spatial resolution, while the guidance induced by target-background similarity is constrained by the efficiency of object segmentation. Additionally, our results showed that tracking capacity varied with the target-distractor distinctiveness and the target-background similarity, even though the number of targets being tracked was fixed. It suggests that there may be a trade-off between the difficulty of tracking and the number of targets that can be tracked. Thus, tracking capacity is more likely to be limited by the flexible attention resources rather than the number of fixed slots. [ABSTRACT FROM AUTHOR]

Copyright of Zhejiang Electric Power is the property of Zhejiang Electric Power Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Current multi-object tracking and segmentation (MOTS) methods follow the tracking-by-detection paradigm and adopt 2D or 3D convolutions to extract instance embeddings for instance association. However, due to the large receptive field of deep convolutional neural networks, the foreground areas of the current instance and the surrounding areas containing the nearby instances or environments are usually mixed up in the learned instance embeddings, resulting in ambiguities in tracking. In this paper, we propose a highly effective method for learning instance embeddings based on segments by converting the compact image representation to un-ordered 2D point cloud representation. In this way, the non-overlapping nature of instance segments can be fully exploited by strictly separating the foreground point cloud and the background point cloud. Moreover, multiple informative data modalities are formulated as point-wise representations to enrich point-wise features. For each instance, the embedding is learned on the foreground 2D point cloud, the environment 2D point cloud, and the smallest circumscribed bounding box. Then, similarities between instance embeddings are measured for the inter-frame association. In addition, to enable the practical utility of MOTS, we modify the one-stage instance segmentation method SpatialEmbedding for instance segmentation. The resulting efficient and effective framework, named PointTrackV2, outperforms all the state-of-the-art methods including 3D tracking methods by large margins (4.8 percent higher sMOTSA for pedestrians over MOTSFusion) with the near real-time speed (20 FPS evaluated on a single 2080Ti). Extensive evaluations on three datasets demonstrate both the effectiveness and efficiency of our method. Furthermore, as crowded scenes for cars are insufficient in current MOTS datasets, we provide a more challenging dataset named APOLLO MOTS with a much higher instance density. [ABSTRACT FROM AUTHOR]

Video object segmentation (VOS) is the fundamental problem of vision-based intelligent transportation, and many VOS algorithms relying on inference from reference masks have been proposed. Due to the inherent defects of the inference strategy and the complex changes of targets, VOS methods that perform well on public datasets are usually ineffective in airport scenarios. We propose a spatiotemporal alignment network (STA-Net) that makes use of Automatic Dependent Surveillance-Broadcast (ADS-B) data as prior information to guide the long-term segmentation of aircraft. ADS-B is an airport-specific signal, which indicates the location of aircraft in real time. Based on ADS-B, we continuously generate new reference masks instead of using previous masks for inference, which greatly reduces the accumulation of inference errors. To achieve this, previous masks of each aircraft are aligned on the temporal domain based on the position information in ADS-B. All temporally-aligned masks are compared, and the one most similar to the current instant is reserved. This mask is both temporally and spatially aligned; hence it is a better reference mask for inference. Aligned masks are updated every time new ADS-B data arrive, so that they can support long-term inference. With the selected mask as a reference, aircraft of interest are segmented within a unified encoder-decoder framework over the long term. Experiments on a benchmark dataset and in a real airport scenario verify the effectiveness of the presented method. [ABSTRACT FROM AUTHOR]

This paper investigates the principles of embedding learning to tackle the challenging semi-supervised video object segmentation. Unlike previous practices that focus on exploring the embedding learning of foreground object (s), we consider background should be equally treated. Thus, we propose a Collaborative video object segmentation by Foreground-Background Integration (CFBI) approach. CFBI separates the feature embedding into the foreground object region and its corresponding background region, implicitly promoting them to be more contrastive and improving the segmentation results accordingly. Moreover, CFBI performs both pixel-level matching processes and instance-level attention mechanisms between the reference and the predicted sequence, making CFBI robust to various object scales. Based on CFBI, we introduce a multi-scale matching structure and propose an Atrous Matching strategy, resulting in a more robust and efficient framework, CFBI+. We conduct extensive experiments on two popular benchmarks, i.e., DAVIS and YouTube-VOS. Without applying any simulated data for pre-training, our CFBI+ achieves the performance ($\mathcal {J}$ J & $\mathcal {F}$ F ) of 82.9 and 82.8 percent, outperforming all the other state-of-the-art methods. Code: https://github.com/z-x-yang/CFBI. [ABSTRACT FROM AUTHOR]

With the development of deep learning, semantic segmentation has received considerable attention within the robotics community. For semantic segmentation to be applied to mobile robots or autonomous vehicles, real-time processing is essential. In this article, a new real-time semantic segmentation network, called the adjacent feature propagation network (AFPNet), is proposed to achieve high performance and fast inference. AFPNet executes in real time on a commercial embedded GPU. The network includes two new modules. The local memory module (LMM) is the first; it improves the upsampling accuracy by propagating the high-level features to the adjacent grids. The cascaded pyramid pooling module (CPPM) is the second; it reduces computational time by changing the structure of the pyramid pooling module. Using these two modules, the proposed AFPNet achieved 76.4% mean intersection-over-union on the Cityscapes test dataset, outperforming other real-time semantic segmentation networks. Furthermore, AFPNet was successfully deployed on an embedded board Jetson AGX Xavier and applied to the real-world navigation of a mobile robot, proving that AFPNet can be effectively used in a variety of real-time applications. [ABSTRACT FROM AUTHOR]

Unsupervised video object segmentation (UVOS) aims at automatically separating the primary foreground object(s) from the background in a video sequence. Existing UVOS methods either lack robustness when there are visually similar surroundings (appearance-based) or suffer from deterioration in the quality of their predictions because of dynamic background and inaccurate flow (flow-based). To overcome the limitations, we propose an implicit motion-compensated network (IMCNet) combining complementary cues (i.e., appearance and motion) with aligned motion information from the adjacent frames to the current frame at the feature level without estimating optical flows. The proposed IMCNet consists of an affinity computing module (ACM), an attention propagation module (APM), and a motion compensation module (MCM). The light-weight ACM extracts commonality between neighboring input frames based on appearance features. The APM then transmits global correlation in a top-down manner. Through coarse-to-fine iterative inspiring, the APM will refine object regions from multiple resolutions so as to efficiently avoid losing details. Finally, the MCM aligns motion information from temporally adjacent frames to the current frame which achieves implicit motion compensation at the feature level. We perform extensive experiments on $\textit {DAVIS}_{\textit {16}}$ and $\textit {YouTube-Objects}$. Our network achieves favorable performance while running at a faster speed compared to the state-of-the-art methods. Our code is available at https://github.com/xilin1991/IMCNet. [ABSTRACT FROM AUTHOR]

Convolutional Neural Networks (CNNs) are considered state of the art segmentation methods for biomedical images in general and microscopy sequences of living cells, in particular. The success of the CNNs is attributed to their ability to capture the structural properties of the data, which enables accommodating complex spatial structures of the cells, low contrast, and unclear boundaries. However, in their standard form CNNs do not exploit the temporal information available in time-lapse sequences, which can be crucial to separating touching and partially overlapping cell instances. In this work, we exploit cell dynamics using a novel CNN architecture which allows multi-scale spatio-temporal feature extraction. Specifically, a novel recurrent neural network (RNN) architecture is proposed based on the integration of a Convolutional Long Short Term Memory (ConvLSTM) network with the U-Net. The proposed ConvLSTM-UNet network is constructed as a dual-task network to enable training with weakly annotated data, in the form of approximate cell centers, termed markers, when the complete cells’ outlines are not available. We further use the fast marching method to facilitate the partitioning of clustered cells into individual connected components. Finally, we suggest an adaptation of the method for 3D microscopy sequences without drastically increasing the computational load. The method was evaluated on the Cell Segmentation Benchmark and was ranked among the top three methods on six submitted datasets. Exploiting the proposed built-in marker estimator we also present state-of-the-art cell detection results for an additional, publicly available, weekly annotated dataset. The source code is available at https://gitlab.com/shaked0/lstmUnet . [ABSTRACT FROM AUTHOR]

Pressure ulcers are ailments that require constant care and monitoring. They are painful and sometimes life-threatening. Evaluation of pressure ulcers is an important step towards treatment and healing. Presently, evaluation is carried out invasively using methods like PUSH and Bates-Jensen Wound Assessment Tool, that is not comfortable to the patient. Wounds present themselves with altered thermal profile as the activity of degrading or healing cause changes in the temperature of the affected skin/tissues. Infrared thermography is emerging as a viable medical imaging modality. The major upsides of infrared imaging are it being non-radiating and non-contact. Advances in the field of artificial intelligence, and specifically, deep learning, has widened the application domain of thermal imaging in healthcare. In this study, participants (n $=10$) with pre-existing pressure ulcers were recruited and imaged using a FLIR IR thermal camera. The acquired images were used to train an object detection model that successfully detected the existing pressure ulcers by drawing a bounding box over them. This allows the region of interest to contain the wound along with surrounding healthy skin for thermal reference. The developed model detected the pressure ulcers with a confidence level of 96-100% in the test images. The present study promises the potential application of IR-based thermal imaging in automated detection and segmentation of pressure ulcers. [ABSTRACT FROM AUTHOR]

One of the major challenges facing video object segmentation (VOS) is the gap between the training and test datasets due to unseen category in test set, as well as object appearance change over time in the video sequence. To overcome such challenges, an adaptive online framework for VOS is developed with bi-decoders mutual learning. We learn object representation per pixel with bi-level attention features in addition to CNN features, and then feed them into mutual learning bi-decoders whose outputs are further fused to obtain the final segmentation result. We design an adaptive online learning mechanism via a deviation correcting trigger such that bi-decoders online mutual learning will be activated when the previous frame is segmented well meanwhile the current frame is segmented relatively worse. Knowledge distillation from the well segmented previous frames, along with mutual learning between bi-decoders, improves generalization ability and robustness of VOS model. Thus, the proposed model adapts to the challenging scenarios including unseen categories, object deformation, and appearance variation during inference. We extensively evaluate our model on widely-used VOS benchmarks including DAVIS-2016, DAVIS-2017, YouTubeVOS-2018, YouTubeVOS-2019, and UVO. Experimental results demonstrate the superiority of the proposed model over state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper focuses on the mask utilization of video object segmentation (VOS). The mask here mains the reference masks in the memory bank, i.e., several chosen high-quality predicted masks, which are usually used with the reference frames together. The reference masks depict the edge and contour features of the target object and indicate the boundary of the target against the background, while the reference frames contain the raw RGB information of the whole image. It is obvious that the reference masks could play a significant role in the VOS, but this is not well explored yet. To tackle this, we propose to investigate the mask advantages of both the encoder and the matcher. For the encoder, we provide a unified codebase to integrate and compare eight different mask-fused encoders. Half of them are inherited or summarized from existing methods, and the other half are devised by ourselves. We find the best configuration from our design and give valuable observations from the comparison. Then, we propose a new mask-enhanced matcher to reduce the background distraction and enhance the locality of the matching process. Combining the mask-fused encoder, mask-enhanced matcher and a standard decoder, we formulate a new architecture named MaskVOS, which sufficiently exploits the mask benefits for VOS. Qualitative and quantitative results demonstrate the effectiveness of our method. We hope our exploration could raise the attention of mask utilization in VOS. [ABSTRACT FROM AUTHOR]

Existing self-supervised methods pose one-shot video object segmentation (O-VOS) as pixel-level matching to enable segmentation mask propagation across frames. However, the two tasks are not fully equivalent since O-VOS is more reliant on semantic correspondence rather than accurate pixel matching. To remedy this issue, we explore a new self-supervised framework that integrates pixel-level correspondence learning with semantic-level adaptation. The pixel-level correspondence learning is performed through photometric reconstruction of adjacent RGB frames during offline training, while semantic-level adaption operates at test-time by enforcing a bi-directional agreement of the predicted segmentation masks. In addition, we further propose a new network architecture with multi-perspective feature mining mechanism which can not only enhance reliable features but also suppress noisy ones to facilitate more robust image matching. By training the network using the proposed self-supervised framework, we achieve state-of-the-art performance on widely adopted datasets, further closing up the gap between self-supervised learning methods and their fully supervised counterparts. [ABSTRACT FROM AUTHOR]

The Normalized Cut (NCut) model is a popular graph-based model for image segmentation. But it suffers from the excessive normalization problem and weakens the small object and twig segmentation. In this paper, we propose an Explored Normalized Cut (ENCut) model that establishes a balance graph model by adopting a meaningful-loop and a k-step random walk, which reduces the energy of small salient region, so as to enhance the small object segmentation. To improve the twig segmentation, our ENCut model is further enhanced by a new Random Walk Refining Term (RWRT) that adds local attention to our model with the help of an un-supervising random walk. Finally, a move-making based strategy is developed to efficiently solve the ENCut model with RWRT. Experiments on three standard datasets indicate that our model can achieve state-of-the-art results among the NCut-based segmentation models. [ABSTRACT FROM AUTHOR]