Search

In this work, we consider transferring the structure information from large networks to compact ones for dense prediction tasks in computer vision. Previous knowledge distillation strategies used for dense prediction tasks often directly borrow the distillation scheme for image classification and perform knowledge distillation for each pixel separately, leading to sub-optimal performance. Here we propose to distill structured knowledge from large networks to compact networks, taking into account the fact that dense prediction is a structured prediction problem. Specifically, we study two structured distillation schemes: i) pair-wise distillation that distills the pair-wise similarities by building a static graph; and ii) holistic distillation that uses adversarial training to distill holistic knowledge. The effectiveness of our knowledge distillation approaches is demonstrated by experiments on three dense prediction tasks: semantic segmentation, depth estimation and object detection. Code is available at: https://git.io/StructKD, Comment: v1:10 pages cvpr2019 accepted; v2:15 pages for a journal version; Code is available at: https://github.com/irfanICMLL/structure_knowledge_distillation; fix typos

Purpose: Reliable and objective measures of abdominal fat distribution across imaging modalities are essential for various clinical and research scenarios, such as assessing cardiometabolic disease risk due to obesity. We aimed to compare quantitative measures of subcutaneous (SAT) and visceral (VAT) adipose tissues in the abdomen between computed tomography (CT) and Dixon-based magnetic resonance (MR) images using a unified computer-assisted software framework. Materials and Methods: This study included 21 subjects who underwent abdominal CT and Dixon MR imaging on the same day. For each subject, two matched axial CT and fat-only MR images at the L2-L3 and the L4-L5 intervertebral levels were selected for fat quantification. For each image, an outer and an inner abdominal wall regions as well as SAT and VAT pixel masks were automatically generated by our software. The computer-generated results were then inspected and corrected by an expert reader. Results: There were excellent agreements for both abdominal wall segmentation and adipose tissue quantification between matched CT and MR images. Pearson coefficients were 0.97 for both outer and inner region segmentation, 0.99 for SAT, and 0.97 for VAT quantification. Bland–Altman analyses indicated minimum biases in all comparisons. Conclusion: We showed that abdominal adipose tissue can be reliably quantified from both CT and Dixon MR images using a unified computer-assisted software framework. This flexible framework has a simple-to-use workflow to measure SAT and VAT from both modalities to support various clinical research applications.

Recently, referring image localization and segmentation has aroused widespread interest. However, the existing methods lack a clear description of the interdependence between language and vision. To this end, we present a bidirectional relationship inferring network (BRINet) to effectively address the challenging tasks. Specifically, we first employ a vision-guided linguistic attention module to perceive the keywords corresponding to each image region. Then, language-guided visual attention adopts the learned adaptive language to guide the update of the visual features. Together, they form a bidirectional cross-modal attention module (BCAM) to achieve the mutual guidance between language and vision. They can help the network align the cross-modal features better. Based on the vanilla language-guided visual attention, we further design an asymmetric language-guided visual attention, which significantly reduces the computational cost by modeling the relationship between each pixel and each pooled subregion. In addition, a segmentation-guided bottom-up augmentation module (SBAM) is utilized to selectively combine multilevel information flow for object localization. Experiments show that our method outperforms other state-of-the-art methods on three referring image localization datasets and four referring image segmentation datasets.

Image segmentation has been a well-addressed problem in pattern recognition for the last few decades. As a sub-problem of image segmentation, the background separation in biomedical images generated by magnetic resonance imaging (MRI) has also been of interest in the applied mathematics literature. Level set evolution of active contours idea can successfully be applied to MRI images to extract the region of interest (ROI) as a crucial preprocessing step for medical image analysis. In this study, we use the classical level set solution to create binary masks of various brain MRI images in which black color implies background and white color implies the ROI. We further used the MRI image and mask image pairs to train a deep neural network (DNN) architecture called U-Net, which has been proven to be a successful model for biomedical image segmentation. Our experiments have shown that a properly trained U-Net can achieve a matching performance of the level set method. Hence we were able to train a U-Net by using automatically generated input and label data successfully. The trained network can detect ROI in MRI images faster than the level-set method and can be used as a preprocessing tool for more enhanced medical image analysis studies.

International audience; According to the industry, the value of wood logs is heavily influenced by their internal structure, particularly the distribution of knots within the trees. Nowadays, CT scanners combined with classical computer vision approach are the most common tool for obtaining reliable and accurate images of the interior structure of trees. Knowing where the tree semantic features, especially knots, contours and centers are within a tree could improve the efficiency of the overall tree industry by minimizing waste and enhancing the quality of wood-log by-products. However, this requires to automatically process the CT-scanner images so as to extract the different elements such as tree centerline, knot localization and log contour, in a robust and efficient manner. In this paper, we propose an effective methodology based on deep learning for performing these different tasks by processing CT-scanner images with deep convolutional neural networks. To meet this objective, three end-to-end trainable pipelines are proposed. The first pipeline is focused on centers detection using CNNs architecture with a regression head, the second and the third one address contour estimation and knot detection as a binary segmentation task based on an Encoder-Decoder architecture. The different architectures are tested on several tree species. With these experiments, we demonstrate that our approaches can be used to extract the different elements of trees in a precise manner while preserving good performances of robustness. The main objective was to demonstrate that methods based on deep learning might be used and have a relevant potential for segmentation and regression on CT-scans of tree trunks.

Bias and fairness of biometric algorithms have been key topics of research in recent years, mainly due to the societal, legal and ethical implications of potentially unfair decisions made by automated decision-making models. A considerable amount of work has been done on this topic across different biometric modalities, aiming at better understanding the main sources of algorithmic bias or devising mitigation measures. In this work, we contribute to these efforts and present the first study investigating bias and fairness of sclera segmentation models. Although sclera segmentation techniques represent a key component of sclera-based biometric systems with a considerable impact on the overall recognition performance, the presence of different types of biases in sclera segmentation methods is still underexplored. To address this limitation, we describe the results of a group evaluation effort (involving seven research groups), organized to explore the performance of recent sclera segmentation models within a common experimental framework and study performance differences (and bias), originating from various demographic as well as environmental factors. Using five diverse datasets, we analyze seven independently developed sclera segmentation models in different experimental configurations. The results of our experiments suggest that there are significant differences in the overall segmentation performance across the seven models and that among the considered factors, ethnicity appears to be the biggest cause of bias. Additionally, we observe that training with representative and balanced data does not necessarily lead to less biased results. Finally, we find that in general there appears to be a negative correlation between the amount of bias observed (due to eye color, ethnicity and acquisition device) and the overall segmentation performance, suggesting that advances in the field of semantic segmentation may also help with mitigating bias.

Now days, the human being are affecting by so many diseases in all over the world. If we will consider about the tumor in the encephalon, then detection of tumor often involves radiological imaging. Radiological Imaging is used to check the spread of tumor and progress of treatment and also used to monitor cancer. The manual detection of the Encephalon tumor becomes a meticulous and chaotic task for the clinical experts. If the doctors or various departmental specialists want to eliminate the spoiled cancer areas from Magnetic Resonance Images, then it becomes very hectic and takes more time. The main aim of this work is to detect the encephalon tumor regions. Once the brain is going to be scan then it offers proficient and quick identification of encephalon tumor. To modernize the performance and abbreviate the intricacy involved in the image detection process in the proposed system as well as the Fuzzy CMean predicated image segmentation processes are used. Furthermore, to expand the variance of different images, particularly when the data are worn in image is signified beside different variance of images and then another technique has been implemented in the paper that is Histogram Equalization. The proposed method is completely automatic and tested on different kinds of brain MR images and proved as robust.

Physiological studies have shown that there are some differences in speech and facial activities between depressive and healthy individuals. Based on this fact, we propose a novel Spatio-Temporal Attention (STA) network and a Multimodal Attention Feature Fusion (MAFF) strategy to obtain the multimodal representation of depression cues for predicting the individual depression level. Specifically, we firstly divide the speech amplitude spectrum/video into fixed-length segments and input these segments into the STA network, which not only integrates the spatial and temporal information through attention mechanism, but also emphasizes the audio/video frames related to depression detection. The audio/video segment-level feature is obtained from the output of the last full connection layer of the STA network. Secondly, this paper employs the eigen evolution pooling method to summarize the changes of each dimension of the audio/video segment-level features to aggregate them into the audio/video level feature. Thirdly, the multimodal representation with modal complementary information is generated using the MAFF and inputs into the support vector regression predictor for estimating depression severity. Experimental results on the AVEC2013 and AVEC2014 depression databases illustrate the effectiveness of our method.

The theme of this study is to provide a detailed description of its recent improvements in image segmentation and lesion classification in disease prognosis. Previous studies have shown that gray-white matter hyperintensities (GWMH) is one of the hallmarks of Nipah encephalitis, which sometimes occurs during the incubation period. Predicting this type of inflammation is a challenging task because it involves some unknown medical risk factors. A typical Magnetic Resonance Imaging (MRIs) is the best non-invasive system to analyze the anatomical structure of the brain. In-depth analysis of the defined pathological structure from isolated MR imaging leads to a reduction in the processing time of the prognostic model. Modern learning techniques such as Machine Learning, Computer Vision, and Deep Learning are the most promising techniques for determining the optimal outcome, computer can able to learn and extract useful information from historical data using various algorithms. Disease prognosis based on deep learning is sophisticated, so it can handle a variety of difficult tasks including image processing, classification, and feature extraction, noise and object detection. Diffusion-weighted imaging (DWI) in MRIs is a clinical prototype that can be used to diagnose brain abnormalities and to evaluate the microscopic architectural and molecular function of human organs or tissues. In this study, we summarize the results of diagnosing Nipah encephalitis using some publicly available brain encephalitis and encephalopathy databases.

Agriculture plays a major role in developing countries like India, however the food security still remains a vital issue. Most of the crops get wasted due to lack of storage facility, transportation, and plant diseases. More than 15% of the crops get wasted in India due to diseases and hence it has become one of the major concern to be resolved. There is a need of automatic system that can identify these diseases and help farmers to take appropriate steps to get rid of crop loss. Farmers have followed the conventional method of identifying the plant disease with their naked eyes, and it not possible for all the farmers to identify these diseases the same way. With the advance in Artificial Intelligence, there is a need to incorporate the facilities of the computer vision in the field of agriculture. Deep Learning rich libraries and user as well as developer friendly environment to work with, all these qualities make Deep Learning as the favorable method to get started with this problem. In this paper we have used Deep Learning because of the advantages it offers to work with images especially in image classification to get improvised results. The methodology includes taking leaves of infected crops and label them as per the disease pattern. The images of infected leaves are processed pixel based operations are applied to improve the information from the image. As a next step feature extraction is done followed by image segmentation and at the last classification of crop diseases based on the patterns extracted from the diseased leaves. The CNN (Convolutional Neural Network) is used for the classification of diseases, for the demonstration purpose the public dataset is used consisting of around 87 K images (RGB type images) including healthy as well as diseased leaves.

The object of this study is the process of making a management decision based on the analysis of information from space surveillance systems. Unlike the well-known ones, the method of making a management decision based on the analysis of information from space surveillance systems involves: –segmentation of an optoelectronic image; –determination and prediction of a priori probabilities of possible environmental states; –an application for making a management decision of a combination of Bayes criteria and a minimum of variance. Experimental studies have been carried out on making a management decision based on the analysis of information from space surveillance systems. To conduct experimental research on making a management decision based on the analysis of information from space surveillance systems, a model problem has been stated. As images from space surveillance systems, images obtained from the WorldView-2 spacecraft (USA) with a difference of four days were considered. The vegetation index was calculated, and the probabilities of degradation dynamics of plant segments were determined. It was established that the maximum value of the estimated functional is achieved when choosing a solution φ1, which is optimal according to the Bayesian criterion and the criterion of minimum variance. The quality of management decision-making was assessed by the well-known and developed methods. To assess the quality of management decision-making, the concepts of objectivity of the decision-making method and the selectivity of the decision-making method by known and developed method were introduced. It has been established that both methods are objective, and the improved method is more selective (the gain is 2.6 times). This becomes possible through the use of information from space surveillance systems.

This study presents an extensive analysis of ten emerging swarm intelligence metaheuristic techniques, namely Emperor Penguins Colony (EPC), Harris Hawks Optimizer (HHO), Butterfly Optimization Algorithm (BOA), Spotted Hyena Optimizer (SHO), Crow search algorithm(CSA), Whale optimization algorithm(WOA), Red Deer Algorithm (RDA), Ant Lion Optimizer (ALO), Dragonfly Algorithm (DA) and Grey wolf optimization (GWO). Here, a Quad–fold review strategy comprised of planning, shortlisting, extraction, and execution have been adhered to compile this meta-analysis. The mathematical models and working principles of these techniques have been briefly elucidated. The variants of these meta-heuristic techniques have also been explored and presented. The research trend of these metaheuristic methods has also been highlighted. The findings indicate that these methods are widely used to solve different problems viz: image segmentation, optimal power flow, air pollution forecasting, drug design, wireless sensor networks, disease diagnosis, transport, and routing. Furthermore, in the healthcare sector, the use of SI techniques in selecting optimal features for diagnosis of different diseases like Cancer, Alzheimer's, Kidney disease, Anemia, Viral infection, Skin diseases have also been highlighted. Moreover, it is observed that the education-related optimization problems have been deeply explored by these meta-heuristic techniques whereas, weather forecasting is recognized as the least explored area. The binary, chaotic, and hybrid variants of EPC, HHO, BOA, SHO, CSA, WOA RDA, ALO, DA, and GWO of these metaheuristics techniques need to be deeply explored in healthcare for skin diseases, ophthalmology, viral infection, allergy along with distinct mental disorders. Finally, for better performance, the exploitation and exploration phases of these methods need to be carefully balanced.

Image-based intelligent detection of road cracks with high accuracy and efficiency is vital to the overall condition assessment of the pavement. However, significant problems of continuous cracks interruption and background discrete noise misidentification are frequently observed in current semantic segmentation of pavement cracks, which mainly caused by traditional segmentation convolutional neural networks. This paper proposes a skip-level round-trip sampling block structure with the implementation of convolutional neural networks, thereby constructed a novel pixel level semantic segmentation network called CrackW-Net. After that, two datasets, including the widely recognized Crack500 dataset and a self-built dataset, were used to train two versions CrackW-Net, FCN, U-Net and ResU-Net. Meanwhile, comparative experiments are conducted among all these network models for crack detection. Results show that CrackW-Net without residual block performs the best in the task of pavement crack segmentation.

Accurate medical image segmentation of brain tumors is necessary for the diagnosing, monitoring, and treating disease. In recent years, with the gradual emergence of multi-sequence magnetic resonance imaging (MRI), multi-modal MRI diagnosis has played an increasingly important role in the early diagnosis of brain tumors by providing complementary information for a given lesion. Different MRI modalities vary significantly in context, as well as in coarse and fine information. As the manual identification of brain tumors is very complicated, it usually requires the lengthy consultation of multiple experts. The automatic segmentation of brain tumors from MRI images can thus greatly reduce the workload of doctors and buy more time for treating patients. In this paper, we propose a multi-modal brain tumor segmentation framework that adopts the hybrid fusion of modality-specific features using a self-supervised learning strategy. The algorithm is based on a fully convolutional neural network. Firstly, we propose a multi-input architecture that learns independent features from multi-modal data, and can be adapted to different numbers of multi-modal inputs. Compared with single-modal multi-channel networks, our model provides a better feature extractor for segmentation tasks, which learns cross-modal information from multi-modal data. Secondly, we propose a new feature fusion scheme, named hybrid attentional fusion. This scheme enables the network to learn the hybrid representation of multiple features and capture the correlation information between them through an attention mechanism. Unlike popular methods, such as feature map concatenation, this scheme focuses on the complementarity between multi-modal data, which can significantly improve the segmentation results of specific regions. Thirdly, we propose a self-supervised learning strategy for brain tumor segmentation tasks. Our experimental results demonstrate the effectiveness of the proposed model against other state-of-the-art multi-modal medical segmentation methods.

Amaçları önceden tanımlanmış görevlerle ortaya konulan ve genellikle otomatik uygulamalar bağlamında anlamlı olan imge bölütleme problemi, ilgilenilen belli piksellerin çevrelerinden yalıtılmasını ele alır. İmgelerde çok ve ham durumda bulunarak artıklık, yararsızlık ve hatta görev-zorlaştırma barındıran verinin basitleştirilip, yalnız ilginç bölümleri içeren derlitoplu gösterimlerinin elde edilmesi ve bu bölümlerden tanımlayıcı özniteliklerin çıkarılması gerekir. Bu çalışmada, özel bir alanla ilgili olan ve ultrason görüntüleme ile edinilen enine-kesit fetal kafataslarını gösteren monokrom imgelerdeki kafatası çevritlerinin bulunması için kullanılan buluşsal bir yaklaşımdan söz edilmektedir. Bölütleme sürecinin başında, kullanıcının girdi imgedeki kafatası çevriti üstünde az sayıda noktayı elle işaretlemesi beklenmektedir. Çevritlerin parlak yoğunluklu piksellerden oluştuğu olgusundan ve görüntüleme teknolojisinden kaynaklanan bölütler arasında kopukluklar gözlenmesinden hareketle, açıklanan buluşsal bölütleme yöntemi, ortalama şekil modeli ve yoğunluğa-dayalı ortalama konum bulma kavramlarından yararlanmaktadır. Örnek imgelerdeki sonuçlar, hem görsel olarak hem de otomatik tanı sistemlerinde girdi olarak kullanıldığında, doyurucudur., The image segmentation problem, whose purposes are set by predefined tasks and which is generally significant in the context of automatic applications, deals with isolating image pixels that are of particular interest. Data; that are either redundant, useless or even task-complicating by being too much and raw, must be simplified, compact representations including only interesting portions must be obtained and descriptively useful features out of those must be extracted. In this work, a specific domain is considered and a heuristic approach for detecting skull contours in monochrome images acquired via the ultrasound technology displaying transcerebellar fetal skulls is described. At the start of the segmentation process, the user is expected to mark few points manually along the skull boundary on the input image. Knowing that skull contours are composed of pixels of high intensity and that discontinuities are observed between segments arising from the imaging technology, the described heuristic segmentation method utilizes the concepts of average shape model and intensity-based computation of average positions. The results for sample images, both visually and when they are used as inputs to automated diagnosis systems, are satisfactory.

Unsupervised domain adaptation (UDA) is to learn classification models that make predictions for unlabeled data on a target domain, given labeled data on a source domain whose distribution diverges from the target one. Mainstream UDA methods strive to learn domain-aligned features such that classifiers trained on the source features can be readily applied to the target ones. Although impressive results have been achieved, these methods have a potential risk of damaging the intrinsic data structures of target discrimination, raising an issue of generalization particularly for UDA tasks in an inductive setting. To address this issue, we are motivated by a UDA assumption of structural similarity across domains, and propose to directly uncover the intrinsic target discrimination via constrained clustering, where we constrain the clustering solutions using structural source regularization that hinges on the very same assumption. Technically, we propose a hybrid model of Structurally Regularized Deep Clustering, which integrates the regularized discriminative clustering of target data with a generative one, and we thus term our method as H-SRDC. Our hybrid model is based on a deep clustering framework that minimizes the Kullback-Leibler divergence between the distribution of network prediction and an auxiliary one, where we impose structural regularization by learning domain-shared classifier and cluster centroids. By enriching the structural similarity assumption, we are able to extend H-SRDC for a pixel-level UDA task of semantic segmentation. We conduct extensive experiments on seven UDA benchmarks of image classification and semantic segmentation. With no explicit feature alignment, our proposed H-SRDC outperforms all the existing methods under both the inductive and transductive settings. We make our implementation codes publicly available at https://github.com/huitangtang/H-SRDC., Journal extension of our preliminary CVPR conference paper, under review, 16 pages, 8 figures, 9 tables

Purpose Leukaemia is diagnosed conventionally by observing the peripheral blood and bone marrow smear using a microscope and with the help of advanced laboratory tests. Image processing-based methods, which are simple, fast, and cheap, can be used to detect and classify leukemic cells by processing and analysing images of microscopic smear. The proposed study aims to classify Acute Lymphoblastic Leukaemia (ALL) by Deep Learning (DL) based techniques. Procedures The study used Deep Convolutional Neural Networks (DNNs) to classify ALL according to WHO classification scheme without using any image segmentation and feature extraction that involves intense computations. Images from an online image bank of American Society of Haematology (ASH) were used for the classification. Findings A classification accuracy of 94.12% is achieved by the study in isolating the B-cell and T-cell ALL images using a pretrained CNN AlexNet as well as LeukNet, a custom-made deep learning network designed by the proposed work. The study also compared the classification performances using three different training algorithms. Conclusions The paper detailed the use of DNNs to classify ALL, without using any image segmentation and feature extraction techniques. Classification of ALL into subtypes according to the WHO classification scheme using image processing techniques is not available in literature to the best of the knowledge of the authors. The present study considered the classification of ALL only, and detection of other types of leukemic images can be attempted in future research.

With the unprecedented developments in deep learning, automatic segmentation of main abdominal organs seems to be a solved problem as state-of-the-art (SOTA) methods have achieved comparable results with inter-rater variability on many benchmark datasets. However, most of the existing abdominal datasets only contain single-center, single-phase, single-vendor, or single-disease cases, and it is unclear whether the excellent performance can generalize on diverse datasets. This paper presents a large and diverse abdominal CT organ segmentation dataset, termed AbdomenCT-1K, with more than 1000 (1K) CT scans from 12 medical centers, including multi-phase, multi-vendor, and multi-disease cases. Furthermore, we conduct a large-scale study for liver, kidney, spleen, and pancreas segmentation and reveal the unsolved segmentation problems of the SOTA methods, such as the limited generalization ability on distinct medical centers, phases, and unseen diseases. To advance the unsolved problems, we further build four organ segmentation benchmarks for fully supervised, semi-supervised, weakly supervised, and continual learning, which are currently challenging and active research topics. Accordingly, we develop a simple and effective method for each benchmark, which can be used as out-of-the-box methods and strong baselines. We believe the AbdomenCT-1K dataset will promote future in-depth research towards clinical applicable abdominal organ segmentation methods. The datasets, codes, and trained models are publicly available at https://github.com/JunMa11/AbdomenCT-1K., 20 pages

The subject of research in the article is the module of automatic segmentation and subtraction of the background, which is created, based on the sequential application of methods of image preprocessing and modified method of interactive segmentation of images and implemented in the system of optical monitoring of the air situation. The aim of the work is to develop an image segmentation module to increase the efficiency of recognition of an air object type on a video image in the system of visual monitoring of the air environment by means of qualitative automatic segmentation. To solve this problem, a modified interactive algorithm in the mode of automatic selection of an object in the image, which allows more accurately, without the participation of the operator, to determine the foreground pixels of the image for further recognition of the type of airborne object. The following tasks are solved in the article: the analysis of existing methods of binarization of color images for semantic segmentation of images, which are used in image recognition systems; the development of a pipeline of methods for automatic segmentation of images in the system of optical monitoring of the air environment. In the work, the following methods are used: methods of digital image processing, methods of filtering and semantic segmentation of images, methods of graph analysis. The following results are obtained: the results of image processing with the proposed module of segmentation and background subtraction confirm the performance of the module procedures. The developed pipeline of methods included in the module demonstrates correct segmentation in 93% of test images in automatic mode without operator participation, which allows us to conclude about the effectiveness of the proposed module. Conclusions: The implementation of the developed module of segmentation and background subtraction for the system of optical monitoring of the air environment allowed to solve the problem of segmentation of video images for further recognition of aerial objects in the system of optical monitoring of the air environment in automatic mode with a high degree of reliability, thus increasing the operational efficiency of this system., Предметом исследования в статье является модуль автоматической сегментации и вычитания фона, созданный на основе конвейера методов предварительной обработки изображения и модифицированного метода интерактивной сегментации изображений, и внедренный в систему оптического мониторинга воздушной обстановки. Целью работы является разработка модуля сегментации изображений для повышения эффективности распознавания типа воздушного объекта на видеоизображении в системе визуального мониторинга воздушной обстановки путем качественной автоматической сегментации. Для решения этой задачи используется модифицированный интерактивный алгоритм в режиме автоматического выделения объекта на изображении, который позволяет точнее без участия оператора определять пиксели переднего плана изображения для дальнейшего распознавания типа воздушного объекта. В статье решаются следующие задачи: анализ существующих методов бинаризации цветного изображения для семантической сегментации изображений, которые используются в системах распознавания образов; разработка конвейера методов для автоматической сегментации изображений в системе оптического мониторинга воздушной обстановки. В работе используются методы: методы цифровой обработки изображений, методы фильтрации и семантической сегментации изображений, методы анализа графов. Получены следующие результаты: результаты обработки изображений с помощью предложенного модуля сегментации и вычитания фона подтверждают работоспособность процедур модуля. Разработанный конвейер методов, включенных в модуль, демонстрирует правильную сегментацию в 93% тестовых изображений в автоматическом режиме без участия оператора, что позволяет сделать вывод об эффективности применения предложенного модуля. Выводы: Внедрение разработанного модуля сегментации и вычитания фона для системы оптического мониторинга воздушной обстановки позволило с высокой степенью достоверности решить задачу сегментации видеоизображений для дальнейшего распознавания воздушных объектов в системе оптического мониторинга воздушной обстановки в автоматическом режиме, благодаря чему увеличилась эффективность эксплуатации данной системы., Предметом дослідження у статті є модуль автоматичної сегментації та віднімання фону, створений на основі конвеєра методів попередньої обробки зображення та модифікованого методу інтерактивної сегментації зображень, та впроваджений у систему оптичного моніторингу повітряної обстановки. Метою роботи є розробка модуля сегментації зображень для підвищення ефективності розпізнавання типу повітряного об’єкта на відеозображенні у системі візуального моніторингу повітряної обстановки шляхом якісної автоматичної сегментації. Для вирішення цього завдання використовується модифікований інтерактивний алгоритм у режимі автоматичного виділення об’єкта на зображенні, що дозволяє точніше без участі оператора визначати пікселі переднього плану зображення для подальшого розпізнавання типу повітряного об’єкта. У статті вирішуються такі задачі: аналіз існуючих методів бінаризації кольорового зображення для семантичної сегментації зображень, що використовуються у системах розпізнавання образів; розробка конвеєра методів автоматичної сегментації зображень у системі оптичного моніторингу повітряної обстановки. У роботі використовуються методи: методи цифрової обробки зображень, методи фільтрації та семантичної сегментації зображень, методи аналізу графів. Отримано такі результати: результати обробки зображень за допомогою запропонованого модуля сегментації та віднімання фону підтверджують працездатність процедур модуля. Розроблений конвеєр методів, включених у модуль, демонструє правильну сегментацію 93% тестових зображень в автоматичному режимі без участі оператора, що дозволяє зробити висновок про ефективність застосування запропонованого модуля. Висновки: Впровадження розробленого модуля сегментації та віднімання фону для системи оптичного моніторингу повітряної обстановки дозволило з високим ступенем достовірності вирішити задачу сегментації відеозображень для подальшого розпізнавання повітряних об’єктів у системі оптичного моніторингу повітряної обстановки в автоматичному режимі, завдяки чому збільшилася ефективність експлуатації цієї системи.

This study proposes an optimal approach to reduce noise in mammographic images and to identify salt-and-pepper, Gaussian, Poisson, and impact noises to determine the exact mass detection operation after these noise reductions. It therefore offers a method for noise reduction operations called quantum wavelet transform filtering and a method for precision mass segmentation called the image morphological operations in mammographic images based on the classification with an atrous pyramid convolutional neural network (APCNN) as a deep learning model. The hybrid approach called a QWT-APCNN is evaluated in terms of criteria compared with previous methods such as peak signal-to-noise ratio (PSNR) and mean-squared error (MSE) in noise reduction and accuracy of detection for mass area recognition. The proposed method presents more performance of noise reduction and segmentation in comparison with state-of-the-art methods. In this paper, we used the APCNN based on the convolutional neural network (CNN) as a new deep learning method, which is able to extract features and perform classification simultaneously, but it is intended as far as possible, empirically for the purpose of this research to be able to determine breast cancer and then identify the exact area of the masses and then classify them according to benign, malignant, and suspicious classes. The obtained results presented that the proposed approach has better performance than others based on some evaluation criteria such as accuracy with 98.57%, sensitivity with 90%, specificity with 85%, and also ROC and AUC with a rate of 86.77.

Over the years, photometric three-dimensional (3D) reconstruction gained increasing importance in several disciplines, especially in cultural heritage preservation. While increasing sizes of images and datasets enhanced the overall reconstruction results, requirements in storage got immense. Additionally, unsharp areas in the background have a negative influence on 3D reconstructions algorithms. Handling the sharp foreground differently from the background simultaneously helps to reduce storage size requirements and improves 3D reconstruction results. In this article, we examine regions outside the Depth of Field (DoF) and eliminate their inaccurate information to 3D reconstructions. We extract DoF maps from the images and use them to handle the foreground and background with different compression backends, making sure that the actual object is compressed losslessly. Our algorithm achieves compression rates between 1:8 and 1:30 depending on the artifact and DoF size and improves the 3D reconstruction.

Image segmentation with language referring expression can complete object segmentation based on expression text. Existing image segmentation methods show good results on high-performance computers, but most robot systems need real-time and high accuracy. At present, most methods cannot meet these requirements well. Therefore, we propose a high-precision and real-time deep learning network that integrates the two tasks of image segmentation with language referring expression and referring expression comprehension and then treats them as two branches. Specifically, the proposed network first merges the two tasks. The feature maps of different scales extracted by each branch are fed back to the two branches to obtain prediction results. These two tasks promote and restrict each other. Experiments show that our method has better real-time performance and higher accuracy than existing methods.

Automatic crack detection is vital for efficient and economical road maintenance. With the explosive development of convolutional neural networks (CNNs), recent crack detection methods are mostly based on CNNs. In this article, we propose a deeply supervised convolutional neural network for crack detection via a novel multiscale convolutional feature fusion module. Within this multiscale feature fusion module, the high-level features are introduced directly into the low-level features at different convolutional stages. Besides, deep supervision provides integrated direct supervision for convolutional feature fusion, which is helpful to improve model convergency and final performance of crack detection. Multiscale convolutional features learned at different convolution stages are fused together to robustly represent cracks, whose geometric structures are complicated and hardly captured by single-scale features. To demonstrate its superiority and generalizability, we evaluate the proposed network on three public crack data sets, respectively. Sufficient experimental results demonstrate that our method outperforms other state-of-the-art crack detection, edge detection, and image segmentation methods in terms of F1-score and mean IU.

A finite element-based image segmentation strategy enhanced by an anisotropic mesh adaptation procedure is presented. The methodology relies on a split Bregman algorithm for the minimisation of a region-based energy functional and on an anisotropic recovery-based error estimate to drive mesh adaptation. More precisely, a Bayesian energy functional is considered to account for image spatial information, ensuring that the methodology is able to identify inhomogeneous spatial patterns in complex images. In addition, the anisotropic mesh adaptation guarantees a sharp detection of the interface between background and foreground of the image, with a reduced number of degrees of freedom. The resulting split-adapt Bregman algorithm is tested on a set of real images showing the accuracy and robustness of the method, even in the presence of Gaussian, salt and pepper and speckle noise., 41 pages, 13 figures, 5 tables

Most of the recent image segmentation methods have tried to achieve the utmost segmentation results using large-scale pixel-level annotated data sets. However, obtaining these pixel-level annotated training data is usually tedious and expensive. In this work, we address the task of semisupervised semantic segmentation, which reduces the need for large numbers of pixel-level annotated images. We propose a method for semisupervised semantic segmentation by improving the confidence of the predicted class probability map via two parts. First, we build an adversarial framework that regards the segmentation network as the generator and uses a fully convolutional network as the discriminator. The adversarial learning makes the prediction class probability closer to 1. Second, the information entropy of the predicted class probability map is computed to represent the unpredictability of the segmentation prediction. Then, we infer the label-error map of the segmentation prediction and minimize the uncertainty on misclassified regions for unlabeled images. In contrast to existing semisupervised and weakly supervised semantic segmentation methods, the proposed method results in more confident predictions by focusing on the misclassified regions, especially the boundary regions. Our experimental results on the PASCAL VOC 2012 and PASCAL-CONTEXT data sets show that the proposed method achieves competitive segmentation performance.

Augmented reality can enhance human perception to experience a virtual-reality intertwined world by computer vision techniques. However, the basic techniques cannot handle complex large-scale scenes, tackle real-time occlusion, and render virtual objects in augmented reality. Therefore, this paper studies potential solutions, such as visual SLAM and image segmentation, that can address these challenges in the augmented reality visualizations. This paper provides a review of advanced visual SLAM and image segmentation techniques for augmented reality. In addition, applications of machine learning techniques for improving augmented reality are presented.

Fuzzy c-means (FCM) algorithms with spatial information have been widely applied in the field of image segmentation. However, most of them suffer from two challenges. One is that introduction of fixed or adaptive single neighboring information with narrow receptive field limits contextual constraints leading to clutter segmentations. The other is that incorporation of superpixels with wide receptive field enlarges spatial coherency leading to block effects. To address these challenges, we propose fuzzy Students t-distribution model based on richer spatial combination (FRSC) for image segmentation. In this Paper, we make two significant contributions. The first is that both narrow and wide receptive fields are integrated into the objective function of FRSC, which is convenient to mine image features and distinguish local difference. The second is that the rich spatial combination under Students t-distribution ensures that spatial information is introduced into the updated parameters of FRSC,which is helpful in finding a balance between the noise-immunity and detail-preservation. Experimental results on synthetic and publicly available images, further demonstrate that the proposed FRSC addresses successfully the limitations of FCM algorithms with spatial information and provides better segmentation results than state-of-the-art clustering algorithms. National Natural Science Foundation of China (Grant Number: Grant 61871259, Grant 61861024); Natural Science Basic Research Program of Shaanxi (Grant Number: 2021JC-47); Key Research and Development Program of Shaanxi (Grant Number: 2021ZDLGY08-07).

Although spatial information of images usually enhance the robustness of the Fuzzy C-Means (FCM) algorithm, it greatly increases the computational costs for image segmentation. To achieve a sound trade-off between the segmentation performance and the speed of clustering, we come up with a Kullback-Leibler (KL) divergence-based FCM algorithm by incorporating a tight wavelet frame transform and a morphological reconstruction operation. To enhance FCM's robustness, an observed image is first filtered by using the morphological reconstruction. A tight wavelet frame system is employed to decompose the observed and filtered images so as to form their feature sets. Considering these feature sets as data of clustering, an modified FCM algorithm is proposed, which introduces a KL divergence term in the partition matrix into its objective function. The KL divergence term aims to make membership degrees of each image pixel closer to those of its neighbors, which brings that the membership partition becomes more suitable and the parameter setting of FCM becomes simplified. On the basis of the obtained partition matrix and prototypes, the segmented feature set is reconstructed by minimizing the inverse process of the modified objective function. To modify abnormal features produced in the reconstruction process, each reconstructed feature is reassigned to the closest prototype. As a result, the segmentation accuracy of KL divergence-based FCM is further improved. What's more, the segmented image is reconstructed by using a tight wavelet frame reconstruction operation. Finally, supporting experiments coping with synthetic, medical and color images are reported. Experimental results exhibit that the proposed algorithm works well and comes with better segmentation performance than other comparative algorithms. Moreover, the proposed algorithm requires less time than most of the FCM-related algorithms., Comment: This paper has been withdrawn by the author due to a crucial definition error of objective function

Background Machine learning (ML) prepares and trains a model through supervised or unsupervised learning methods. Sputum, a respiratory tract secretion, is a common laboratory specimen that aids in diagnosing respiratory diseases, including pulmonary tuberculosis (TB). Gram stain is an easy, cost-effective stain, which may be applied to sputum smears to screen out an unsatisfactory sample. ML model may help in screening sputum smears. Methods This collaborative project was carried out from June 2020-July 2021. In this study, a color-based segmentation ML algorithm using K-Means clustering was developed. A library of stained sputum smears was built. The Bartletts criteria (based on neutrophil and squamous cell count) for screening and selecting satisfactory sputum smears were used. A smartphone camera was used to take several photographs of satisfactory, as well as unsatisfactory, smears. The image segmentation algorithm was applied to medical image analysis, color-segmentation of sputum images was done. The hue saturation value (HSV) color ranges were defined on a prototype image. Then, all connected pixels were identified as a single object, and morphological operations were applied. Results Usage of AI-driven model on the slide-image revealed the slide adequacy as the cell count was acceptable based on Bartlett’s criteria. Both the manual cell counts (Range: 126–203 neutrophils, 14–47 squamous cells) and the model counts (Range: 117–242 neutrophils, 14–37 squamous cells) are within acceptable limits. Conclusion The use of a model to screen a large number of sputum slides may be a boon in resource-limited settings where trained microscopists may not be easily available.

Medical image segmentation and registration are two tasks to analyze the anatomical structures in clinical research. Still, deep-learning solutions utilizing the connections between segmentation and registration remain underdiscovered. This article designs a joint learning framework named RSegNet that can realize concurrent deformable registration and segmentation by minimizing an integrated loss function, including three parts: diffeomorphic registration loss, segmentation similarity loss, and dual-consistency supervision loss. The probabilistic diffeomorphic registration branch could benefit from the auxiliary segmentations available from the segmentation branch to achieve anatomical consistency and better deformation regularity by dual-consistency supervision. Simultaneously, the segmentation performance could also be improved by data augmentation based on the registration with well-behaved diffeomorphic guarantees. Experiments on the human brain 3-D magnetic resonance images have been implemented to demonstrate the effectiveness of our approach. We trained and validated RSegNet with 1000 images and tested its performances on four public datasets, which shows that our method successfully yields concurrent improvements of both segmentation and registration compared with separately trained networks. Specifically, our method can increase the accuracy of segmentation and registration by 7.0% and 1.4%, respectively, in terms of Dice scores.

The sudden climate change occurring in different places in the world has made disasters more unpredictable than before. In addition, responses are often late due to manual processes that have to be performed by experts. Consequently, major advances in computer vision (CV) have prompted researchers to develop smart models to help these experts. We need a strong image representation model, but at the same time, we also need to prepare for a deep learning environment at a low cost. This research attempts to develop transfer learning models using low-cost masking pre-processing in the experimental building damage (xBD) dataset, a large-scale dataset for advancing building damage assessment. The dataset includes eight types of disasters located in fifteen different countries and spans thousands of square kilometers of satellite images. The models are based on U-Net, i.e., AlexNet, visual geometry group (VGG)-16, and ResNet-34. Our experiments show that ResNet-34 is the best with an F1 score of 71.93%, and an intersection over union (IoU) of 66.72%. The models are built on a resolution of 1,024 pixels and use only first-tier images compared to the state-of-the-art baseline. For future orientations, we believe that the approach we propose could be beneficial to improve the efficiency of deep learning training.

An important safety problem in healthcare is diagnostic error, with major errors being found in 10%-20% of autopsies. This suggests that an average of 60,000 patients die annually in the United States from error during diagnosis. Medical diagnosis is a critical task that must be performed efficiently and accurately to ensure adequate patient treatment. Factors including lack of communication, doctor inexperience, and lack of time with patients contribute to these errors. With the complexity and rise of data in healthcare, machine learning (ML) technology has the potential to transform many aspects of patient care including diagnosing diseases. There are studies suggesting that artificial intelligence (AI) can perform as well as or better than humans at key healthcare tasks such as identifying a malignant tumor. Google’s ML applications in healthcare detected breast cancer with 89% accuracy, which is an 11% increase compared to currently used diagnostic methods. ML is a common form of AI that ‘learns’ by training models with data and can be used for tasks like personalized treatment, medical imaging, and robotic surgery. In 2021, about 90% of hospitals in the United States had some sort of AI strategy in place, with 75% of healthcare executives believing that AI initiatives are crucial for hospital success. However, out of that 90%, only 33% of hospitals had systems past the pilot stage in place due to lack of infrastructure or financial means. Lack of access to certain AI technologies, such as robotic surgical systems, may be detrimental to hospitals and patients seeking treatment. Automated medical imaging, a type of AI technology, is becoming more common since it is cost-effective for hospitals. Thus, the focus of this project is to develop a ML model for automated imaging of the heart. However, access to robotic surgery, another type of AI technology, varies greatly between hospitals. Therefore, the disparity in access to robotic surgery technology between demographic groups in the United States will be analyzed to discuss the need for stronger robotic surgery initiatives in hospitals.

Three-dimensional (3D) imaging, such as micro-computed tomography (micro-CT), is increasingly being used by organismal biologists for precise and comprehensive anatomical characterization. However, the segmentation of anatomical structures remains a bottleneck in research, often requiring tedious manual work. Here, we propose a pipeline for the fully-automated segmentation of anatomical structures in micro-CT images utilizing state-of-the-art deep learning methods, selecting the ant brain as a test case. We implemented the U-Net architecture for 2D image segmentation for our convolutional neural network (CNN), combined with pixel-island detection. For training and validation of the network, we assembled a dataset of semi-manually segmented brain images of 94 ant species. The trained network predicted the brain area in ant images fast and accurately; its performance tested on validation sets showed good agreement between the prediction and the target, scoring 80% Intersection over Union (IoU) and 90% Dice Coefficient (F1) accuracy. While manual segmentation usually takes many hours for each brain, the trained network takes only a few minutes. Furthermore, our network is generalizable for segmenting the whole neural system in full-body scans, and works in tests on distantly related and morphologically divergent insects (e.g., fruit flies). The latter suggest that methods like the one presented here generally apply across diverse taxa. Our method makes the construction of segmented maps and the morphological quantification of different species more efficient and scalable to large datasets, a step toward a big data approach to organismal anatomy.

This paper presents a survey of image processing techniques proposed in the literature for extracting key cereal crop growth metrics from high spatial resolution, typically proximal images. The descriptive crop growth metrics considered are: crop canopy cover, above ground biomass, leaf area index (including green area index), chlorophyll content, and growth stage. The paper includes an overview of relevant fundamental image processing techniques including camera types, colour spaces, colour indexes, and image segmentation. The descriptive crop growth metrics are defined. Reference methods for ground-truth measurement are described. Image processing methods for metric estimation are described in detail. The performance of the methods is reviewed and compared. The survey reveals limitations in image processing techniques for cereal crop monitoring such as lack of robustness to lighting conditions, camera position, and self-obstruction. Directions for future research to improve performance are identified.

Optical Character Recognition OCR is an essential part of many real-world applications such as digital archiving, automatic number plate recognition, handle cheques, etc. However, developing an OCR for printed Arabic text is still a challenging and open research field due to the special characteristics of Arabic cursive script. In this paper, we propose a segmentation-based, omnifont, open-vocabulary OCR for printed Arabic text. The proposed approach doesn’t require an explicit font type recognition stage. It uses an explicit, indirect character segmentation method. The presented segmentation method is baseline dependent and employs a hybrid, three-steps character segmentation algorithm to handle the problem of character overlapping. Besides, it uses a set of topological features that are designed and generalized to make the segmentation approach font independent. The segmented characters are fed as an input to a convolutional neural network for feature extraction and recognition. The APTID-MF data set has been used for testing and evaluation. The average accuracy of the proposed segmentation stage is 95%, while the average accuracy of the recognition stage is 99.97%. The whole approach achieves an average accuracy of 95% without using font-type recognition or any post-processing techniques.

A proactive road maintenance system enables agencies to better allocate resources to manage their road networks. An inventory of the roads’ conditions is an essential component of such a maintenance program. This research project proposes a hybrid system to assess the condition of the asphalt roads, which uses a dashboard-mounted smartphone to simultaneously collect the acceleration response of a vehicle and the video footage of the road surface while driving. The system analyzes acceleration data for anomalous events that could indicate a defect. Then the computer vision module of the system applies semantic segmentation in the corresponding frame to the detected anomaly to identify defects. This system demonstrated 84% recall and 88% precision rates in detection of anomalies in two road segments. Despite these promising results, the system can only detect the defects that are passed over and it could miss some defects with small acceleration responses, such as traverse cracks.

Approximate multipliers are used in error-tolerant applications, sacrificing the accuracy of results to minimize power or delay. In this paper we investigate approximate multipliers using static segmentation. In these circuits a set of m contiguous bits (a segment of m bits) is extracted from each of the two n-bits operand, the two segments are in input to a small mx m internal multiplier whose output is suitably shifted to obtain the result. We investigate both signed and unsigned multipliers, and for the latter we propose a new segmentation approach. We also present simple and effective correction techniques that can significantly reduce the approximation error with reduced hardware costs. We perform a detailed comparison with previously proposed approximate multipliers, considering a hardware implementation in 28 nm technology. The comparison shows that static segmented multipliers with the proposed correction technique have the desirable characteristic of being on (or close to) the Pareto-optimal frontier for both power vs normalized mean error distance and power vs mean relative error distance trade-off plots. These multipliers, therefore, are promising candidates for applications where their error performance is acceptable. This is confirmed by the results obtained for image processing and image classification applications.

Self-supervised representation learning for videos has been very attractive recently because these methods exploit the information inherently obtained from the video itself instead of annotated labels that is quite time-consuming. However, existing methods ignore the importance of global observation while performing spatio-temporal transformation perception, which highly limits the expression capabilities of the video representation. This paper proposes a novel pretext task that combines the temporal information perception of the video with the motion amplitude perception of moving objects to learn the spatio-temporal representation of the video. Specifically, given a video clip containing several video segments, each video segment is sampled by different sampling rates and the order of video segments is disrupted. Then, the network is used to regress the sampling rate of each video segment and classify the order of input video segments. In the pre-training stage, the network can learn rich spatio-temporal semantic information where content-related contrastive learning is introduced to make the learned video representation more discriminative. To alleviate the appearance dependency caused by contrastive learning, we design a novel and robust vector similarity measurement approach, which can take feature alignment into consideration. Moreover, a view synthesis framework is proposed to further improve the performance of contrastive learning by automatically generating reasonable transformed views. We conduct benchmark experiments with several 3D backbone networks on two datasets. The results show that our proposed method outperforms the existing state-of-the-art methods across the three backbones on two downstream tasks of human action recognition and video retrieval.

Due to the lack of properly annotated medical data, exploring the generalization capability of the deep model is becoming a public concern. Zero-shot learning (ZSL) has emerged in recent years to equip the deep model with the ability to recognize unseen classes. However, existing studies mainly focus on natural images, which utilize linguistic models to extract auxiliary information for ZSL. It is impractical to apply the natural image ZSL solutions directly to medical images, since the medical terminology is very domain-specific, and it is not easy to acquire linguistic models for the medical terminology. In this work, we propose a new paradigm of ZSL specifically for medical images utilizing cross-modality information. We make three main contributions with the proposed paradigm. First, we extract the prior knowledge about the segmentation targets, called relation prototypes, from the prior model and then propose a cross-modality adaptation module to inherit the prototypes to the zero-shot model. Second, we propose a relation prototype awareness module to make the zero-shot model aware of information contained in the prototypes. Last but not least, we develop an inheritance attention module to recalibrate the relation prototypes to enhance the inheritance process. The proposed framework is evaluated on two public cross-modality datasets including a cardiac dataset and an abdominal dataset. Extensive experiments show that the proposed framework significantly outperforms the state of the arts., IEEE TMI

Fish classification (FC) is an expansively studied problem in the domains of image segmentation, pattern recognition, and information retrieval. It has been applied in a countless number of domains including target marketing. Meanwhile, governments are obliged to maintain the fish supply and balance between the ecosystem, commercial, agriculture field, marine scientists, and industrial arena of fish including the nutrition and canning factories. The various FC techniques performance is compared relying on the availability of preprocessing and feature extraction methods, the number of extracted features and classification accuracy, the number of fish families/species recognized. This survey also reviewed the use of Databases such as Fish4-Knowledge (F4K), knowledge database, and Global Information System (GIS) on Fishes and other FC databases. The study on preprocessing methods features extraction techniques and classifiers are gathered from recent works to enhance the understanding of the characteristics of preprocessing methods, features extraction techniques, and classifiers to guide future research directions and compensate for current research gaps.

Deep neural networks (DNNs) technique has achieved impressive performance on semantic segmentation,while its training process requires a large amount of pixel-wise labelled data. Domain adaptation,as a promising solution,can break the restriction by training the model on synthetic data,and generalising it in real-world data. However,there is still a lack of attention paid to the imbalance problems on semantic segmentation adaptation,including the imbalance problem between i) source and target data,ii) different classes. To solve these problems,a progressive hierarchical feature alignment method is proposed in this paper. To alleviate the data imbalance problem,the network is progressively trained by the data from multisource domains,so as to obtain domain-invariant features. To address the class imbalance problem,the features are aligned hierarchically across domains. According to the experimental results,our method shows the competitive adapted segmentation performance on three benchmark datasets.

Accurate and robust segmentation of lung cancers from CT, even those located close to mediastinum, is needed to more accurately plan and deliver radiotherapy and to measure treatment response. Therefore, we developed a new cross-modality educed distillation (CMEDL) approach, using unpaired CT and MRI scans, whereby an informative teacher MRI network guides a student CT network to extract features that signal the difference between foreground and background. Our contribution eliminates two requirements of distillation methods: (i) paired image sets by using an image to image (I2I) translation and (ii) pre-training of the teacher network with a large training set by using concurrent training of all networks. Our framework uses an end-to-end trained unpaired I2I translation, teacher, and student segmentation networks. Architectural flexibility of our framework is demonstrated using 3 segmentation and 2 I2I networks. Networks were trained with 377 CT and 82 T2w MRI from different sets of patients, with independent validation (N=209 tumors) and testing (N=609 tumors) datasets. Network design, methods to combine MRI with CT information, distillation learning under informative (MRI to CT), weak (CT to MRI) and equal teacher (MRI to MRI), and ablation tests were performed. Accuracy was measured using Dice similarity (DSC), surface Dice (sDSC), and Hausdorff distance at the 95$^{th}$ percentile (HD95). The CMEDL approach was significantly (p $, Comment: This manuscript is has been accepted by IEEE Transactions on Medical Imaging

Real-time semantic segmentation is in intense demand for the application of autonomous driving. Most of the semantic segmentation models tend to use large feature maps and complex structures to enhance the representation power for high accuracy. However, these inefficient designs increase the amount of computational costs, which hinders the model to be applied on autonomous driving. In this paper, we propose a lightweight real-time segmentation model, named Parallel Complement Network(PCNet), to address the challenging task with fewer parameters. A Parallel Complement layer is introduced to generate complementary features with a large receptive field. It provides the ability to overcome the problem of similar feature encoding among different classes, and further produces discriminative representations. With the inverted residual structure, we design a Parallel Complement block to construct the proposed PCNet. Extensive experiments are carried out on challenging road scene datasets, i.e., CityScapes and CamVid, to make comparison against several state-of-the-art real-time segmentation models. The results show that our model has promising performance. Specifically, PCNet* achieves 72.9% Mean IoU on CityScapes using only 1.5M parameters and reaches 79.1 FPS with 1024×2048resolution images on GTX 2080Ti. Moreover, our proposed system achieves the best accuracy when being trained from scratch.

Tree height is a parameter useful for calculating above-ground forest biomass and is mostly measured traditionally by ground survey. On the other hand, measuring the forest tree height and biomass estimation through field survey is labor-intensive and time-consuming. The application of remote sensing for forest above-ground biomass (AGB) estimation without forest destruction is important in order to estimate the carbon sequestration potential of the forest. The unmanned aerial vehicle (UAV) is an elating technology, which can help to estimate tree height and it is evolving at a rapid speed. Moreover, assessing the relationship between estimated and measured tree height is necessary for the future application of estimated tree height on AGB estimation. However, tree height estimation from photogrammetric UAV imagery in the dry Afromontane Forest and its sensitivity to AGB estimation are not investigated. Thus, this study aimed to assess the accuracy of tree height estimated from photogrammetric UAV imagery and the sensitivity of the estimated tree height on AGB estimation. Photogrammetric UAV acquired images and sample trees height measured on the ground were collected in Desa’a dry Afromontane Forest, Northern Ethiopia. Tree height was estimated from photogrammetric UAV acquired images and compared with tree heights measured on the ground. Moreover, the sensitivity of the estimated tree height on AGB estimation was investigated. The estimated tree height explained 89% of the tree height measured in the field. A considerable difference between estimated and measured tree height has an insignificant effect on AGB estimation. Thus, in the dry land Afromontane Forest the application of UAV aerial imagery for tree height estimation is promising to estimate AGB.

This work tackles scene understanding for outdoor robotic navigation, solely relying on images captured by an on-board camera. Conventional visual scene understanding interprets the environment based on specific descriptive categories. However, such a representation is not directly interpretable for decision-making and constrains robot operation to a specific domain. Thus, we propose to segment egocentric images directly in terms of how a robot can navigate in them, and tailor the learning problem to an autonomous navigation task. Building around an image segmentation network, we present a generic affordance consisting of 3 driveability levels which can broadly apply to both urban and off-road scenes. By encoding these levels with soft ordinal labels, we incorporate inter-class distances during learning which improves segmentation compared to standard "hard" one-hot labelling. In addition, we propose a navigation-oriented pixel-wise loss weighting method which assigns higher importance to safety-critical areas. We evaluate our approach on large-scale public image segmentation datasets ranging from sunny city streets to snowy forest trails. In a cross-dataset generalization experiment, we show that our affordance learning scheme can be applied across a diverse mix of datasets and improves driveability estimation in unseen environments compared to general-purpose, single-dataset segmentation., Comment: Accepted in Robotics and Automation Letters (RA-L 2022). Supplementary video available at https://youtu.be/q_XfjUDO39Y

Defect detection of trains is of great significance for operation safety and maintenance efficiency for railway maintenance. Nowadays, China railway system utilizes high-speed line scan cameras to capture images of critical parts of moving trains. The visual inspection on the images still heavily relies on manual interpretation. To reduce the labor requirements, we propose a novel two-stage deep learning based framework for component defect detection of moving trains. The proposed framework is composed of two major successive stages: detecting train components by using our proposed hierarchical object detection scheme (HOD), and detecting component defects based on multiple neural networks and image processing methods. Our proposed HOD can effectively detect and localize train components from large to small in a hierarchical way. Furthermore, a gated feature fusion method that can extract and combine the hierarchical contextual features and spatial contexts is also proposed to improve the performance. To the best of our knowledge, it is the first time in the literature that component defect detection of moving trains is systematically analyzed. Extensive experiments on real images from China railway system have demonstrated that our framework outperforms the state-of-the-art baselines significantly.

The application of Artificial Intelligence in dental healthcare has a very promising role due to the abundance of imagery and non-imagery-based clinical data. Expert analysis of dental radiographs can provide crucial information for clinical diagnosis and treatment. In recent years, Convolutional Neural Networks have achieved the highest accuracy in various benchmarks, including analyzing dental X-ray images to improve clinical care quality. The Tufts Dental Database, a new X-ray panoramic radiography image dataset, has been presented in this paper. This dataset consists of 1000 panoramic dental radiography images with expert labeling of abnormalities and teeth. The classification of radiography images was performed based on five different levels: anatomical location, peripheral characteristics, radiodensity, effects on the surrounding structure, and the abnormality category. This first-of-its-kind multimodal dataset also includes the radiologist's expertise captured in the form of eye-tracking and think-aloud protocol. The contributions of this work are 1) publicly available dataset that can help researchers to incorporate human expertise into AI and achieve more robust and accurate abnormality detection; 2) a benchmark performance analysis for various state-of-the-art systems for dental radiograph image enhancement and image segmentation using deep learning; 3) an in-depth review of various panoramic dental image datasets, along with segmentation and detection systems. The release of this dataset aims to propel the development of AI-powered automated abnormality detection and classification in dental panoramic radiographs, enhance tooth segmentation algorithms, and the ability to distill the radiologist's expertise into AI.

State-of-the-art stixel methods fuse dense stereo disparity and semantic class information, e.g. from a Convolutional Neural Network (CNN), into a compact representation of driveable space, obstacles and background. However, they do not explicitly differentiate instances within the same semantic class. We investigate several ways to augment single-frame stixels with instance information, which can be extracted by a CNN from the RGB image input. As a result, our novel Instance Stixels method efficiently computes stixels that account for boundaries of individual objects, and represents instances as grouped stixels that express connectivity. Experiments on the Cityscapes dataset demonstrate that including instance information into the stixel computation itself, rather than as a post-processing step, increases the segmentation performance (i.e. Intersection over Union and Average Precision). This holds especially for overlapping objects of the same class. Furthermore, we show the superiority of our approach in terms of segmentation performance and computational efficiency compared to combining the separate outputs of Semantic Stixels and a state-of-the-art pixel-level CNN. We achieve processing throughput of 28 frames per second on average for 8 pixel wide stixels on images from the Cityscapes dataset at 1792x784 pixels. Our Instance Stixels software is made freely available for non-commercial research purposes.