Search

This document is a guest editorial from the journal IET Computer Vision, discussing the topic of advanced image restoration and enhancement. The editorial highlights the challenges faced in this field, such as the complexity of degradation models for real-world low-quality images and the difficulty of acquiring paired data. It also introduces a special issue of the journal that includes five accepted papers, which focus on video reconstruction and image super-resolution. The editorial concludes by providing brief summaries of each accepted paper. The guest editors of the special issue are also mentioned, along with their research interests and affiliations. [Extracted from the article]

Artistic style transfer involves the adaption of an input image to reflect the style of a reference image while maintaining its original content. This technique, now a prominent focus due to its prospective use in creative fields like digital art and graphic design, typically applies normalization techniques and attention mechanisms. While these methods yield decent results, they often fall short due to distortion of content image details and non‐artefact styles. In this paper, we introduce a novel approach that synergizes adaptive instance normalization (AdaIN), attention mechanisms, knowledge distillation (KD) and strategically placed internal layers, and new enhancements designed to preserve content details and provide a nuanced control over the style transfer process. We introduce a Detail Enhancement Module to amplify high‐frequency details in the content image, enhancing edge and texture preservation. A Multi‐scale Strategy is implemented to ensure uniform style application across various detail levels, leading to more coherent stylization. The Content Feature Refinement process refines content features, sharpening and emphasizing details to preserve structural and textural integrity. AdaIN's distinctive feature of efficiently collecting style data is exploited in our approach, coupled with attention mechanisms' inherent ability to conserve content information. We supplement this blend with KD for the enhancement of model accuracy and efficiency. Additionally, the introduction of internal layers acts as a conduit to further improve the style transfer process, increasing the transfer level of features and fostering better stylized results. The cornerstone of our technique lies in preserving the content structure amidst complex style transfers. Experimental results affirm the superior performance of our method over existing techniques in both quantitative and qualitative evaluations. [ABSTRACT FROM AUTHOR]

Whole brain segmentation from magnetic resonance images (MRI) is crucial in diagnosing brain diseases and analyzing neuroimaging data. Despite advances through deep learning, challenges such as uneven gray distribution and the presence of artifacts still present hurdles in medical image processing. These limitations are often a result of insufficient spatial contextual information and lack of attention to important regions within existing models. To address these issues, this paper presents SSA‐UNet (Squeeze‐and‐Excitation and Self‐Attention UNet), a uniquely designed deep convolutional neural network that integrates spatial constraints by converting three consecutive 2D MRI slices into a single 2.5D image. This facilitates capturing inter‐slice dependencies effectively. Additionally, the newly formulated SSA block, which sequentially incorporates channel attention and Self‐Attention mechanisms, is placed before the decoders in the conventional U‐Net architecture. This enables the network to automatically weight different feature maps and focus more effectively on regions requiring precise segmentation. Rigorous evaluations on LPBA40 and IBSR18 datasets substantiate the remarkable improvements in accuracy and stability achieved by SSA‐UNet. Results indicate Dice coefficients of 98.38% and 97.47%, specificity of 99.69% and 99.57%, and sensitivity of 98.5% and 97.98% for the respective datasets. Compared to other existing models, SSA‐UNet shows significant improvements on both the LPBA40 and IBSR18 datasets. On the LPBA40 dataset, SSA‐UNet's Dice coefficient improved by 0.33% compared to the sub‐optimal model, while on the IBSR18 dataset, the improvement reached 1.78%. These empirical findings demonstrate SSA‐UNet's heightened capability in addressing the long‐standing challenges in MRI‐based whole‐brain segmentation. [ABSTRACT FROM AUTHOR]

In the field of scientific visualization, the upscaling of time‐varying volume is meaningful. It can be used in in situ visualization to help scientists overcome the limitations of I/O speed and storage capacity when analysing and visualizing large‐scale, time‐varying simulation data. This paper proposes self‐attention residual network‐based spatial super‐resolution (SARN‐SSR), a spatial super‐resolution model based on self‐attention residual networks that can generate time‐varying data with temporal coherence. SARN‐SSR consists of two components: a generator and a discriminator. The generator takes the low‐resolution volume sequences as the input and gives the corresponding high‐resolution volume sequences as the output. The discriminator takes both synthesized and real high‐resolution volume sequence as the input and gives a matrix to predict the realness as the output. To verify the validity of SARN‐SSR, four sets of time‐varying volume datasets are applied from scientific simulation. In addition, SARN‐SSR is compared on these datasets, both qualitatively and quantitatively, with two deep learning‐based techniques and one traditional technique. The experimental results show that by using this method, the closest time‐varying data to the ground truth can be obtained. [ABSTRACT FROM AUTHOR]

Pixel‐level labeling of regions of interest in an image is a key step in building a labeled training dataset for supervised deep learning networks of images. However, traditional manual labeling of cancerous regions in digital pathological images by doctors is time‐consuming and inefficient. To address this issue, this paper proposes an automatic labeling method for whole slide images, which combines clustering and edge detection techniques. The proposed method utilizes the multi‐level feature fusion model and the Long‐Short Term Memory network to discriminate the cancerous nature of the whole slide images, thereby improving the classification accuracy of the whole slide images. Subsequently, the automatic labeling of cancerous regions is achieved by integrating a density‐based clustering algorithm and an edge point extraction algorithm, both based on the discriminated results of the cancerous properties of whole slide images. The experimental results demonstrate the effectiveness of the proposed method, which offers an efficient and accurate solution to the challenging task of cancerous region labeling in digital pathological images. [ABSTRACT FROM AUTHOR]

In recent years, the task of maritime emergency rescue has increased, while the cost of time for traditional methods of search and rescue is pretty long with poor effect subject to the constraints of the complex circumstances around the sea, the effective conditions, and the support capability. This paper applies deep learning and proposes a YOLOv5s‐SwinDS algorithm for target detection in distress at sea. Firstly, the backbone network of the YOLOv5s algorithm is replaced by swin transformer, and a multi‐level feature fusion module is introduced to enhance the feature expression ability for maritime targets. Secondly, deformable convolutional networks v2 (DCNv2) is used instead of traditional convolution to improve the recognition capability for irregular targets when the neck network features are output. Finally, the CIoU loss function is replaced with SIoU to reduce the redundant box effectively while accelerating the convergence and regression of the predicted box. Experimenting on the publicly dataset SeaDronesSee, the Precision$Precision$, Recall$Recall$, mAP0.5$mAP_{0.5}$ and mAP0.5−0.95$mAP_{0.5-0.95}$ of YOLOv5s‐SwinDS model are 87.9%, 75.8%, 79.1% and 42.9%, respectively, which get higher results than the original YOLOv5s model, the YOLOv7 series of models, and the YOLOv8 series of models. The experiments verifies that the algorithm has good performance in detecting maritime distress targets. [ABSTRACT FROM AUTHOR]

Image inpainting networks based on deep learning techniques have been widely used in many important fields. However, most inpainting networks fail to generate desirable repaired images. This may be due to their failure to extract effective features and accurately assign high weights to the undamaged regions. To alleviate these problems, an image inpainting network based on gated convolution and multi‐level attention mechanism (IIN‐GCMAM) is proposed in this paper. This network follows encoder–decoder architecture, consisting of the gated convolution encoder (GC‐encoder) and the multi‐level attention mechanism decoder (MAM‐decoder). The GC‐encoder weighs the extracted features with gated convolutions, which reduces the interference caused by the damaged regions. The multi‐level attention mechanism employed in the MAM‐decoder uses multi‐scale feature maps spatially and channel‐wise to improve the consistency in global structure and the fineness of repaired results. Extensive experiments are conducted on the common datasets, Paris StreetView and CelebA. Experimental results indicate that the proposed IIN‐GCMAM can achieve a good performance on the common evaluation metrics and visual effects. It can achieve 0.0408, 0.720, and 22.27 in MAE, SSIM, and PSNR at the mask ratio of 50%–60%, respectively. [ABSTRACT FROM AUTHOR]

License plate detection is an important task in Intelligent Transportation Systems (ITS) and has a wide range of applications in vehicle management, traffic control, and public safety. In order to improve the accuracy and speed of mobile recognition, an improved lightweight YOLOv5s model is proposed for license plate detection. First, an improved Stemblock network is used to replace the original Focus layer in the network, which ensures strong feature expression capability and reduces a large number of parameters to lower the computational complexity; then, an improved lightweight network, ShuffleNetv2, is used to replace the backbone network of the YOLOv5s, which makes the model lighter and ensures the detection accuracy at the same time. Then, a feature enhancement module is designed to reduce the information loss caused by the rearrangement of the backbone network channels, which facilitates the information interaction in the feature fusion process; finally, the low‐, medium‐ and high‐level features in the Shufflenetv2 network structure are fused to form the final high‐level output features. Experimental results on the CCPD dataset show that compared to other methods this paper obtains better performance and faster speed in the license plate detection task, in which the average precision mean value reaches 96.6%, and can achieve a detection speed of 43.86 frame/s, and the parameter volume is reduced to 5.07 M. [ABSTRACT FROM AUTHOR]

Power line insulator defect identification usually suffers from complex backgrounds, small defect target sizes, and inconspicuous defect features. Traditional identification methods based on image processing, image analysis, and pattern classification have many limitations in solving the aforementioned problems. In recent decades, deep learning classification methods have gradually replaced traditional identification methods in the task of power line insulator defect identification. To accurately identify the locations of insulator defects, this paper proposes an insulator defect detection algorithm using an improved lightweight YOLOv4‐tiny network (ILYTN). First, the CBL (Conv‐BN‐LeakyReLU) modules of the backbone network are replaced with MobileViT blocks to enhance the feature extraction capability of the backbone network. Second, coordinate attention (CA) is introduced in the feature fusion part to improve the network's ability to focus on the location of defects. Finally, an EIOU (efficient intersection over union) loss function, instead of the original CIOU loss function, is used so that the convergence speed of the network can be improved. To verify the effectiveness of the proposed algorithm in this paper, the mViT‐yolo algorithm is compared with the mainstream Faster‐RCNN algorithm, SSD algorithm, YOLOv3 algorithm, and YOLOv4‐tiny algorithm. The experimental results show that the algorithm proposed in this paper outperforms all the above algorithms in terms of detection accuracy. Compared with the traditional YOLOv4‐tiny algorithm, the proposed algorithm increases the mean average precision (mAP) by 1.64%, the average precision (AP) of missing insulator defects by 0.32%, and the average precision (AP) of broken insulator defects by 4.96%. © 2023 The Authors. IEEJ Transactions on Electrical and Electronic Engineering published by Institute of Electrical Engineer of Japan and Wiley Periodicals LLC. [ABSTRACT FROM AUTHOR]

With rapid developments in convolutional neural networks for image processing, deep learning methods based on pixel classification have been extensively applied in medical image segmentation. One popular strategy for such tasks is the encoder‐decoder‐based U‐Net architecture and its variants. Most segmentation methods based on fully convolutional networks will cause the loss of spatial and contextual information due to continuous pooling operations or strided convolution when decreasing image resolution, and make less use of contextual information and global information under different receptive fields. To overcome this shortcoming, this paper proposes a novel structure called RAAU‐Net. In our proposed RAAU‐Net structure, which is a modified U‐shaped architecture, we aim to capture high‐level information while preserving spatial information and focusing on the regions of interest. RAAU‐Net comprises three main components: a feature encoder module that utilizes a pre‐trained ResNet‐18 model as a fixed feature extractor, a multi‐receptive field extraction module that we developed, and a feature decoder module. We have tested our method on several 2D medical image segmentation tasks such as retinal nerve, breast tumor, skin lesion, lung, gland, and polyp segmentation. All the indexes of the model reached the best in the dataset of skin lesions, in which Accuracy, Precision, IoU, Recall, and Dice Score were 3.26%, 5.42%, 9.92%, 6.52%, and 5.95% higher than UNet. [ABSTRACT FROM AUTHOR]

Background: Radiation dose reduction has been the focus of many research activities in x‐ray CT. Various approaches were taken to minimize the dose to patients, ranging from the optimization of clinical protocols, refinement of the scanner hardware design, and development of advanced reconstruction algorithms. Although significant progress has been made, more advancements in this area are needed to minimize the radiation risks to patients. Purpose: Reconstruction algorithm‐based dose reduction approaches focus mainly on the suppression of noise in the reconstructed images while preserving detailed anatomical structures. Such an approach effectively produces synthesized high‐dose images (SHD) from the data acquired with low‐dose scans. A representative example is the model‐based iterative reconstruction (MBIR). Despite its widespread deployment, its full adoption in a clinical environment is often limited by an undesirable image texture. Recent studies have shown that deep learning image reconstruction (DLIR) can overcome this shortcoming. However, the limited availability of high‐quality clinical images for training and validation is often the bottleneck for its development. In this paper, we propose a novel approach to generate SHD with existing low‐dose clinical datasets that overcomes both the noise texture issue and the data availability issue. Methods: Our approach is based on the observation that noise in the image can be effectively reduced by performing image processing orthogonal to the imaging plane. This process essentially creates an equivalent thick‐slice image (TSI), and the characteristics of TSI depend on the nature of the image processing. An advantage of this approach is its potential to reduce impact on the noise texture. The resulting image, however, is likely corrupted by the anatomical structural degradation due to partial volume effects. Careful examination has shown that the differential signal between the original and the processed image contains sufficient information to identify regions where anatomical structures are modified. The differential signal, unfortunately, contains significant noise and has to be removed. The noise removal can be accomplished by performing iterative noise reduction to preserve structural information. The processed differential signal is subsequently subtracted from TSI to arrive at SHD. Results: The algorithm was evaluated extensively with phantom and clinical datasets. For better visual inspection, difference images between the original and SHD were generated and carefully examined. Negligible residual structure could be observed. In addition to the qualitative inspection, quantitative analyses were performed on clinical images in terms of the CT number consistency and the noise reduction characteristics. Results indicate that no CT number bias is introduced by the proposed algorithm. In addition, noise reduction capability is consistent across different patient anatomical regions. Further, simulated water phantom scans were utilized in the generation of the noise power spectrum (NPS) to demonstrate the preservation of the noise‐texture. Conclusions: We present a method to generate SHD datasets from regularly acquired low‐dose CT scans. Images produced with the proposed approach exhibit excellent noise‐reduction with the desired noise‐texture. Extensive clinical and phantom studies have demonstrated the efficacy and robustness of our approach. Potential limitations of the current implementation are discussed and further research topics are outlined. [ABSTRACT FROM AUTHOR]

Accurate fish size measurement in breeding areas is crucial for the fishing industry. Unlike acoustic methods with high equipment cost and low measurement accuracy, current image‐based methods offer a promising alternative. However, these image‐based methods still face challenges in selecting measurement points. To address this issue and achieve precise measurements of individual fish, this paper introduces an automatic fish size measurement method based on key point detection. We established a Fish‐Keypoints dataset and utilized deep learning techniques for the detection of fish and their key points. Using a binocular camera system, we reconstruct a three‐dimensional coordinate system to measure key points at the fish's head and tail, facilitating fish length calculation. The detection model achieves an accuracy of 85.1% in key point detection. The proposed method is tested in both land and underwater environments, demonstrating a relative measurement error of approximately 7% for fish in pools. This confirms the proposed method's ability to accurately detect measurement points, offering superior accuracy compared to other methods. [ABSTRACT FROM AUTHOR]

The existing monocular depth estimation methods based on deep learning have difficulty in estimating the depth near the edges of the objects in an image when the depth distance between these objects changes abruptly and decline in accuracy when an image has more noises. Furthermore, these methods consume more hardware resources because they have huge network parameters. To solve these problems, this paper proposes a depth estimation method based on weighted fusion and point‐wise convolution. The authors design a maximum‐average adaptive pooling weighted fusion module (MAWF) that fuses global features and local features and a continuous point‐wise convolution module for processing the fused features derived from the (MAWF) module. The two modules work closely together for three times to perform weighted fusion and point‐wise convolution of features of multi‐scale from the encoder output, which can better decode the depth information of a scene. Experimental results show that our method achieves state‐of‐the‐art performance on the KITTI dataset with δ1 up to 0.996 and the root mean square error metric down to 8% and has demonstrated the strong generalisation and robustness. [ABSTRACT FROM AUTHOR]

COVID‐19 has had a profound global impact, necessitating the development of infection detection systems based on machine learning. This paper presents a Multi‐task architecture that addresses the classification and segmentation tasks for COVID‐19 detection. The model comprises an encoder for feature representation, a decoder for segmentation, and a multi‐layer perceptron for classification. Evaluations conducted on two datasets demonstrate the model's performance in both classification and segmentation. To enhance efficiency and diagnosis accuracy, CT‐scan images undergo pre‐processing using image processing algorithms like histogram equalization, median filtering, and mathematical morphology operations. The combination of the median filter pre‐processing and the proposed model yields impressive results in the classification task, achieving high accuracy, sensitivity, and specificity, with values of 0.97, 0.97, and 0.96, respectively, for dataset 1, and 0.96 in mentioned metrics for dataset 2. For segmentation, the proposed model, particularly with the average morphology pre‐processing, exhibits excellent performance with high accuracy, low mean squared error, high peak signal‐to‐noise ratio, high structural similarity index, and a mean dice coefficient of 88.86 ± 0.05 for dataset 1, and 87.97 ± 0.02 for dataset 2. Furthermore, the pre‐trained models consistently demonstrate the superiority of the median filter and proposed model in the classification task on the same datasets. In conclusion, the proposed multi‐task model, incorporating image processing techniques, achieves remarkable results in both classification and segmentation. The utilization of pre‐processing algorithms and the multi‐task framework significantly contribute to superior performance metrics. This study encourages further exploration of combining diverse image processing algorithms to advance infection diagnosis and treatment. [ABSTRACT FROM AUTHOR]

The worldwide spread of the coronavirus (COVID‐19) outbreak has proven devastating to public health. The severity of pneumonia relies on a rapid and accurate diagnosis of COVID‐19 in CT images. Accordingly, a computed tomography (CT) scan is an excellent screening tool for detecting COVID‐19. This paper proposes a deep learning‐based strategy for recognizing and segmenting a COVID‐19 lesion from chest CT images, which would introduce an accurate computer aided decision criteria for the physicians about the severity rate of the patients. Two main stages have been proposed for detecting COVID‐19; first, a convolutional neural network (CNN) deep structure recognizes and classifies COVID‐19 from CT images. Second, a U‐Net deep structure segments the COVID‐19 regions in a semantic manner. The proposed system is trained and evaluated on three different CT datasets for COVID‐19, two of which are used to illustrate the system's segmentation performance and the other is to demonstrate the system's classification ability. Experiment results reveal that the proposed CNN can achieve classification accuracy greater than 0.99, and the proposed U‐Net model outperforms the state‐of‐the‐art in segmentation with an IOU greater than 0.92. [ABSTRACT FROM AUTHOR]

Overhead transmission line detection based on deep learning of aerial images taken by UAVs has been widely investigated. Despite its success, it is limited by several factors, including inappropriate evaluation criteria and dramatic scaling of components in the images. To mitigate these issues, a relative mean Average Precision evaluation index is proposed to accurately measure the model's detection performance for smaller objects. A data enhancement strategy including multi‐scale transformation is adopted to alleviate the problem of drastic scaling. The existing Cascade RCNN target detection technology is enhanced by incorporating Swin‐v2 and a balanced feature pyramid to improve feature characterization capabilities, while side‐aware boundary localization is utilized to improve the positioning accuracy of the model. Experimental results demonstrate that the proposed method outperforms state‐of‐the‐art methods on CPLID and achieves 7.8%, 11.8%, and 5.5% higher detection accuracy than the baseline for mAP50, relative small and medium mAP, respectively. Additionally, the paper discusses the influence of adopted data enhancement on the robustness of the model. [ABSTRACT FROM AUTHOR]

In recent years, meaningful visual image encryption schemes that the plain image is compressed and encrypted and then hidden into the carrier image have received increasing attention. This paper proposes a new meaningful visual image encryption scheme, which consists of three stages: compression (compression network)—encryption (2D‐SLC hyperchaotic map)—hiding (matrix encoding). First, the advantages of deep learning are explored. It can compress the width, height, channel, and pixel values of the plain image simultaneously. Second, a new 2D‐SLC hyperchaotic map is designed to ensure security. It has a larger chaotic space and better randomness. Finally, to obtain a high‐quality cipher image, the secure secret image is hidden in the grey carrier image by matrix encoding. The scheme can compress and encrypt the grey or colour plain image and then hide it in a grey carrier image. In addition, the theoretical peak signal‐to‐noise ratio (PSNR) between the cipher image and the carrier image is improved from 40.9292 to 42.1785 dB. The total running time is only about 0.35, 0.87 and 3.1 s for a 256 × 256, 512 × 512 and 1024 × 1024 grey or colour plain image, respectively. [ABSTRACT FROM AUTHOR]

With the improvement of smart grid, utilizing unmanned aerial vehicles (UAV) to detect the operation status of insulators has attracted widespread attention. The insulator defects can lead to serious power loss, damage the service life of power lines, and even result in power outages in serious cases. The small‐scale object, complex background, and limited‐number collected data make insulator defect still a challenging problem. Benefitted by the advances in deep learning, deep learning‐based insulator defects have achieved great progress in recent years. In the paper, the authors present a novel systematic survey of these advances, where further analysis about different processing stages methods is introduced: (i) insulator processing stage methods exploit the specific image pre‐processing algorithm for data augmentation and low‐level vision information extraction; (ii) defect detection stage model can locate and classify diagnosis fault with different task targets, like sequential task strategy and multi‐task strategy. In addition, the authors also review publicly available benchmark and datasets. The future research direction and open problem are discussed to promote the development of the community. [ABSTRACT FROM AUTHOR]

The progress of deep learning architectures, machine learning models and pathology slide digitization is an encouraging step toward meeting the growing demand for more precise classification and prediction diagnosis for the breast tumours. The BreakHis dataset with four magnification factors (40X, 100X, 200X and 400X), as well as seven deep learning architectures used for feature extraction (DenseNet 201, Inception ResNet V2, Inception V3, ResNet 50, MobileNet V2,VGG16 and VGG19), four machine learning models for classification (MLP, SVM, DT, and KNN), and two combination rules (hard and weighted voting) were investigated in this paper to design and evaluate a new proposed approach consisting of building deep hybrid homogenous ensemble. Additionally, the best proposed models were compared to deep stacked, deep bagging, deep boosting, and deep hybrid heterogenous ensemble to choose the best strategy in building deep ensemble learning techniques. The four performance measures accuracy, precision, recall, and F1‐score were used in the empirical evaluations, as well as 5‐fold cross‐validation, the Scott Knott statistical test, and the Borda Count voting method. The results demonstrated the new approach's potential since it outscored both singles and other deep ensemble learning strategies, achieving accuracy values of 98.3% and 97.7% for the MFs 40X, 100X and 200X, 400X, respectively. The empirical results demonstrated that the proposed ensembles are impactful for histopathological breast cancer images classification, and they provided a promising tool to assist pathologists in the diagnosis of breast cancer. [ABSTRACT FROM AUTHOR]

Spatial resolution is an important indicator that measures the quality of remote sensing images. Image texture has been successfully recovered by generative adversarial networks in deep learning super‐resolution (SR) methods. However, the existing methods are prone to image texture distortion. To solve the above problems, this paper proposes an improved generative adversarial network to enhance the super‐resolution reconstruction effect of medium‐ and low‐resolution (LR) remote sensing images. This network is based on the Super Resolution Generative Adversarial Network (SRGAN), which makes great improvements in the structure of the connection between the inside and outside of the residual block and the design of the model loss function. At the same time, the G1–G2–G3 structure between residuals effectively combines the image information of the three scales of small, medium and large. The model loss function can be designed based on the Charbonnier loss function to narrow the pixel distance between the reconstructed remote sensing image and the original image. Furthermore, targeted perceptual loss can direct the network to restore the texture details of the image according to the semantic category. The subjective and objective evaluation of the generated images and the ablation experiments prove that compared with SRGAN and other networks, our method can generate more realistic and reliable textures. Additionally, the indicators [peak signal‐to‐noise ratio (PSNR), structural similarity (SSIM), multiscale structural similarity (MS‐SSIM)] used to measure the quality of the reconstructed image obtain improved objective quantitative evaluation. [ABSTRACT FROM AUTHOR]

Image denoising is one of the hottest topics in image restoration area, it has achieved great progress both in terms of quantity and quality in recent years, especially after the wide and intensive application of deep neural networks. In many deep learning based image denoising models, the performance can greatly benefit from the prepared clean/noisy image pairs used for model training, however, it also limits the application of these models in real denoising scenes. Therefore, more and more researchers tend to develop models that can be learned without image pairs, namely the denoising models that can be well generalised in real‐world denoising tasks. This motivates to make a survey on the recent development of image denoising methods. In this paper, the typical denoising methods from the perspective of model training are reviewed, the reviewed methods are categorised into four classes: the models need clean/noisy image pairs to train, the models trained on multiple noisy images, the models can be learned from a single noisy image, and the visual transformer based models. The denoising results of different denoisers were compared on some public datasets to discover the performance and advantages. The challenges and future directions in image denoising area are also discussed. [ABSTRACT FROM AUTHOR]

Marine scientists use remote underwater image and video recording to survey fish species in their natural habitats. This helps them get a step closer towards understanding and predicting how fish respond to climate change, habitat degradation and fishing pressure. This information is essential for developing sustainable fisheries for human consumption, and for preserving the environment. However, the enormous volume of collected videos makes extracting useful information a daunting and time‐consuming task for a human being. A promising method to address this problem is the cutting‐edge deep learning (DL) technology. DL can help marine scientists parse large volumes of video promptly and efficiently, unlocking niche information that cannot be obtained using conventional manual monitoring methods. In this paper, we first provide a survey of computer visions (CVs) and DL studies conducted between 2003 and 2021 on fish classification in underwater habitats. We then give an overview of the key concepts of DL, while analysing and synthesizing DL studies. We also discuss the main challenges faced when developing DL for underwater image processing and propose approaches to address them. Finally, we provide insights into the marine habitat monitoring research domain and shed light on what the future of DL for underwater image processing may hold. This paper aims to inform marine scientists who would like to gain a high‐level understanding of essential DL concepts and survey state‐of‐the‐art DL‐based fish classification in their underwater habitat. [ABSTRACT FROM AUTHOR]

Abstract Images taken in low‐light conditions often suffer from reduced contrast, amplified noise, and colour distortions. Such degradation falls short of viewer expectations and negatively impacts high‐level tasks, such as object detection. In this paper, the above problems are addressed by leveraging the properties of the LAB colour space and a multi‐scale feature pyramid. Specifically, the model first generates a light attention map and a noise map, which are proficient at manipulating the luminance channel for brightness recovery while preserving the colour information and separating noise from the image content. Moreover, enriched contextual features is progressively extracted and then the low‐light images are refined by incorporating the light attention map and the noise map as priors into a coarse‐to‐fine architecture. The enhanced results are characterized by improved contrast, vibrant colour, and noise suppression.In this letter, a novel approach to enhance low‐light images by leveraging the LAB colour space and a multi‐scale feature pyramid. The method utilizes a light attention map and a Noise map to guide the enhancement process, resulting in improved contrast, vibrant colour, and noise suppression. By incorporating these priors into a coarse‐to‐fine architecture, superior image enhancement is achieved in low‐light conditions. [ABSTRACT FROM AUTHOR]

Deep learning based image quality assessment models usually learn to predict image quality from a single dataset, leading the model to overfit specific scenes. To account for this, mixed datasets training can be an effective way to enhance the generalization capability of the model. However, it is nontrivial to combine different image quality assessment datasets, as their quality evaluation criteria, score ranges, view conditions, as well as subjects are usually not shared during the image quality annotation. Instead of aligning the annotations, this paper proposes a monotonic neural network for image quality assessment model learning with different datasets combined. In particular, this model consists of a dataset‐shared quality regressor and several dataset‐specific quality transformers. The quality regressor aims to obtain the perceptual quality of each image of each dataset and the quality transformer maps the perceptual quality to the corresponding annotation monotonically. The experimental results verify the effectiveness of the proposed learning strategy and the code is available at https://github.com/fzp0424/MonotonicIQA. [ABSTRACT FROM AUTHOR]

This paper introduce an efficient object detection network named Trident‐You Only Look Once (YOLO), which is designed for mobile devices with limited computing power. The new architecture is improved based on YOLO v4‐tiny. The authors redesign the network structure and propose a trident feature pyramid network (Trident‐FPN), which can improve the precision and recall of lightweight object detection. Specifically, Trident‐FPN increases the computational complexity by only a small amount of floating point operations per second (FLOPs) and obtains a multi‐scale feature map of the model, which significantly lightweight object detection performance. To enlarge the receptive field of the network with the fewest FLOPs, this paper redesign the receptive field block (RFB) and spatial pyramid pooling (SPP) layer and propose tinier cross‐stage partial RFBs and smaller cross‐stage partial SPPs. This paper present extensive experiments, and Trident‐YOLO shows strong performance compared to that of other popular models on the PASCAL VOC and MS COCO. On the MS COCO and PASCAL VOC 2007 test sets, the mean average precision (mAP) of Trident‐YOLO improved by 4.5% and 5.0%, respectively. Trident‐YOLO also reduce the network size by more than 54.4% compared to YOLO v4‐tiny. With a 23.7% FLOP reduction, the FPS is improved by 1.9 on an Nvidia Jetson Xavier NX. [ABSTRACT FROM AUTHOR]

Automated retinal disease detection and grading is one of the most researched areas in medical image analysis. In recent years, Deep Learning models have attracted much attention in this field. Hence, in this paper, we present a Deep Learning‐based, lightweight, fully automated end‐to‐end diagnostic system for the detection of the two major retinal diseases, namely diabetic macular oedema (DME) and drusen macular degeneration (DMD). Early detection of these diseases is important to prevent vision impairment. Optical coherence tomography (OCT) is the main imaging technique for detecting these diseases. The model proposed in this work is based on residual blocks and channel attention modules. The performance of the model is evaluated using the publicly available Mendeley OCT dataset and the Duke dataset. We were able to achieve a classification accuracy of 99.5% in the Mendeley test dataset and 94.9% in the Duke dataset with the proposed model. For the application, we performed an extensive evaluation of pre‐trained models (LeNet, AlexNet, VGG‐16, ResNet50 and SE‐ResNet). The proposed model has a much smaller number of trainable parameters and shows superior performance compared to existing methods. [ABSTRACT FROM AUTHOR]

Background: Auxiliary diagnosis and monitoring of lung diseases based on lung ultrasound (LUS) images is important clinical research. A‐line is one of the most common indicators of LUS that can offer support for the assessment of lung diseases. A traditional A‐line detection method mainly relies on experienced clinicians, which is inefficient and cannot meet the needs of these areas with backward medical level. Therefore, how to realize the automatic detection of A‐line in LUS image is important. Purpose: In order to solve the disadvantages of traditional A‐line detection methods, realize automatic and accurate detection, and provide theoretical support for clinical application, we proposed a novel A‐line detection method for LUS images with different probe types in this paper. Methods: First, the improved Faster R‐CNN model with a selection strategy of localization box was designed to accurately locate the pleural line. Then, the LUS image below the pleural line was segmented for independent analysis excluding the influence of other similar structures. Next, image‐processing methods based on total variation, matched filter, and gray difference were applied to achieve the automatic A‐line detection. Finally, the "depth" index was designed to verify the accuracy by judging whether the automatic measurement results belong to corresponding manual results (±5%). In experiments, 3000 convex array LUS images were used for training and validating the improved pleural line localization model by five‐fold cross validation. 850 convex array LUS images and 1080 linear array LUS images were used for testing the trained pleural line localization model and the proposed image‐processing‐based A‐line detection method. The accuracy analysis, error statistics, and Harsdorff distance were employed to evaluate the experimental results. Results: After 100 epochs, the mean loss value of training and validation set of improved Faster R‐CNN model reached 0.6540 and 0.7882, with the validation accuracy of 98.70%. The trained pleural line localization model was applied in the testing set of convex and linear probes and reached the accuracy of 97.88% and 97.11%, respectively, which were 3.83% and 8.70% higher than the original Faster R‐CNN model. The accuracy, sensitivity, and specificity of A‐line detection reached 95.41%, 0.9244%, 0.9875%, and 94.63%, 0.9230%, and 0.9766% for convex and linear probes, respectively. Compared to the experienced clinicians' results, the mean value and p value of depth error were 1.5342 ± 1.2097 and 0.9021, respectively, and the Harsdorff distance was 5.7305 ± 1.8311. In addition, the accumulated accuracy of the two‐stage experiment (pleural line localization and A‐line detection) was calculated as the final accuracy of the whole A‐line detection system. They were 93.39% and 91.90% for convex and linear probes, respectively, which were higher than these previous methods. Conclusions: The proposed method combining image processing and deep learning can automatically and accurately detect A‐line in LUS images with different probe types, which has important application value for clinical diagnosis. [ABSTRACT FROM AUTHOR]

Summary: Intelligent unmanned aerial vehicles (UAVs) are drawing more and more attention from industry to academia. UAV navigation plays an important role in the cooperative scenario where multiple UAVs are deployed, while image data that capture the information of the UAV area are often used as input for UAV navigation. Deep learning is a common and powerful technique for UAV image processing, but a complex model generated by deep learning technique is hardly suitable for the limited computing capacity of edge computing devices such as UAVs. Therefore, this paper designs an efficient deep learning model on UAVs to fit the restriction of low computational powers and low power consumption. Traditional UAV object detection methods mostly use static images as the basis for object recognition, or collect images for offline detection. Our method combines the existing fast single‐frame detection methods with the spatial‐temporal relationship of video sequences, to build an efficient end‐to‐end model. In addition, the convolutional LSTM module is used to propagate the temporal context of the video frame sequences. Based on the temporal context, we propose a module for calculating spatial correlation. At the same time, we establish our experimental dataset in our real application and conduct the experiment, which shows that the proposed method reduces the size of models and meanwhile maintains the detection rate. Compared with the existing static images approaches, our method is faster and more accurate. Inference speeds of nearly 20fps can be achieved while performing real‐time tasks. [ABSTRACT FROM AUTHOR]

Summary: High latency and power consumption are two major problems that need to be addressed in convolutional neural networks (CNN). In this paper, the convolutional layer is replaced with a discrete‐time cellular neural network (CellNN) to overcome these problems. Multiple configurations of CellNNs are trained in a framework called TensorFlow to classify objects from the CIFAR‐10 database. Effects of the number of iterations, the number of channels, batch normalization, and activation functions on the classification accuracies are presented. It is shown that TensorFlow is a tool that is capable of training discrete‐time CellNNs. Although the accuracies of the proposed networks on CIFAR‐10 are slightly lesser than the existing CNNs, with reduced parameters and multiply‐accumulates (MACs), power consumption and computation time of our networks will be less than CNNs. [ABSTRACT FROM AUTHOR]

This paper presents a proof of concept of a method to identify substructures in 2D NMR spectra of mixtures using a bespoke image‐based convolutional neural network application. This is done using HSQC and HMBC spectra separately and in combination. The application can reliably detect substructures in pure compounds, using a simple network. Results indicate that it can work for mixtures when trained on pure compounds only. HMBC data and the combination of HMBC and HSQC show better results than HSQC alone in this pilot study. [ABSTRACT FROM AUTHOR]

Advances in equipment enable photographers to take ultra‐high‐definition (UHD) photos, and photo retouching enables them to create an impressive image by artistically adjusting and enhancing the brightness, color etc. However, such image enhancement is an artistic and challenging task, which requires a lot of relevant experience and technology. Thus, using an automated algorithm is an attractive option. In recent years, deep learning has made great progress in image processing, which motivated the authors to explore this application in UHD image enhancement task. In this paper, an end‐to‐end perceptual enhancement method for UHD portrait images is proposed. Since the artistry of images comes from human perception, including the perception of exposure, structure, color etc., an image preprocessing method is proposed which can help the network learn the brightness change from the original image to the target image more accurately. Then, a composite perceptual loss function is proposed that combines global, structural, and color losses. This loss allows the network to simultaneously optimize the distance and perceptual similarity to a given target image during back‐propagation. To preserve the details of the UHD image and reduce the number of network calculations and parameters, a Perceptual Enhancement Network (called PEN) based on dilated convolution is proposed. Extensive experiments on the datasets show that the method excels in baselines in both subjective and objective evaluations. [ABSTRACT FROM AUTHOR]

Single‐Image Super‐Resolution (SISR) has always been an important topic in the field of image processing, which attempts to improve the image resolution and is of great significance in practice. Recently, SISR has made substantial progress aided by deep learning (DL), which has demonstrated impressive potential in many low‐level tasks. In the current DL‐based SISR approaches, most of them are based on supervised learning. However, in the real world, only low‐resolution (LR) images with unknown degradation are provided, which limit the application of current supervised models. To mitigate this problem, in this paper, a two‐stage semi‐supervised SISR method called SRAttentionGAN, is proposed. First, an upsampling network SRResNet, which is pre‐trained in a supervised manner, is employed to scale the LR image to the desired size. Then, the upsampled results are fed into our improved unsupervised CycleGAN framework, which does not need paired samples, to obtain sharper and more realistic super‐resolution (SR) images. Specifically, in the improved CycleGAN part, an attention‐guided generator is proposed to perceive the discriminative semantic parts between the source and target images, to avoid the impact from low‐level information. It also prevents the overall color tone from being changed. A multi‐scale discriminator is also adopted to further rich texture details. The effectiveness of the proposed SRAttentionGAN experiments is validated using four benchmarks (Set5, Set14, Urban100, and BSDS100) in both quantitative and qualitative aspects. Compared with the state‐of‐the‐arts, the results are visually promising and show competitive performance in perceptual metrics, Natural Image Quality Evaluator (NIQE) and Perception Index (PI), which have better agreement with the human visual perception. [ABSTRACT FROM AUTHOR]

Automated examination of biomedical signals plays a vital role to diagnose diseases and offers useful data to several applications in the areas of physiology, sports medicine, and human–computer interface. The latest advancements in Artificial Intelligence (AI) have the ability to manage and analyse enormous biomedical datasets resulting in clinical decision making and real time applications. At the same time, Colorectal cancer (CRC) is the third most deadly disease affecting people over the globe. The utilization of AI techniques for the earlier identification of CRC has gained significant interest among the research communities. Therefore, this paper presents a novel AI based fusion model for CRC disease diagnosis and classification, named AIFM‐CRC. The presented AIFM‐CRC model primarily undergoes Gaussian filtering based noise removal and contrast enhancement as a preprocessing stage. In addition, a fusion based feature extraction process takes place where the SIFT based handcrafted features and Inception v4 based deep features are fused together. Besides, whale optimization algorithm tuned deep support vector machine model is employed as a classification technique to determine the existence of CRC. In order to highlight the proficient results analysis of the AIFM‐CRC model, a comprehensive simulation analysis takes place. The resultant experimental values pointed out the betterment of the AIFM‐CRC model by accomplishing a maximum accuracy of 96.18%. [ABSTRACT FROM AUTHOR]

With the advances of deep learning research in the past few years, healthcare and smart medicines have been significantly developed. Inspired by the wide application of deep learning in medical image classification and disease diagnosis, this paper further proposes a variant of the Residual Network framework to classify the oral ulcer images in real‐time. In particular, image pre‐processing and enhancement techniques are used to enrich the datasets and reduce model overfitting. Besides, the transfer learning is further introduced into the residual blocks to improve the classification accuracy, with the later layers trained from the labeled datasets. To validate the performance of authors' proposal, it is compared with other classic deep learning models with respect to the classification sensitivity, specificity, and accuracy. The experimental results show that authors' approach outperforms those classic classification networks when the oral ulcers are classified and diagnosed in real‐time. [ABSTRACT FROM AUTHOR]

Rail surface defects (RSDs) are a major problem that reduces operation safety. Unfortunately, the existing RSD detection systems have very limited accuracy. Current image processing methods are not tailored for the railway track and many fully convolutional networks (FCN)‐based methods suffer from the blurry rail edges (RE). This paper proposes a new rail boundary guidance network (RBGNet) for salient RS detection. First, a novel architecture is proposed to fully utilize the complementarity between the RS and the RE to accurately identify the RS with well‐defined boundaries. The newly developed RBGNet injects high‐level RS object information into shallow RS edge features by a progressive fused way for obtaining fine edge features. Then, the system integrates the refined edge features with RS features at different high‐level layers to predict the RS precisely. Second, an innovative hybrid loss consisting of binary cross entropy (BCE), structural similarity index measure (SSIM), and intersection‐over‐union (IoU) is proposed and equipped into the RBGNet to supervise the network and learn the transformation between the input and ground truth. The input and ground truth then further refine the RS location and edges. Conveniently, an image‐based model for RSD detection and quantification is also developed and integrated for an automatic inspection purpose. Finally, experiments conducted on the complex unmanned aerial vehicle (UAV) rail dataset indicate the system can achieve a high detection rate with good adaptation capability in complicated environments. [ABSTRACT FROM AUTHOR]

The intelligentisation of coal mines is the only approach to the high‐quality development of the coal industry. Detection, identification and sorting of coal gangue is an important part of the intelligentisation of coal mines. Focusing on various problems in coal gangue detecting and recognising algorithms, such as limited receptive field, slow convergence rate and low accuracy of small particle recognition, this paper proposes a coal gangue detection and recognition algorithm based on deformable convolution YOLOv3 (DCN‐YOLOv3). To improve the accuracy of anchor frame positioning and enhance the diversity of the dataset, the deformed convolution YOLOv3 network model is established based on the detection algorithm YOLOv3, using deformable convolution, multiple k‐means clustering results average method and data enhancement technology as means. The model was trained through the self‐designed dataset, and the algorithm's correctness and accuracy for coal gangue recognition under different size and illumination conditions are verified. The test results showed that the algorithm effectively detects and recognises coal gangue, improves the accuracy and efficiency of detecting and recognising small‐size coal and gangue and improves environmental robustness. Furthermore, compared with the traditional recognition algorithm, the network convergence speed of this algorithm is significantly improved, the mAP is increased to 99.45%, and the maximum FLOPs value is reduced by 61.4%. Accordingly, this research is considered to be of certain theoretical value and technical reference for identifying coal gangue. [ABSTRACT FROM AUTHOR]

In the dark scene at night, the face images captured by ordinary visible light (VIS) are generally poor quality and very dim, while the near‐infrared (NIR) can capture high definition and recognizable face images at night. The NIR‐VIS Heterogeneous face recognition has become a hot research field, which helps to build an all‐weather face recognition system. NIR‐VIS HFR is sophisticated because of the large visual difference between NIR images and VIS images. In order to reduce the difficulty of such cross‐modality invariant feature learning, this paper proposes a cross‐modality data gap decomposed by auxiliary modality method (DGD) for NIR‐VIS HFR. First, the brightness component (Y component) of VIS image YCbCr space is used as the auxiliary modality to decompose the cross‐modality data gap. The lightness component retained the structural information of VIS image and was similar to the colour information of NIR modality; in this way, the huge gap between the NIR data and the VIS data is decomposed into two smaller gaps, thus reducing the difficulty of network learning. Second, the data of the three modalities are input into the weight sharing network and training under the combined guidance of cross‐modality gap decomposition loss and intra‐modality gap loss; in this way, the modality invariant features can be learned faster and better. Extensive experiments were conducted on two commonly used datasets CASIA NIR‐VIS 2.0 and Oulu‐CASIA NIR‐VIS to evaluate DGD method. Experimental results indicate DGD method has competitive performance compared with the latest methods. [ABSTRACT FROM AUTHOR]

Deraining has been attracting a lot of attention from researchers, and various methods have been proposed, especially deep‐networks are widely adopted in recent years. Their structures and learning become more and more complicated and diverse, making it difficult to analyze the contributions and improvements. In this paper, a comprehensive review for current rain removal methods is first provided to show their contributions. Specifically, they are reviewed in terms of handing rain streaks and rain mist. Second, besides evaluating their rain removal ability, they are also evaluated in terms of their impact on subsequent stereo‐matching task. To this end, a new deraining dataset is first prepared, called Rain‐Kitti2012 and Rain‐Kitti2015. They are created by adding rain part to clean image‐pairs in Kitti2012 and Kitti2015. By then, nine state‐of‐the‐art deraining methods are evaluated with full‐reference and no‐reference image quality assessment metrics. Furthermore, the blurriness and distortion types introduced during deraining are measured. Finally, three learning‐based stereo matching methods are compared, and they take the outputs of deraining methods as inputs. It is further discussed how derained images influence the accuracy of stereo matching, which can provide some insight for jointly handling rain removal and stereo matching. 1: A comprehensive review for the current rain removal methods is provided. They are categorized into rain‐streak‐oriented and rain‐mist‐oriented approaches in terms of degradation type, and are categorized into model‐driven and data‐driven approaches in terms of methodology. 2: A new image deraining dataset is introduced, which is the first dataset that can be used to perform stereo‐matching‐driven evaluation for deraining methods. The dataset is created by adding rain part to clean images in KITTI2012 and KITTI2015. 3: We evaluate 9 deep learning based deraining methods with full‐reference and no‐ reference metrics. In addition, the types of distortions produced by these methods are discussed and measured quantitatively. And, the impact of 9 deraining methods on the subsequent stereo matching task is evaluated, which can provide some insight on how to design stereo matching task‐driven deraining methods. [ABSTRACT FROM AUTHOR]

With the performance increase of convolutional neural network (CNN), the disadvantages of CNN's high storage and high power consumption are followed. Among the methods mentioned in various literature, filter pruning is a crucial method for constructing lightweight networks. However, the current filter pruning method is still challenged by complicated processes and training inefficiency. This paper proposes an effective filter pruning method, which uses the saliency of the feature map (SFM), i.e. information entropy, as a theoretical guide for whether the filter is essential. The pruning principle use here is that the filter with a weak saliency feature map in the early stage will not significantly improve the final accuracy. Thus, one can efficiently prune the non‐salient feature map with a smaller information entropy and the corresponding filter. Besides, an over‐parameterized convolution method is employed to improve the pruned model's accuracy without increasing parameter at inference time. Experimental results show that without introducing any additional constraints, the effectiveness of this method in FLOPs and parameters reduction with similar accuracy has advanced the state‐of‐the‐art. For example, on CIFAR‐10, the pruned VGG‐16 achieves only a small loss of 0.39% in Top‐1 accuracy with a factor of 83.3% parameters, and 66.7% FLOPs reductions. On ImageNet‐100, the pruned ResNet‐50 achieves only a small accuracy degradation of 0.76% in Top‐1 accuracy with a factor of 61.19% parameters, and 62.98% FLOPs reductions. [ABSTRACT FROM AUTHOR]

This paper proposes a novel image classification architecture named ensemble cross‐stage partial attention network based on the backbone network DarkNet53 of Yolov3 to improve the feature extraction capability and the interpretability of image classification. This network has multiple advantages, including light model parameters, fast classification speed, and high classification accuracy for small objects and complex images. Local network architectures of different cross‐phases are added in the proposed network structure to reduce the calculation. Furthermore, channel and hybrid domain attention modules, which, respectively, fuse the branch feature with the extracted channel and spatial attention features, are designed for feature extraction of images. Experimental results confirm the improved performance of the proposed approach on the CIFAR‐100, ImageNet, and UCMerced datasets. In addition, experiments on the MSCOCO dataset suggest the application of the proposed method to object detection with satisfactory accuracy. [ABSTRACT FROM AUTHOR]

To address the existing problems of capsule networks in deep feature extraction and spatial‐spectral feature fusion of hyperspectral images, this paper proposes a hyperspectral image classification method that combines a deep residual 3D capsule network and Markov random field. Based on this method, the deep spatial‐spectral features of hyperspectral images are extracted using the deep residual 3D convolutional structure, the vector capsules of the features are obtained by the initial capsule layer and mapped into probability capsules via the 3D dynamic routing mechanism to construct the classification probability map, and the spatial structure of the classification results is regularised by the Markov random field to further improve the classification accuracy and performance of the images. Two sets of benchmark hyperspectral images, namely Indian Pines and Pavia University data sets, were used to conduct comparative experiments and ablation study. The experimental results showed that, compared with the conventional convolutional neural network and existing capsule network models, the proposed method not only improves the classification accuracy of the images but also partly eliminates the category noise and affords a more regular classification probability map. [ABSTRACT FROM AUTHOR]

Purpose: Involuntary patient movement results in data discontinuities during computed tomography (CT) scans which lead to a serious degradation in the image quality. In this paper, we specifically address artifacts induced by patient motion during a head scan. Method: Instead of trying to solve an inverse problem, we developed a motion simulation algorithm to synthesize images with motion‐induced artifacts. The artifacts induced by rotation, translation, oscillation and any possible combination are considered. Taking advantage of the powerful learning ability of neural networks, we designed a novel 3D network structure with both a large reception field and a high image resolution to map the artifact‐free images from artifact‐contaminated images. Quantitative results of the proposed method were evaluated against the results of U‐Net and proposed networks without dilation structure. Thirty sets of motion contaminated images from two hospitals were selected to do a clinical evaluation. Result: Facilitating the training dataset with artifacts induced by variable motion patterns and the neural network, the artifact can be removed with good performance. Validation dataset with simulated random motion pattern showed outperformed image correction, and quantitative results showed the proposed network had the lowest normalized root‐mean‐square error, highest peak signal‐to‐noise ratio and structure similarity, indicating our network gave the best approximation of gold standard. Clinical image processing results further confirmed the effectiveness of our method. Conclusion: We proposed a novel deep learning‐based algorithm to eliminate motion artifacts. The convolutional neural networks trained with synthesized image pairs achieved promising results in artifacts reduction. The corrected images increased the diagnostic confidence compared with artifacts contaminated images. We believe that the correction method can restore the ability to successfully diagnose and avoid repeated CT scans in certain clinical circumstances. [ABSTRACT FROM AUTHOR]

Diabetes mellitus is a major medical concern worldwide. Long‐term diabetes can affect the retina of the eye and lead to diabetic retinopathy (DR) and diabetic macular edema (DME). Proper screening and consultation with an ophthalmologist are necessary to prevent avoidable vision loss. As DR and DME have become more prevalent, automated screening is essential to provide cost‐effective and rapid solutions with reduced human resources requirements. This paper aims to provide a comprehensive review of the literature on computer‐aided diagnosis of DR and DME. We identified the studies on automated five‐class grading of DR according to International Clinical Diabetic Retinopathy severity scale and three class grading of diabetic maculopathy, using fundus images. A systematic search on research repositories was conducted, and relevant studies were scrutinized and included in the review. The studies were reported in nearly 100 different journals. We have reviewed the studies in all aspects including datasets, preprocessing, non‐deep learning, and deep learning‐based algorithms, and evaluation metrics. Significant contributions in developing automated tools for DR/DME grading are highlighted. We have identified and discussed research gaps and challenges. This will help researchers to get an updated summary of work done in the area. Deep learning‐based algorithms have outperformed the traditional algorithms in the domain. Despite their promising performance, these algorithms reveal the potential for significant improvements to become a reliable tool in clinical settings. [ABSTRACT FROM AUTHOR]

Deep learning‐based approaches have made considerable progress in image classification tasks, but most of the approaches lack interpretability, especially in revealing the decisive information causing the categorization of images. This paper seeks to answer the question of what clues encode the discriminative visual information between image categories and can help improve the classification performance. To this end, an attention‐based clue extraction network (ACENet) is introduced to mine the decisive local visual information for image classification. ACENet constructs a clue‐attention mechanism, that is global‐local attention, between the image and visual clue proposals extracted from it and then introduces a contrastive loss defined over the achieved discrete attention distribution to increase the discriminability of clue proposals. The loss encourages considerable attention to be devoted to discriminative clue proposals, that is those similar within the same category and dissimilar across categories. The experimental results for the Negative Web Image (NWI) dataset and the public ImageNet2012 dataset demonstrate that ACENet can extract true clues to improve the image classification performance and outperforms the baselines and the state‐of‐the‐art methods. [ABSTRACT FROM AUTHOR]

One of the main challenges in computer science and image processing is 2D human pose estimation. Specifically, occlusion and in particular occlusion of human joints caused by camera angle are of paramount importance. In this paper, a new highly accurate network was proposed that can estimate 2D human poses in video images using deep learning. We employ the Single Shot MultiBox Detector network to detect the centre position of each human within a video frame and then use the stacked hourglass network to estimate the 2D human pose. We approximate the human motion as a linear motion between different frames in a certain period; and optimize the human centres based on the local outlier factor and Kalman filters. The same method is applied to optimize the human pose estimations in video, which can address the inaccurate prediction caused by human joints occlusion. The proposed adaptive network is tested using the two well‐known benchmarks for human pose estimation (MPII and Joint‐annotated Human Motion Data Base datasets), and we also generate some 2D human pose estimating qualitative results of single and multiple people in Internet videos. The experimental results show that the proposed network has strong practicability and can achieve high accuracy on adaptive estimating the 2D human pose in video. [ABSTRACT FROM AUTHOR]

This paper intends to present an automated mango grading system under four stages (1) pre‐processing, (2) feature extraction, (3) optimal feature selection and (4) classification. Initially, the input image is subjected to the pre‐processing phase, where the reading, sizing, noise removal and segmentation process happens. Subsequently, the features are extracted from the pre‐processed image. To make the system more effective, from the extracted features, the optimal features are selected using a new hybrid optimization algorithm termed the lion assisted firefly algorithm (LA‐FF), which is the combination of LA and FF, respectively. Then, the optimal features are given for the classification process, where the optimized deep convolutional neural network (CNN) is deployed. As a major contribution, the configuration of CNN is fine‐tuned via selecting the optimal count of convolutional layers. This obviously enhances the classification accuracy in grading system. For fine‐tuning the convolutional layers in the deep CNN, the LA‐FF algorithm is used so that the classifier is optimized. The grading is evaluated on the basis of healthydiseased, ripeunripe and bigmediumvery big cases with respect to type I and type II measures and the performance of the proposed grading model is compared over the other state‐of‐the‐art models. [ABSTRACT FROM AUTHOR]