Search

The research groups in computer vision, graphics, and machine learning have dedicated a substantial amount of attention to the areas of 3D object reconstruction, augmentation, and registration. Deep learning is the predominant method used in artificial intelligence for addressing computer vision challenges. However, deep learning on three-dimensional data presents distinct obstacles and is now in its nascent phase. There have been significant advancements in deep learning specifically for three-dimensional data, offering a range of ways to address these issues. This study offers a comprehensive examination of the latest advancements in deep learning methodologies. We examine many benchmark models for the tasks of 3D object registration, augmentation, and reconstruction. We thoroughly analyse their architectures, advantages, and constraints. In summary, this report provides a comprehensive overview of recent advancements in three-dimensional deep learning and highlights unresolved research areas that will need to be addressed in the future. [ABSTRACT FROM AUTHOR]

This document is an editorial for a special issue titled "Machine Learning in Computer Vision and Image Sensing: Theory and Applications." The editorial highlights the diverse applications of machine learning (ML) models in various domains such as medical imaging, signal processing, remote sensing, and human activity detection. The special issue includes 11 papers that cover topics such as image segmentation, fluvial navigation, Alzheimer's disease classification, pneumothorax detection, lung cancer malignancy prediction, amniotic fluid volume detection, COVID-19 detection, and Parkinson's disease detection. The papers showcase the progress and potential of ML models in computer vision applications and provide valuable insights for future research. [Extracted from the article]

Emotion recognition has become increasingly important in the field of Deep Learning (DL) and computer vision due to its broad applicability by using human–computer interaction (HCI) in areas such as psychology, healthcare, and entertainment. In this paper, we conduct a systematic review of facial and pose emotion recognition using DL and computer vision, analyzing and evaluating 77 papers from different sources under Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our review covers several topics, including the scope and purpose of the studies, the methods employed, and the used datasets. The scope of this work is to conduct a systematic review of facial and pose emotion recognition using DL methods and computer vision. The studies were categorized based on a proposed taxonomy that describes the type of expressions used for emotion detection, the testing environment, the currently relevant DL methods, and the datasets used. The taxonomy of methods in our review includes Convolutional Neural Network (CNN), Faster Region-based Convolutional Neural Network (R-CNN), Vision Transformer (ViT), and "Other NNs", which are the most commonly used models in the analyzed studies, indicating their trendiness in the field. Hybrid and augmented models are not explicitly categorized within this taxonomy, but they are still important to the field. This review offers an understanding of state-of-the-art computer vision algorithms and datasets for emotion recognition through facial expressions and body poses, allowing researchers to understand its fundamental components and trends. [ABSTRACT FROM AUTHOR]

Photovoltaic (PV) fault detection is crucial because undetected PV faults can lead to significant energy losses, with some cases experiencing losses of up to 10%. The efficiency of PV systems depends upon the reliable detection and diagnosis of faults. The integration of Artificial Intelligence (AI) techniques has been a growing trend in addressing these issues. The goal of this systematic review is to offer a comprehensive overview of the recent advancements in AI-based methodologies for PV fault detection, consolidating the key findings from 31 research papers. An initial pool of 142 papers were identified, from which 31 were selected for in-depth review following the PRISMA guidelines. The title, objective, methods, and findings of each paper were analyzed, with a focus on machine learning (ML) and deep learning (DL) approaches. ML and DL are particularly suitable for PV fault detection because of their capacity to process and analyze large amounts of data to identify complex patterns and anomalies. This study identified several AI techniques used for fault detection in PV systems, ranging from classical ML methods like k-nearest neighbor (KNN) and random forest to more advanced deep learning models such as Convolutional Neural Networks (CNNs). Quantum circuits and infrared imagery were also explored as potential solutions. The analysis found that DL models, in general, outperformed traditional ML models in accuracy and efficiency. This study shows that AI methodologies have evolved and been increasingly applied in PV fault detection. The integration of AI in PV fault detection offers high accuracy and effectiveness. After reviewing these studies, we proposed an Artificial Neural Network (ANN)-based method for PV fault detection and classification. [ABSTRACT FROM AUTHOR]

The purpose of image dehazing is to remove the interference from weather factors in degraded images and enhance the clarity and color saturation of images to maximize the restoration of useful features. Single image dehazing is one of the most important tasks in the field of image restoration. In recent years, due to the progress of deep learning, single image dehazing has made great progress. With the success of Transformer in advanced computer vision tasks, some research studies also began to apply Transformer to image dehazing tasks and obtained surprising results. However, both the deconvolution-neural-network-based dehazing algorithm and Transformer based dehazing algorithm magnify their advantages and disadvantages separately. Therefore, this paper proposes a novel Transformer–Convolution fusion dehazing network (TCFDN), which uses Transformer's global modeling ability and convolutional neural network's local modeling ability to improve the dehazing ability. In the Transformer–Convolution fusion dehazing network, the classic self-encoder structure is used. This paper proposes a Transformer–Convolution hybrid layer, which uses an adaptive fusion strategy to make full use of the Swin-Transformer and convolutional neural network to extract and reconstruct image features. On the basis of previous research, this layer further improves the ability of the network to remove haze. A series of contrast experiments and ablation experiments not only proved that the Transformer–Convolution fusion dehazing network proposed in this paper exceeded the more advanced dehazing algorithm, but also provided solid and powerful evidence for the basic theory on which it depends. [ABSTRACT FROM AUTHOR]

Street view imagery (SVI) is a rich source of information for architectural and urban analysis using computer vision techniques, but its integration with other building-level data sources requires an additional step of visual building identification. This step is particularly challenging in architecturally homogeneous, dense residential streets featuring narrow buildings, due to a combination of SVI geolocation errors and occlusions that significantly increase the risk of confusing a building with its neighboring buildings. This paper introduces a robust deep learning-based method to identify buildings across multiple street views taken at different angles and times, using global optimization to correct the position and orientation of street view panoramas relative to their surrounding building footprints. Evaluating the method on a dataset of 2000 street views shows that its identification accuracy (88%) outperforms previous deep learning-based methods (79%), while methods solely relying on geometric parameters correctly show the intended building less than 50% of the time. These results indicate that previous identification methods lack robustness to panorama pose errors when buildings are narrow, densely packed, and subject to occlusions, while collecting multiple views per building can be leveraged to increase the robustness of visual identification by ensuring that building views are consistent. [ABSTRACT FROM AUTHOR]

Recent technological developments have enabled computers to identify and categorize facial expressions to determine a person's emotional state in an image or a video. This process, called "Facial Expression Recognition (FER)", has become one of the most popular research areas in computer vision. In recent times, deep FER systems have primarily concentrated on addressing two significant challenges: the problem of overfitting due to limited training data availability, and the presence of expression-unrelated variations, including illumination, head pose, image resolution, and identity bias. In this paper, a comprehensive survey is provided on deep FER, encompassing algorithms and datasets that offer insights into these intrinsic problems. Initially, this paper presents a detailed timeline showcasing the evolution of methods and datasets in deep facial expression recognition (FER). This timeline illustrates the progression and development of the techniques and data resources used in FER. Then, a comprehensive review of FER methods is introduced, including the basic principles of FER (components such as preprocessing, feature extraction and classification, and methods, etc.) from the pro-deep learning era (traditional methods using handcrafted features, i.e., SVM and HOG, etc.) to the deep learning era. Moreover, a brief introduction is provided related to the benchmark datasets (there are two categories: controlled environments (lab) and uncontrolled environments (in the wild)) used to evaluate different FER methods and a comparison of different FER models. Existing deep neural networks and related training strategies designed for FER, based on static images and dynamic image sequences, are discussed. The remaining challenges and corresponding opportunities in FER and the future directions for designing robust deep FER systems are also pinpointed. [ABSTRACT FROM AUTHOR]

Given the pronounced impact COVID-19 continues to have on society—infecting 700 million reported individuals and causing 6.96 million deaths—many deep learning works have recently focused on the virus's diagnosis. However, assessing severity has remained an open and challenging problem due to a lack of large datasets, the large dimensionality of images for which to find weights, and the compute limitations of modern graphics processing units (GPUs). In this paper, a new, iterative application of transfer learning is demonstrated on the understudied field of 3D CT scans for COVID-19 severity analysis. This methodology allows for enhanced performance on the MosMed Dataset, which is a small and challenging dataset containing 1130 images of patients for five levels of COVID-19 severity (Zero, Mild, Moderate, Severe, and Critical). Specifically, given the large dimensionality of the input images, we create several custom shallow convolutional neural network (CNN) architectures and iteratively refine and optimize them, paying attention to learning rates, layer types, normalization types, filter sizes, dropout values, and more. After a preliminary architecture design, the models are systematically trained on a simplified version of the dataset-building models for two-class, then three-class, then four-class, and finally five-class classification. The simplified problem structure allows the model to start learning preliminary features, which can then be further modified for more difficult classification tasks. Our final model CoSev boosts classification accuracies from below 60% at first to 81.57% with the optimizations, reaching similar performance to the state-of-the-art on the dataset, with much simpler setup procedures. In addition to COVID-19 severity diagnosis, the explored methodology can be applied to general image-based disease detection. Overall, this work highlights innovative methodologies that advance current computer vision practices for high-dimension, low-sample data as well as the practicality of data-driven machine learning and the importance of feature design for training, which can then be implemented for improvements in clinical practices. [ABSTRACT FROM AUTHOR]

Waste disposal remains a challenge due to land availability, and environmental and health issues related to the main disposal method, landfilling. Combining computer vision (machine learning) and robotics to sort waste is a cost-effective solution for landfilling activities limitation. The objective of this study was to combine transfer and ensemble learning to process collected waste images and classify landfill waste into nine classes. Pretrained CNN models (Inception–ResNet-v2, EfficientNetb3, and DenseNet201) were used as base models to develop the ensemble network, and three other single CNN models (Models 1, 2, and 3). The single network performances were compared to the ensemble model. The waste dataset, initially grouped in two classes, was obtained from Kaggle, and reorganized into nine classes. Classes with a low number of data were improved by downloading additional images from Google search. The Ensemble Model showed the highest prediction precision (90%) compared to the precision of Models 1, 2, and 3, 86%, 87%, and 88%, respectively. All models had difficulties predicting overlapping classes, such as glass and plastics, and wood and paper/cardboard. The environmental costs for the Ensemble network, and Models 2 and 3, approximately 15 g CO2 equivalent per training, were lower than the 19.23 g CO2 equivalent per training for Model 1. [ABSTRACT FROM AUTHOR]

Human pose estimation refers to accurately estimating the position of the human body from a single RGB image and detecting the location of the body. It serves as the basis for several computer vision tasks, such as human tracking, 3D reconstruction, and autonomous driving. Improving the accuracy of pose estimation has significant implications for the advancement of computer vision. This paper addresses the limitations of single-branch networks in pose estimation. It presents a top-down single-target pose estimation approach based on multi-branch self-calibrating networks combined with graph convolutional neural networks. The study focuses on two aspects: human body detection and human body pose estimation. The human body detection is for athletes appearing in sports competitions, followed by human body pose estimation, which is divided into two methods: coordinate regression-based and heatmap test-based. To improve the accuracy of the heatmap test, the high-resolution feature map output from HRNet is used for deconvolution to improve the accuracy of single-target pose estimation recognition. [ABSTRACT FROM AUTHOR]

Simple Summary: The use of computer vision technology has improved the effectiveness of individual sheep identification, but existing methods face challenges such as large parameter sizes, slow recognition speeds, and difficult deployment. To address this issue, we have made improvements and optimizations based on the Retinaface face recognition model to create a balanced speed-accuracy sheep face recognition model. The optimized model has fewer parameters, simpler computations, faster inference speeds, and higher recognition accuracy, making it well-suited for deployment on resource-constrained edge devices. This research is expected to promote the application of deep learning-based sheep face recognition methods in production. The recognition of sheep faces based on computer vision has improved the efficiency and effectiveness of individual sheep identification, providing technical support for the development of smart farming. However, current recognition models have problems such as large parameter sizes, slow recognition speed, and difficult deployment. Therefore, this paper proposes an efficient and fast basic module called Eblock and uses it to build a lightweight sheep face recognition model called SheepFaceNet, which achieves the best balance between speed and accuracy. SheepFaceNet includes two modules: SheepFaceNetDet for detection and SheepFaceNetRec for recognition. SheepFaceNetDet uses Eblock to construct the backbone network to enhance feature extraction capability and efficiency, designs a bidirectional FPN layer (BiFPN) to enhance geometric location ability, and optimizes the network structure, which affects inference speed, to achieve fast and accurate sheep face detection. SheepFaceNetRec uses Eblock to construct the feature extraction network, uses ECA channel attention to improve the effectiveness of feature extraction, and uses multi-scale feature fusion to achieve fast and accurate sheep face recognition. On our self-built sheep face dataset, SheepFaceNet recognized 387 sheep face images per second with an accuracy rate of 97.75%, achieving an advanced balance between speed and accuracy. This research is expected to further promote the application of deep-learning-based sheep face recognition methods in production. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is an important research area with a wide range of application scenarios, such as smart homes, healthcare, abnormal behavior detection, etc. Wearable sensors, computer vision, radar, and other technologies are commonly used to detect human activity. However, they are severely limited by issues such as cost, lighting, context, and privacy. Therefore, this paper explores a high-performance method of using channel state information (CSI) to identify human activities, which is a deep learning-based spatial module-temporal convolutional network (SM-TCNNET) model. The model consists of a spatial feature extraction module and a temporal convolutional network (TCN) that can extract the spatiotemporal features in CSI signals well. In this paper, extensive experiments are conducted on the self-picked dataset and the public dataset (StanWiFi), and the results show that the accuracy reaches 99.93% and 99.80%, respectively. Compared with the existing methods, the recognition accuracy of the SM-TCNNET model proposed in this paper is improved by 1.8%. [ABSTRACT FROM AUTHOR]

Advancements in imaging, computer vision, and automation have revolutionized various fields, including field-based high-throughput plant phenotyping (FHTPP). This integration allows for the rapid and accurate measurement of plant traits. Deep Convolutional Neural Networks (DCNNs) have emerged as a powerful tool in FHTPP, particularly in crop segmentation—identifying crops from the background—crucial for trait analysis. However, the effectiveness of DCNNs often hinges on the availability of large, labeled datasets, which poses a challenge due to the high cost of labeling. In this study, a deep learning with bagging approach is introduced to enhance crop segmentation using high-resolution RGB images, tested on the NU-Spidercam dataset from maize plots. The proposed method outperforms traditional machine learning and deep learning models in prediction accuracy and speed. Remarkably, it achieves up to 40 % higher Intersection-over-Union (IoU) than the threshold method and 11 % over conventional machine learning, with significantly faster prediction times and manageable training duration. Crucially, it demonstrates that even small labeled datasets can yield high accuracy in semantic segmentation. This approach not only proves effective for FHTPP but also suggests potential for broader application in remote sensing, offering a scalable solution to semantic segmentation challenges. This paper is accompanied by publicly available source code. [ABSTRACT FROM AUTHOR]

Moving object detection is a vital research area that plays an essential role in intelligent transportation systems (ITSs) and various applications in computer vision. Recently, researchers have utilized convolutional neural networks (CNNs) to develop new techniques in object detection and recognition. However, with the increasing number of machine learning strategies used for object detection, there has been a growing need for large datasets with accurate ground truth used for the training, usually demanding their manual labeling. Moreover, most of these deep strategies are supervised and only applicable for specific scenes with large computational resources needed. Alternatively, other object detection techniques such as classical background subtraction need low computational resources and can be used with general scenes. In this paper, we propose a new a reliable semi-automatic method that combines a modified version of the detection-based CNN You Only Look Once V4 (YOLOv4) technique and background subtraction technique to perform an unsupervised object detection for surveillance videos. In this proposed strategy, background subtraction-based low-rank decomposition is applied firstly to extract the moving objects. Then, a clustering method is adopted to refine the background subtraction (BS) result. Finally, the refined results are used to fine-tune the modified YOLO v4 before using it in the detection and classification of objects. The main contribution of this work is a new detection framework that overcomes manual labeling and creates an automatic labeler that can replace manual labeling using motion information to supply labeled training data (background and foreground) directly from the detection video. Extensive experiments using real-world object monitoring benchmarks indicate that the suggested framework obtains a considerable increase in mAP compared to state-of-the-art results on both the CDnet 2014 and UA-DETRAC datasets. [ABSTRACT FROM AUTHOR]

A fundamental task in computer vision is the process of differentiation and identification of different objects or entities in a visual scene using semantic segmentation methods. The advancement of transformer networks has surpassed traditional convolutional neural network (CNN) architectures in terms of segmentation performance. The continuous pursuit of optimal performance, with respect to the popular evaluation metric results, has led to very large architectures that require a significant amount of computational power to operate, making them prohibitive for real-time applications, including autonomous driving. In this paper, we propose a model that leverages a visual transformer encoder with a parallel twin decoder, consisting of a visual transformer decoder and a CNN decoder with multi-resolution connections working in parallel. The two decoders are merged with the aid of two trainable CNN blocks, the fuser that combined the information from the two decoders and the scaler that scales the contribution of each decoder. The proposed model achieves state-of-the-art performance on the Cityscapes and ADE20K datasets, maintaining a low-complexity network that can be used in real-time applications. [ABSTRACT FROM AUTHOR]

Image restoration under adverse weather conditions poses a challenging task. Previous research efforts have predominantly focused on eliminating rain and fog phenomena from images. However, snow, being another common atmospheric occurrence, also significantly impacts advanced computer vision tasks such as object detection and semantic segmentation. Recently, there has been a surge of methods specifically targeting snow removal, with the majority employing visual Transformers as the backbone network to enhance restoration effectiveness. Nevertheless, due to the quadratic computations required by Transformers to model long-range dependencies, this significantly escalates the time and space consumption of deep learning models. To address this issue, this paper proposes an efficient snow removal Transformer with a global windowing network (SGNet). This method forgoes the localized windowing strategy of previous visual Transformers, opting instead to partition the image into multiple low-resolution subimages containing global information using wavelet sampling, thereby ensuring higher performance while reducing computational overhead. Extensive experimentation demonstrates that our approach achieves outstanding performance across a wide range of benchmark datasets and can rival methods employing CNNs in terms of computational cost. [ABSTRACT FROM AUTHOR]

With the rapid advancements in computer vision, using deep learning for strawberry disease recognition has emerged as a new trend. However, traditional identification methods heavily rely on manual discernment, consuming valuable time and imposing significant financial losses on growers. To address these challenges, this paper presents BerryNet-Lite, a lightweight network designed for precise strawberry disease identification. First, a comprehensive dataset, encompassing various strawberry diseases at different maturity levels, is curated. Second, BerryNet-Lite is proposed, utilizing transfer learning to expedite convergence through pre-training on extensive datasets. Subsequently, we introduce expansion convolution into the receptive field expansion, promoting more robust feature extraction and ensuring accurate recognition. Furthermore, we adopt the efficient channel attention (ECA) as the attention mechanism module. Additionally, we incorporate a multilayer perceptron (MLP) module to enhance the generalization capability and better capture the abstract features. Finally, we present a novel classification head design approach which effectively combines the ECA and MLP modules. Experimental results demonstrate that BerryNet-Lite achieves an impressive accuracy of 99.45%. Compared to classic networks like ResNet34, VGG16, and AlexNet, BerryNet-Lite showcases superiority across metrics, including loss value, accuracy, precision, F1-score, and parameters. It holds significant promise for applications in strawberry disease identification. [ABSTRACT FROM AUTHOR]

Simple Summary: Constructing a contactless pig mass estimation method through computer vision technology can help us to adjust our pig breeding program to improve production efficiency. Due to the complexity of the actual production environment, there are few reports on pig mass estimation in an unconstrained environment. In this study, we constructed a pig mass estimate model based on deep learning without constraint. The experimental results proved that the pig body mass estimation model constructed in this paper can quickly and accurately obtain the body mass of pigs. The model can evaluate the body quality of sows in real-time in an unconstrained environment, thereby providing data support for grading and adjusting breeding plans, and has broad application prospects. The body mass of pigs is an essential indicator of their growth and health. Lately, contactless pig body mass estimation methods based on computer vision technology have gained attention thanks to their potential to improve animal welfare and ensure breeders' safety. Nonetheless, current methods require pigs to be restrained in a confinement pen, and no study has been conducted in an unconstrained environment. In this study, we develop a pig mass estimation model based on deep learning, capable of estimating body mass without constraints. Our model comprises a Mask R-CNN-based pig instance segmentation algorithm, a Keypoint R-CNN-based pig keypoint detection algorithm and an improved ResNet-based pig mass estimation algorithm that includes multi-branch convolution, depthwise convolution, and an inverted bottleneck to improve accuracy. We constructed a dataset for this study using images and body mass data from 117 pigs. Our model achieved an RMSE of 3.52 kg on the test set, which is lower than that of the pig body mass estimation algorithm with ResNet and ConvNeXt as the backbone network, and the average estimation speed was 0.339 s·frame−1 Our model can evaluate the body quality of pigs in real-time to provide data support for grading and adjusting breeding plans, and has broad application prospects. [ABSTRACT FROM AUTHOR]

Automatic image description, also known as image captioning, aims to describe the elements included in an image and their relationships. This task involves two research fields: computer vision and natural language processing; thus, it has received much attention in computer science. In this review paper, we follow the Kitchenham review methodology to present the most relevant approaches to image description methodologies based on deep learning. We focused on works using convolutional neural networks (CNN) to extract the characteristics of images and recurrent neural networks (RNN) for automatic sentence generation. As a result, 53 research articles using the encoder-decoder approach were selected, focusing only on supervised learning. The main contributions of this systematic review are: (i) to describe the most relevant image description papers implementing an encoder-decoder approach from 2014 to 2022 and (ii) to determine the main architectures, datasets, and metrics that have been applied to image description. [ABSTRACT FROM AUTHOR]

Computer vision provides a real-time, non-destructive, and indirect way of horticultural crop yield estimation. Deep learning helps improve horticultural crop yield estimation accuracy. However, the accuracy of current estimation models based on RGB (red, green, blue) images does not meet the standard of a soft sensor. Through enriching more data and improving the RGB estimation model structure of convolutional neural networks (CNNs), this paper increased the coefficient of determination (R2) by 0.0284 and decreased the normalized root mean squared error (NRMSE) by 0.0575. After introducing a novel loss function mean squared percentage error (MSPE) that emphasizes the mean absolute percentage error (MAPE), the MAPE decreased by 7.58%. This paper develops a lettuce fresh weight estimation method through the multi-modal fusion of RGB and depth (RGB-D) images. With the multimodal fusion based on calibrated RGB and depth images, R2 increased by 0.0221, NRMSE decreased by 0.0427, and MAPE decreased by 3.99%. With the novel loss function, MAPE further decreased by 1.27%. A MAPE of 8.47% helps to develop a soft sensor for lettuce fresh weight estimation. [ABSTRACT FROM AUTHOR]

Crowd counting, as a basic computer vision task, plays an important role in many fields such as video surveillance, accident prediction, public security, and intelligent transportation. At present, crowd counting tasks face various challenges. Firstly, due to the diversity of crowd distribution and increasing population density, there is a phenomenon of large-scale crowd aggregation in public places, sports stadiums, and stations, resulting in very serious occlusion. Secondly, when annotating large-scale datasets, positioning errors can also easily affect training results. In addition, the size of human head targets in dense images is not consistent, making it difficult to identify both near and far targets using only one network simultaneously. The existing crowd counting methods mainly use density plot regression methods. However, this framework does not distinguish the features between distant and near targets and cannot adaptively respond to scale changes. Therefore, the detection performance in areas with sparse population distribution is not good. To solve such problems, we propose an adaptive multi-scale far and near distance network based on the convolutional neural network (CNN) framework for counting dense populations and achieving a good balance between accuracy, inference speed, and performance. However, on the feature level, in order to enable the model to distinguish the differences between near and far features, we use stacked convolution layers to deepen the depth of the network, allocate different receptive fields according to the distance between the target and the camera, and fuse the features between nearby targets to enhance the feature extraction ability of pedestrians under nearby targets. Secondly, depth information is used to distinguish distant and near targets of different scales and the original image is cut into four different patches to perform pixel-level adaptive modeling on the population. In addition, we add density normalized average precision (nAP) indicators to analyze the accuracy of our method in spatial positioning. This paper validates the effectiveness of NF-Net on three challenging benchmarks in Shanghai Tech Part A and B, UCF_ CC_50, and UCF-QNRF datasets. Compared with SOTA, it has more significant performance in various scenarios. In the UCF-QNRF dataset, it is further validated that our method effectively solves the interference of complex backgrounds. [ABSTRACT FROM AUTHOR]

Object detection is a computer vision task of detecting instances of objects of a certain class, identifying types of objects, determining its location, and accurately labelling them in an input image or a video. The scope of the work presented within this paper proposes a modern object detection network called NextDet to efficiently detect objects of multiple classes which utilizes CondenseNeXt, an award-winning lightweight image classification convolutional neural network algorithm with reduced number of FLOPs and parameters as the backbone, to efficiently extract and aggregate image features at different granularities in addition to other novel and modified strategies such as attentive feature aggregation in the head, to perform object detection and draw bounding boxes around the detected objects. Extensive experiments and ablation tests, as outlined in this paper, are performed on Argoverse-HD and COCO datasets, which provide numerous temporarily sparse to dense annotated images, demonstrate that the proposed object detection algorithm with CondenseNeXt as the backbone result in an increase in mean Average Precision (mAP) performance and interpretability on Argoverse-HD's monocular ego-vehicle camera captured scenarios by up to 17.39% as well as COCO's large set of images of everyday scenes of real-world common objects by up to 14.62%. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of nonlinear and dotted defect detection for multi-vision-based mask inspection systems in mask manufacturing lines. As the mask production amounts increased due to the spread of COVID-19 around the world, the mask inspection systems require more efficient defect detection algorithms. However, the traditional computer vision detection algorithms suffer from various types and very small sizes of the nonlinear and dotted defects on masks. This paper proposes a deep learning-based mask defect detection method, which includes a convolutional neural network (CNN) and efficient preprocessing. The proposed method was developed to be applied to real manufacturing systems, and thus all the training and inference processes were conducted with real data produced by real mask manufacturing systems. Experimental results show that the nonlinear and dotted defects were successfully detected by the proposed method, and its performance was higher than the previous method. [ABSTRACT FROM AUTHOR]

Recent advancements in deep learning and video analysis, combined with the efficiency of contemporary computational resources, have catalyzed the development of advanced real-time computational systems, significantly impacting various fields. This paper introduces a cutting-edge video analysis framework that was specifically designed to bolster security in museum environments. We elaborate on the proposed framework, which was evaluated and integrated into a real-time video analysis pipeline. Our research primarily focused on two innovative approaches: action recognition for identifying potential threats at the individual level and trajectory extraction for monitoring museum visitor movements, serving the dual purposes of security and visitor flow analysis. These approaches leverage a synergistic blend of deep learning models, particularly CNNs, and traditional computer vision techniques. Our experimental findings affirmed the high efficacy of our action recognition model in accurately distinguishing between normal and suspicious behaviors within video feeds. Moreover, our trajectory extraction method demonstrated commendable precision in tracking and analyzing visitor movements. The integration of deep learning techniques not only enhances the capability for automatic detection of malevolent actions but also establishes the trajectory extraction process as a robust and adaptable tool for various analytical endeavors beyond mere security applications. [ABSTRACT FROM AUTHOR]

Convolutional Neural Networks (CNNs) have become essential in deep learning applications, especially in computer vision, yet their complex internal mechanisms pose significant challenges to interpretability, crucial for ethical applications. Addressing this, our paper explores CNNs by examining their topological changes throughout the learning process, specifically employing persistent homology, a core method within Topological Data Analysis (TDA), to observe the dynamic evolution of their structure. This approach allows us to identify consistent patterns in the topological features of CNN kernels, particularly through shifts in Betti curves, which is a key concept in TDA. Our analysis of these Betti curves, initially focusing on the zeroth and first Betti numbers (respectively referred to as Betti-0 and Betti-1, which denote the number of connected components and loops), reveals insights into the learning dynamics of CNNs and potentially indicates the effectiveness of the learning process. We also discover notable differences in topological structures when CNNs are trained on grayscale versus color datasets, indicating the need for more extensive parameter space adjustments in color image processing. This study not only enhances the understanding of the intricate workings of CNNs but also contributes to bridging the gap between their complex operations and practical, interpretable applications. [ABSTRACT FROM AUTHOR]

A lot of research has been conducted in recent years on stereo depth estimation techniques, taking the traditional approach to a new level such that it is in an appreciably good form for competing in the depth estimation market with other methods, despite its few demerits. Sufficient progress in accuracy and depth computation speed has manifested during the period. Over the years, stereo depth estimation has been provided with various training modes, such as supervised, self-supervised, and unsupervised, before deploying it for real-time performance. These modes are to be used depending on the application and/or the availability of datasets for training. Deep learning, on the other hand, has provided the stereo depth estimation methods with a new life to breathe in the form of enhanced accuracy and quality of images, attempting to successfully reduce the residual errors in stages in some of the methods. Furthermore, depth estimation from a single RGB image has been intricate since it is an ill-posed problem with a lack of geometric constraints and ambiguities. However, this monocular depth estimation has gained popularity in recent years due to the development in the field, with appreciable improvements in the accuracy of depth maps and optimization of computational time. The help is mostly due to the usage of CNNs (Convolutional Neural Networks) and other deep learning methods, which help augment the feature-extraction phenomenon for the process and enhance the quality of depth maps/accuracy of MDE (monocular depth estimation). Monocular depth estimation has seen improvements in many algorithms that can be deployed to give depth maps with better clarity and details around the edges and fine boundaries, which thus helps in delineating between thin structures. This paper reviews various recent deep learning-based stereo and monocular depth prediction techniques emphasizing the successes achieved so far, the challenges acquainted with them, and those that can be expected shortly. [ABSTRACT FROM AUTHOR]

Blood oxygen saturation (SpO2) is an essential physiological parameter for evaluating a person's health. While conventional SpO2 measurement devices like pulse oximeters require skin contact, advanced computer vision technology can enable remote SpO2 monitoring through a regular camera without skin contact. In this paper, we propose novel deep learning models to measure SpO2 remotely from facial videos and evaluate them using a public benchmark database, VIPL-HR. We utilize a spatial–temporal representation to encode SpO2 information recorded by conventional RGB cameras and directly pass it into selected convolutional neural networks to predict SpO2. The best deep learning model achieves 1.274% in mean absolute error and 1.71% in root mean squared error, which exceed the international standard of 4% for an approved pulse oximeter. Our results significantly outperform the conventional analytical Ratio of Ratios model for contactless SpO2 measurement. Results of sensitivity analyses of the influence of spatial–temporal representation color spaces, subject scenarios, acquisition devices, and SpO2 ranges on the model performance are reported with explainability analyses to provide more insights for this emerging research field. [ABSTRACT FROM AUTHOR]

Scoring targets in shooting sports is a crucial and time-consuming task that relies on manually counting bullet holes. This paper introduces an automatic score detection model using object detection techniques. The study contributes to the field of computer vision by comparing the performance of seven models (belonging to two different architectural setups) and by making the dataset publicly available. Another value-added aspect is the inclusion of three variants of the object detection model, YOLOv8, recently released in 2023 (at the time of writing). Five of the used models are single-shot detectors, while two belong to the two-shot detectors category. The dataset was manually captured from the shooting range and expanded by generating more versatile data using Python code. Before the dataset was trained to develop models, it was resized (640 × 640) and augmented using Roboflow API. The trained models were then assessed on the test dataset, and their performance was compared using matrices like mAP50, mAP50-90, precision, and recall. The results showed that YOLOv8 models can detect multiple objects with good confidence scores. Among these models, YOLOv8m performed the best, with the highest mAP50 value of 96.7%, followed by the performance of YOLOv8s with the mAP50 value of 96.5%. It is suggested that if the system is to be implemented in a real-time environment, YOLOv8s is a better choice since it took significantly less inference time (2.3 ms) than YOLOv8m (5.7 ms) and yet generated a competitive mAP50 of 96.5%. [ABSTRACT FROM AUTHOR]

The North China type cucumber, characterized by its dense spines and top flowers, is susceptible to damage during the grading process, affecting its market value. Moreover, traditional manual grading methods are time-consuming and labor-intensive. To address these issues, this paper proposes a cucumber quality grader based on machine vision and deep learning. In the electromechanical aspect, a novel fixed tray type grading mechanism is designed to prevent damage to the vulnerable North China type cucumbers during the grading process. In the vision grading algorithm, a new convolutional neural network is introduced named MassNet, capable of predicting cucumber mass using only a top-view image. After obtaining the cucumber mass prediction, mass grading is achieved. Experimental validation includes assessing the electromechanical performance of the grader, comparing MassNet with different models in predicting cucumber mass, and evaluating the online grading performance of the integrated algorithm. Experimental results indicate that the designed cucumber quality grader achieves a maximum capacity of 2.3 t/hr. In comparison with AlexNet, MobileNet, and ResNet, MassNet demonstrates superior cucumber mass prediction, with a MAPE of 3.9% and RMSE of 6.7 g. In online mass grading experiments, the grading efficiency of the cucumber quality grader reaches 93%. [ABSTRACT FROM AUTHOR]

Advances in information and signal processing, driven by artificial intelligence techniques and recent breakthroughs in deep learning, have significantly impacted autonomous driving by enhancing safety and reducing the dependence on human intervention. Generally, prevailing ADASs (advanced driver assistance systems) incorporate costly components, making them financially unattainable for a substantial portion of the population. This paper proposes a solution: an embedded system designed for real-time pedestrian and priority sign detection, offering affordability and universal applicability across various vehicles. The suggested system, which comprises two cameras, an NVIDIA Jetson Nano B01 low-power edge device and an LCD (liquid crystal system) display, ensures seamless integration into a vehicle without occupying substantial space and provides a cost-effective alternative. The primary focus of this research is addressing accidents caused by the failure to yield priority to other drivers or pedestrians. Our study stands out from existing research by concurrently addressing traffic sign recognition and pedestrian detection, concentrating on identifying five crucial objects: pedestrians, pedestrian crossings (signs and road paintings separately), stop signs, and give way signs. Object detection was executed using a lightweight, custom-trained CNN (convolutional neural network) known as SSD (Single Shot Detector)-MobileNet, implemented on the Jetson Nano. To tailor the model for this specific application, the pre-trained neural network underwent training on our custom dataset consisting of images captured on the road under diverse lighting and traffic conditions. The outcomes of the proposed system offer promising results, positioning it as a viable candidate for real-time implementation; its contributions are noteworthy in advancing the safety and accessibility of autonomous driving technologies. [ABSTRACT FROM AUTHOR]

Urban agriculture presents unique challenges, particularly in the context of microclimate monitoring, which is increasingly important in food production. This paper explores the application of convolutional neural networks (CNNs) to forecast key sensor measurements from thermal images within this context. This research focuses on using thermal images to forecast sensor measurements of relative air humidity, soil moisture, and light intensity, which are integral to plant health and productivity in urban farming environments. The results indicate a higher accuracy in forecasting relative air humidity and soil moisture levels, with Mean Absolute Percentage Errors (MAPEs) within the range of 10–12%. These findings correlate with the strong dependency of these parameters on thermal patterns, which are effectively extracted by the CNNs. In contrast, the forecasting of light intensity proved to be more challenging, yielding lower accuracy. The reduced performance is likely due to the more complex and variable factors that affect light in urban environments. The insights gained from the higher predictive accuracy for relative air humidity and soil moisture may inform targeted interventions for urban farming practices, while the lower accuracy in light intensity forecasting highlights the need for further research into the integration of additional data sources or hybrid modeling approaches. The conclusion suggests that the integration of these technologies can significantly enhance the predictive maintenance of plant health, leading to more sustainable and efficient urban farming practices. However, the study also acknowledges the challenges in implementing these technologies in urban agricultural models. [ABSTRACT FROM AUTHOR]

Image deblurring attracts research attention in the field of image processing and computer vision. Traditional deblurring methods based on statistical prior largely depend on the selected prior type, which limits their restoring ability. Moreover, the constructed deblurring model is difficult to solve, and the operation is comparatively complicated. Meanwhile, deep learning has become a hotspot in various fields in recent years. End-to-end convolutional neural networks (CNNs) can learn the pixel mapping relationships between degraded images and clear images. In addition, they can also obtain the result of effectively eliminating spatial variable blurring. However, conventional CNNs have some disadvantages in generalization ability and details of the restored image. Therefore, this paper presents an iterative dual CNN called IDC for image deblurring, where the task of image deblurring is divided into two sub-networks: deblurring and detail restoration. The deblurring sub-network adopts a U-Net structure to learn the semantical and structural features of the image, and the detail restoration sub-network utilizes a shallow and wide structure without downsampling, where only the image texture features are extracted. Finally, to obtain the deblurred image, this paper presents a multiscale iterative strategy that effectively improves the robustness and precision of the model. The experimental results showed that the proposed method has an excellent effect of deblurring on a real blurred image dataset and is suitable for various real application scenes. [ABSTRACT FROM AUTHOR]

Image recognition has long been one of the research hotspots in computer vision tasks. The development of deep learning is rapid in recent years, and convolutional neural networks usually need to be designed with fixed resources. If sufficient resources are available, the model can be scaled up to achieve higher accuracy, for example, VggNet, ResNet, GoogLeNet, etc. Although the accuracy of large-scale models has been improved, the following problems will occur with the expansion of model scale: (1) There may be over-fitting; (2) increasing model parameters; (3) slow model convergence. This paper proposes a design method for a modular convolutional neural network model which solves the problem of over-fitting and large model parameters by connecting multiple modules in parallel. Moreover, each module contains several submodules (three submodules in this paper) and fuses the features extracted from the submodules. The model convergence can be accelerated by using the fused features (the fused features contain more image information). In this study, we add a gate unit based on the attention mechanism to the model, which aims to optimize the structure of the model (select the optimal number of modules), allowing the model to select an optimum network structure by learning and dynamically reducing FLOPs (floating-point operations per second) of the model. Compared to VggNet, ResNet, and GoogLeNet, the structure of the model proposed in this paper is simple and the parameters are small. The proposed model achieves good results in the Kaggle datasets Cats-vs.-Dogs (99.3%), 10-Monkey Species (99.26%), and Birds-400 (99.13%). [ABSTRACT FROM AUTHOR]

The task of semantic segmentation holds a fundamental position in the field of computer vision. Assigning a semantic label to each pixel in an image is a challenging task. In recent times, significant advancements have been achieved in the field of semantic segmentation through the application of Convolutional Neural Networks (CNN) techniques based on deep learning. This paper presents a comprehensive and structured analysis of approximately 150 methods of semantic segmentation based on CNN within the last decade. Moreover, it examines 15 well-known datasets in the semantic segmentation field. These datasets consist of 2D and 3D image and video frames, including general, indoor, outdoor, and street scenes. Furthermore, this paper mentions several recent techniques, such as SAM, UDA, and common post-processing algorithms, such as CRF and MRF. Additionally, this paper analyzes the performance evaluation of reviewed state-of-the-art methods, pioneering methods, common backbone networks, and popular datasets. These have been compared according to the results of Mean Intersection over Union (MIoU), the most popular evaluation metric of semantic segmentation. Finally, it discusses the main challenges and possible solutions and underlines some future research directions in the semantic segmentation task. We hope that our survey article will be useful to provide a foreknowledge to the readers who will work in this field. [ABSTRACT FROM AUTHOR]

Simple Summary: Achieving automatic detection of plant diseases in real agricultural scenarios where low-computing-power platforms are deployed is a significant research topic. As fine-grained agriculture continues to expand and farming methods deepen, traditional manual detection methods demand a high labor intensity. In recent years, the rapid advancement of computer network vision has greatly enhanced the computer-processing capabilities for pattern recognition problems across various industries. Consequently, a deep neural network based on an automatic pruning mechanism is proposed to enable high-accuracy plant disease detection even under limited computational power. Furthermore, an application is developed based on this method to expedite the translation of theoretical results into practical application scenarios. Timely and accurate detection of plant diseases is a crucial research topic. A dynamic-pruning-based method for automatic detection of plant diseases in low-computing situations is proposed. The main contributions of this research work include the following: (1) the collection of datasets for four crops with a total of 12 diseases over a three-year history; (2) the proposition of a re-parameterization method to improve the boosting accuracy of convolutional neural networks; (3) the introduction of a dynamic pruning gate to dynamically control the network structure, enabling operation on hardware platforms with widely varying computational power; (4) the implementation of the theoretical model based on this paper and the development of the associated application. Experimental results demonstrate that the model can run on various computing platforms, including high-performance GPU platforms and low-power mobile terminal platforms, with an inference speed of 58 FPS, outperforming other mainstream models. In terms of model accuracy, subclasses with a low detection accuracy are enhanced through data augmentation and validated by ablation experiments. The model ultimately achieves an accuracy of 0.94. [ABSTRACT FROM AUTHOR]

In addition, this paper also discussed how tabular data sets are converted into images for the application of two-dimensional convolutional neural networks, and how black box models using local and global rule extractions and rule-based methods affect reliability. To improve this, the contribution by Yoo and Kim, "Merchant Recommender System Using Credit Card Payment Data", proposed a model specialized for affiliate store recommendations by reflecting the domain information of credit card payment data [[11]]. Artificial intelligence technology has become an indispensable element in the field of electronics, and its influence is enormous. The multi-model inference accelerator was tested on low-density Zynq-7010 and Zynq-7020 FPGA devices and image classification experiments were performed on the CIFAR-10 dataset. [Extracted from the article]

Continuing progress in machine learning (ML) has led to significant advancements in agricultural tasks. Due to its strong ability to extract high-dimensional features from fruit images, deep learning (DL) is widely used in fruit detection and automatic harvesting. Convolutional neural networks (CNN) in particular have demonstrated the ability to attain accuracy and speed levels comparable to those of humans in some fruit detection and automatic harvesting fields. This paper presents a comprehensive overview and review of fruit detection and recognition based on DL for automatic harvesting from 2018 up to now. We focus on the current challenges affecting fruit detection performance for automatic harvesting: the scarcity of high-quality fruit datasets, fruit detection of small targets, fruit detection in occluded and dense scenarios, fruit detection of multiple scales and multiple species, and lightweight fruit detection models. In response to these challenges, we propose feasible solutions and prospective future development trends. Future research should prioritize addressing these current challenges and improving the accuracy, speed, robustness, and generalization of fruit vision detection systems, while reducing the overall complexity and cost. This paper hopes to provide a reference for follow-up research in the field of fruit detection and recognition based on DL for automatic harvesting. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) super-resolution (SR) is a classical computer vision task that aims to accomplish the conversion of images from lower to higher resolutions. With the booming development of deep learning (DL) technology, more and more researchers are dedicated to the research of image SR techniques based on DL and have made remarkable progress. However, no scholar has provided a comprehensive review of the field. As a response, in this paper we aim to supply a comprehensive summary of the DL-based SR techniques for HSI, including upsampling frameworks, upsampling methods, network design, loss functions, representative works with different strategies, and future directions, in which we design several sets of comparative experiments for the advantages and limitations of two-dimensional convolution and three-dimensional convolution in the field of HSI SR and analyze the experimental results in depth. In addition, the paper also briefly discusses the secondary foci such as common datasets, evaluation metrics, and traditional SR algorithms. To the best of our knowledge, this paper is the first review on DL-based HSI SR. [ABSTRACT FROM AUTHOR]

Strawberry (Fragaria × ananassa Duch.) has been widely accepted as the "Queen of Fruits". It has been identified as having high levels of vitamin C and antioxidants that are beneficial for maintaining cardiovascular health and maintaining blood sugar levels. The implementation of advanced techniques like precision agriculture (PA) is crucial for enhancing production compared to conventional farming methods. In recent years, the successful application of deep learning models was represented by convolutional neural networks (CNNs) in a variety of disciplines of computer vision (CV). Due to the dearth of a comprehensive and detailed discussion on the application of deep learning to strawberry cultivation, a particular review of recent technologies is needed. This paper provides an overview of recent advancements in strawberry cultivation utilizing Deep Learning (DL) techniques. It provides a comprehensive understanding of the most up-to-date techniques and methodologies used in this field by examining recent research. It also discusses the recent advanced variants of the DL model, along with a fundamental overview of CNN architecture. In addition, techniques for fine-tuning DL models have been covered. Besides, various strawberry-planting-related datasets were examined in the literature, and the limitations of using research models for real-time research have been discussed. [ABSTRACT FROM AUTHOR]

Aortic stenosis (AS) is a critical cardiovascular condition that necessitates precise diagnosis for effective patient care. Despite a limited dataset comprising only 202 images, our study employs transfer learning to investigate the efficacy of five convolutional neural network (CNN) models, coupled with advanced computer vision techniques, in accurately classifying AS. The VGG16 model stands out among the tested models, achieving 95% recall and F1-score. To fortify the model's robustness and generalization, we implement various data augmentation techniques, including translation, rotation, flip, and brightness adjustment. These techniques aim to capture real-world image variations encountered in clinical settings. Validation, conducted using authentic data from Hospital Santa Maria, not only affirms the clinical applicability of our model but also highlights the potential to develop robust models with a limited number of images. The models undergo training after the images undergo a series of computer vision and data augmentation techniques, as detailed in this paper. These techniques augment the size of our dataset, contributing to improved model performance. In conclusion, our study illuminates the potential of AI-driven AS detection in MRI scans. The integration of transfer learning, CNN models, and data augmentation yields high accuracy rates, even with a small dataset, as validated in real clinical cases. [ABSTRACT FROM AUTHOR]

Image stitching involves combining multiple images of the same scene captured from different viewpoints into a single image with an expanded field of view. While this technique has various applications in computer vision, traditional methods rely on the successive stitching of image pairs taken from multiple cameras. While this approach is effective for organized camera arrays, it can pose challenges for unstructured ones, especially when handling scene overlaps. This paper presents a deep learning-based approach for stitching images from large unstructured camera sets covering complex scenes. Our method processes images concurrently by using the SandFall algorithm to transform data from multiple cameras into a reduced fixed array, thereby minimizing data loss. A customized convolutional neural network then processes these data to produce the final image. By stitching images simultaneously, our method avoids the potential cascading errors seen in sequential pairwise stitching while offering improved time efficiency. In addition, we detail an unsupervised training method for the network utilizing metrics from Generative Adversarial Networks supplemented with supervised learning. Our testing revealed that the proposed approach operates in roughly ∼1/7th the time of many traditional methods on both CPU and GPU platforms, achieving results consistent with established methods. [ABSTRACT FROM AUTHOR]

Background: Breast cancer is arguably one of the leading causes of death among women around the world. The automation of the early detection process and classification of breast masses has been a prominent focus for researchers in the past decade. The utilization of ultrasound imaging is prevalent in the diagnostic evaluation of breast cancer, with its predictive accuracy being dependent on the expertise of the specialist. Therefore, there is an urgent need to create fast and reliable ultrasound image detection algorithms to address this issue. Methods: This paper aims to compare the efficiency of six state-of-the-art, fine-tuned deep learning models that can classify breast tissue from ultrasound images into three classes: benign, malignant, and normal, using transfer learning. Additionally, the architecture of a custom model is introduced and trained from the ground up on a public dataset containing 780 images, which was further augmented to 3900 and 7800 images, respectively. What is more, the custom model is further validated on another private dataset containing 163 ultrasound images divided into two classes: benign and malignant. The pre-trained architectures used in this work are ResNet-50, Inception-V3, Inception-ResNet-V2, MobileNet-V2, VGG-16, and DenseNet-121. The performance evaluation metrics that are used in this study are as follows: Precision, Recall, F1-Score and Specificity. Results: The experimental results show that the models trained on the augmented dataset with 7800 images obtained the best performance on the test set, having 94.95 ± 0.64%, 97.69 ± 0.52%, 97.69 ± 0.13%, 97.77 ± 0.29%, 95.07 ± 0.41%, 98.11 ± 0.10%, and 96.75 ± 0.26% accuracy for the ResNet-50, MobileNet-V2, InceptionResNet-V2, VGG-16, Inception-V3, DenseNet-121, and our model, respectively. Conclusion: Our proposed model obtains competitive results, outperforming some state-of-the-art models in terms of accuracy and training time. [ABSTRACT FROM AUTHOR]

Scene text detection has become a popular topic in computer vision research. Most of the current research is based on deep learning, using Convolutional Neural Networks (CNNs) to extract the visual features of images. However, due to the limitations of convolution kernel size, CNNs can only extract local features of images with small perceptual fields, and they cannot obtain more global features. In this paper, to improve the accuracy of scene text detection, a feature enhancement module is added to the text detection model. This module acquires global features of an image by computing the multi-headed self-attention of the feature map. The improved model extracts local features using CNNs, while extracting global features through the feature enhancement module. The features extracted by both of these are then fused to ensure that visual features at different levels of the image are extracted. A shifted window is used in the calculation of the self-attention, which reduces the computational complexity from the second power of the input image width-height product to the first power. Experiments are conducted on the multi-oriented text dataset ICDAR2015 and the multi-language text dataset MSRA-TD500. Compared with the pre-improvement method DBNet, the F1-score improves by 0.5% and 3.5% on ICDAR2015 and MSRA-TD500, respectively, indicating the effectiveness of the model improvement. [ABSTRACT FROM AUTHOR]

Deep Learning (DL) has provided a significant breakthrough in many areas of research and industry. The development of Convolutional Neural Networks (CNNs) has enabled the improvement of computer vision-based techniques, making the information gathered from cameras more useful. For this reason, recently, studies have been carried out on the use of image-based DL in some areas of people's daily life. In this paper, an object detection-based algorithm is proposed to modify and improve the user experience in relation to the use of cooking appliances. The algorithm can sense common kitchen objects and identify interesting situations for users. Some of these situations are the detection of utensils on lit hobs, recognition of boiling, smoking and oil in kitchenware, and determination of good cookware size adjustment, among others. In addition, the authors have achieved sensor fusion by using a cooker hob with Bluetooth connectivity, so it is possible to automatically interact with it via an external device such as a computer or a mobile phone. Our main contribution focuses on supporting people when they are cooking, controlling heaters, or alerting them with different types of alarms. To the best of our knowledge, this is the first time a YOLO algorithm has been used to control the cooktop by means of visual sensorization. Moreover, this research paper provides a comparison of the detection performance among different YOLO networks. Additionally, a dataset of more than 7500 images has been generated and multiple data augmentation techniques have been compared. The results show that YOLOv5s can successfully detect common kitchen objects with high accuracy and fast speed, and it can be employed for realistic cooking environment applications. Finally, multiple examples of the identification of interesting situations and how we act on the cooktop are presented. [ABSTRACT FROM AUTHOR]

Transformer models have achieved great results in the field of computer vision over the past 2 years, drawing attention from within the field of remote sensing. However, there are still relatively few studies on this model in the field of remote sensing. Which method is more suitable for remote-sensing segmentation? In particular, how do different transformer models perform in the face of high-spatial resolution and the multispectral resolution of remote-sensing images? To explore these questions, this paper presents a comprehensive comparative analysis of three mainstream transformer models, including the segmentation transformer (SETRnet), SwinUnet, and TransUnet, by evaluating three aspects: a visual analysis of feature-segmentation results, accuracy, and training time. The experimental results show that the transformer structure has obvious advantages for the feature-extraction ability of large-scale remote-sensing data sets and ground objects, but the segmentation performance of different transfer structures in different scales of remote-sensing data sets is also very different. SwinUnet exhibits better global semantic interaction and pixel-level segmentation prediction on the large-scale Potsdam data set, and the SwinUnet model has the highest accuracy metrics for KAPPA, MIoU, and OA in the Potsdam data set, at 76.47%, 63.62%, and 85.01%, respectively. TransUnet has better segmentation results in the small-scale Vaihingen data set, and the three accuracy metrics of KAPPA, MIoU, and OA are the highest, at 80.54%, 56.25%, and 85.55%, respectively. TransUnet is better able to handle the edges and details of feature segmentation thanks to the network structure together built by its transformer and convolutional neural networks (CNNs). Therefore, TransUnet segmentation accuracy is higher when using a small-scale Vaihingen data set. Compared with SwinUnet and TransUnet, the segmentation performance of SETRnet in different scales of remote-sensing data sets is not ideal, so SETRnet is not suitable for the research task of remote-sensing image segmentation. In addition, this paper discusses the reasons for the performance differences between transformer models and discusses the differences between transformer models and CNN. This study further promotes the application of transformer models in remote-sensing image segmentation, improves the understanding of transformer models, and helps relevant researchers to select a more appropriate transformer model or model improvement method for remote-sensing image segmentation. [ABSTRACT FROM AUTHOR]

The application of Transformer in computer vision has had the most significant influence of all the deep learning developments over the past five years. In addition to the exceptional performance of convolutional neural networks (CNN) in hyperspectral image (HSI) classification, Transformer has begun to be applied to HSI classification. However, for the time being, Transformer has not produced satisfactory results in HSI classification. Recently, in the field of image classification, the creators of Sequencer have proposed a Sequencer structure that substitutes the Transformer self-attention layer with a BiLSTM2D layer and achieves satisfactory results. As a result, this paper proposes a unique network called SquconvNet, that combines CNN with Sequencer block to improve hyperspectral classification. In this paper, we conducted rigorous HSI classification experiments on three relevant baseline datasets to evaluate the performance of the proposed method. The experimental results show that our proposed method has clear advantages in terms of classification accuracy and stability. [ABSTRACT FROM AUTHOR]

This paper presents an extensive research carried out for enhancing the performances of convolutional neural network (CNN) object detectors applied to municipal waste identification. In order to obtain an accurate and fast CNN architecture, several types of Single Shot Detectors (SSD) and Regional Proposal Networks (RPN) have been fine-tuned on the TrashNet database. The network with the best performances is executed on one autonomous robot system, which is able to collect detected waste from the ground based on the CNN feedback. For this type of application, a precise identification of municipal waste objects is very important. In order to develop a straightforward pipeline for waste detection, the paper focuses on boosting the performance of pre-trained CNN Object Detectors, in terms of precision, generalization, and detection speed, using different loss optimization methods, database augmentation, and asynchronous threading at inference time. The pipeline consists of data augmentation at the training time followed by CNN feature extraction and box predictor modules for localization and classification at different feature map sizes. The trained model is generated for inference afterwards. The experiments revealed better performances than all other Object Detectors trained on TrashNet or other garbage datasets with a precision of 97.63% accuracy for SSD and 95.76% accuracy for Faster R-CNN, respectively. In order to find the optimal higher and lower bounds of our learning rate where the network is actually learning, we trained our model for several epochs, updating the learning rate after each epoch, starting from 1 × 10−10 and decreasing it until reaching 1 × 10−1. [ABSTRACT FROM AUTHOR]

Human Activity Recognition (HAR) has become an active field of research in the computer vision community. Recognizing the basic activities of human beings with the help of computers and mobile sensors can be beneficial for numerous real-life applications. The main objective of this paper is to recognize six basic human activities, viz., jogging, sitting, standing, walking and whether a person is going upstairs or downstairs. This paper focuses on predicting the activities using a deep learning technique called Convolutional Neural Network (CNN) and the accelerometer present in smartphones. Furthermore, the methodology proposed in this paper focuses on grouping the data in the form of nodes and dividing the nodes into three major layers of the CNN after which the outcome is predicted in the output layer. This work also supports the evaluation of testing and training of the two-dimensional CNN model. Finally, it was observed that the model was able to give a good prediction of the activities with an average accuracy of 89.67%. Considering that the dataset used in this research work was built with the aid of smartphones, coming up with an efficient model for such datasets and some futuristic ideas pose open challenges in the research community. [ABSTRACT FROM AUTHOR]

Development of deep learning has led to progress in computer vision, including metric learning tasks such as image retrieval, through convolutional neural networks. In image retrieval, the metric distance (i.e., the similarity) between the images needs to be computed and then compared to return similar images. Global descriptors are good at extracting holistic features of an image, such as the overall shape of the main object and the silhouette. On the other hand, the local features extract the detailed features which the model uses to help classify similar images together. This paper proposes a descriptor mixer which takes advantage of both local and global descriptors (group of features combined into one) as well as different types of global descriptors for an effect of a lighter version of an ensemble of models (i.e., fewer parameters and smaller model size than those of actual ensemble of networks). As a result, the model's performance improved about 1.36% (recall @ 32) when the combination of the descriptors were used. We empirically found out that the combination of GeM and MAC achieved the highest performance. [ABSTRACT FROM AUTHOR]

The existence of conveyor foreign objects poses a serious threat to the service life of conveyor belts, which will cause abnormal damage or even tearing, so fast and effective detection of conveyor foreign objects is of great significance to ensure the safe and efficient operation of belt conveyors. Considering the need for the foreign object detection algorithm to operate in edge computing devices, this paper proposes a hybrid compression method that integrates network sparse, structured pruning, and knowledge distillation to compress the network parameters and calculations. Combined with a Yolov5 network for practice, three structured pruning strategies are specifically proposed, all of which are proven to have achieved a good compression effect. The experiment results show that under the pruning rate of 0.9, the proposed three pruning strategies can achieve more than 95% compression for network parameters, more than 90% compression for the computation, and more than 90% compression for the size of the network model, and the optimized network is able to accelerate inference on both Central Processing Unit (CPU) and Graphic Processing Unit (GPU) hardware platforms, with a maximum speedup of 70.3% on the GPU platform and 157.5% on the CPU platform, providing an excellent real-time performance but also causing a large accuracy loss. In contrast, the proposed method balances better real-time performance and detection accuracy (>88.2%) when the pruning rate is at 0.6~0.9. Further, to avoid the influence of motion blur, a method of introducing prior knowledge is proposed to improve the resistance of the network, thus strongly ensuring the detection effect. All the technical solutions proposed are of great significance in promoting the intelligent development of coal mine equipment, ensuring the safe and efficient operation of belt conveyors, and promoting sustainable development. [ABSTRACT FROM AUTHOR]