Search

Transformers are Deep Neural Networks (DNN) that utilize a self-attention mechanism to capture contextual relationships within sequential data. Unlike traditional neural networks and variants of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM), Transformer models excel at managing long dependencies among input sequence elements and facilitate parallel processing. Consequently, Transformer-based models have garnered significant attention from researchers in the field of artificial intelligence. This is due to their tremendous potential and impressive accomplishments, which extend beyond Natural Language Processing (NLP) tasks to encompass various domains, including Computer Vision (CV), audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published, spotlighting the Transformer's contributions in specific fields, architectural disparities, or performance assessments, there remains a notable absence of a comprehensive survey paper that encompasses its major applications across diverse domains. Therefore, this paper addresses this gap by conducting an extensive survey of proposed Transformer models spanning from 2017 to 2022. Our survey encompasses the identification of the top five application domains for Transformer-based models, namely: NLP, CV, multi-modality, audio and speech processing, and signal processing. We analyze the influence of highly impactful Transformer-based models within these domains and subsequently categorize them according to their respective tasks, employing a novel taxonomy. Our primary objective is to illuminate the existing potential and future prospects of Transformers for researchers who are passionate about this area, thereby contributing to a more comprehensive understanding of this groundbreaking technology. • The paper presents a comprehensive survey on transformers for deep learning tasks. • The paper conducts a thorough analysis on highly effective models in five domains. • The paper classifies the models based on respective tasks using a proposed taxonomy. • The characteristics of the surveyed models are deeply explored and analyzed. • Future directions and challenges for transformer-based models are deciphered. [ABSTRACT FROM AUTHOR]

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras to record videos and images for understanding fish life and ecology, as well as for preserve the environment. There are currently many permanent underwater camera systems deployed at different places around the globe. In addition, there exists numerous studies that use temporary cameras to survey fish habitats. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. Deep Learning (DL) is a cutting-edge Artificial Intelligence (AI) technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localisation, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring. • Marine environments and fish habitats are widely surveyed using underwater cameras. • Deep learning (DL) offers an effective solution for processing underwater videos. • We write a tutorial on the key concepts and steps of DL for underwater monitoring. • We provide a survey of key DL-based visual fish monitoring methods/datasets. • We shed light on challenges/opportunities of DL for underwater fish monitoring. [ABSTRACT FROM AUTHOR]

Optical Character Recognition (OCR) is a method to convert a scanned photo of"handwritten character recognition (HCR) or printed character recognition (PCR)" into a form of digital text. HCR is a version of OCR, which is remarkably modeled to identify handwritten text, while PCR aims at printed text identification. The identification of handwritten characters and digits is more complicated as compared to PCR because of the diversities in human writing styles, stokes, thickness, and curves of characters. Similarly, achievements in several computer vision tasks consider the Convolutional Neural Networks (CNN) to give an end-to-end solution for HCR with huge success. However, the process of significant feature learning for the identification of images is complicated with little data. Hence, this paper aims to develop a new handwritten character and digit recognition model with the incorporation of a deep learning strategy. Initially, the data related to Indian languages are collected from the standard benchmark datasets. Then, the collected data are given into the feature extraction phase 1, where the ResNet 151 is used for extracting the feature set 1. Similarly, the data gathered are considered in the feature extraction phase 2, where the Optimal Ensemble Pattern extraction approach is developed with Local Binary Pattern (LBP), Local Gradient Patterns (LGP), Local Tetra Pattern (LTrP), and Local Vector Pattern (LVP) for extracting the significant patterns from the language data. These extracted patterns are given into the ResNet 151 for getting the feature set 2. Here, the features from ResNet 151 get optimized with the enhanced optimization algorithm with Fitness-based Sail Fish Optimizer (F-SFO). The obtained feature set 1 and optimal feature set 2 are concatenated for performing final recognition. At last, the HCR is done with the help of developed Bi-LSTM-DNN to achieve the enhanced and accurate recognition of handwritten characters of the Indian languages. The performances of character recognition are further improved with the parameter optimization in Bi-LSTM-DNN with the same enhanced F-SFO. Overall result analysis, the accuracy of the designed method attains 95.12%. [ABSTRACT FROM AUTHOR]

3D point clouds are a crucial type of data collected by LiDAR sensors and widely used in transportation applications due to its concise descriptions and accurate localization. Deep neural networks (DNNs) have achieved remarkable success in processing large amount of disordered and sparse 3D point clouds, especially in various computer vision tasks, such as pedestrian detection and vehicle recognition. Among all the learning paradigms, Self-Supervised Learning (SSL), an unsupervised training paradigm that mines effective information from the data itself, is considered as an essential solution to solve the time-consuming and labor-intensive data labeling problems via smart pre-training task design. This paper provides a comprehensive survey of recent advances on SSL for point clouds. We first present an innovative taxonomy, categorizing the existing SSL methods into four broad categories based on the pretexts' characteristics. Under each category, we then further categorize the methods into more fine-grained groups and summarize the strength and limitations of the representative methods. We also compare the performance of the notable SSL methods in literature on multiple downstream tasks on benchmark datasets both quantitatively and qualitatively. Finally, we propose a number of future research directions based on the identified limitations of existing SSL research on point clouds. [ABSTRACT FROM AUTHOR]

Breakthrough performances have been achieved in computer vision by utilizing deep neural networks. In this paper we propose to use random forest to classify image representations obtained by concatenating multiple layers of learned features of deep convolutional neural networks for scene classification. Specifically, we first use deep convolutional neural networks pre-trained on the large-scale image database Places to extract features from scene images. Then, we concatenate multiple layers of features of the deep neural networks as image representations. After that, we use random forest as the classifier for scene classification. Moreover, to reduce feature redundancy in image representations we derived a novel feature selection method for selecting features that are suitable for random forest classification. Extensive experiments are conducted on two benchmark datasets, i.e. MIT-Indoor and UIUC-Sports. Obtained results demonstrated the effectiveness of the proposed method. The contributions of the paper are as follows. First, by extracting multiple layers of deep neural networks, we can explore more information of image contents for determining their categories. Second, we proposed a novel feature selection method that can be used to reduce redundancy in features obtained by deep neural networks for classification based on random forest. In particular, since deep learning methods can be used to augment expert systems by having the systems essentially training themselves, and the proposed framework is general, which can be easily extended to other intelligent systems that utilize deep learning methods, the proposed method provide a potential way for improving performances of other expert and intelligent systems. [ABSTRACT FROM AUTHOR]

• An automated system for railgun launcher damage detection is proposed. • A new data set of railgun inner bore damage is obtained by the designed device. • Adaptive data augmentation and focal loss are used to balance the uneven data set. • YOLOv5 and SOLOv2 are used for the detection and shape extraction of damage. • Results on the data set show the effectiveness of the proposed methods. The inner bore damage affects the launch performance and service life of electromagnetic railgun launcher. Detection and observation of railgun inner bore damage contribute to the study on mechanism and development rules of railgun damage. This paper analyzes five types of typical railgun inner bore damage. Based on the detection requirement for the damages, this paper proposes an automated damage detection system for the inner bore of electromagnetic railgun launcher consisting of data acquisition device and detection algorithms. The proposed device can step inside the inner bore of the railgun launcher to take photos of the inner bore surface automatically. We use the images obtained by the proposed device to build a data set for the training and verification of the detection algorithms. The object detection algorithm You Only Look Once v5 (YOLOv5) is utilized to achieve the rapid detection of railgun inner bore damages. We introduce the adaptive data augmentation and the focal loss to balance out the uneven category distribution of our data set. The result proves that our YOLOv5 model reaches the state-of-the-art level in the railgun inner bore damage detection task, with its mean Average Precision (mAP) of 0.659 and detection speed of 47.6 frames per second (fps). We choose Segmenting Objects by Locations v2 (SOLOv2) to extract the shape of the damage, with the Average Precision of 0.631. We further achieve damage statistics and the model visualization of damage distribution. The experimental results show that the proposed detection system meets the requirements of rapid detection and accurate feature extraction. It provides researchers with an approach for the study of railgun inner bore damage mechanism. [ABSTRACT FROM AUTHOR]

Highlights • Scene categorization is performed with multi-views and multi-levels of abstraction. • CNNs and LSTMs are coordinated to categorize scene images. • CNNs are used to generate features of multi-levels of abstraction. • LSTMs are used to accommodate multiple image views. Abstract Due to complexities of scene images, scene categorization is a challenging task in the computer vision community. To categorize scene images effectively, in this paper, we propose to coordinate Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs) to perform scene categorization with multi-views and multi-levels of abstraction. Specifically, to utilize the complementary properties of features of different levels of abstraction, we employ CNNs to extract features of multi-levels of abstraction based on its hierarchical structure. Furthermore, in order to deal with variations in scene image contents, we represent each image with multiple views, and in order to take correlation between image views into consideration, we treat image view features from the same image as a sequence and employ Long Short-Term Memory networks (LSTMs) to perform classification. Based on the proposed method, information of multi-views and multi-levels of abstraction can be made full use of in a single framework. We evaluate the proposed method on two challenging scene datasets, MIT indoor scene 67 and SUN 397. Obtained results demonstrate the effectiveness of utilizing CNNs and LSTMs to categorize scene images with multi-views and multi-levels of abstraction. Experiments on comparison to state-of-the-art methods show that the proposed method outperforms all the other methods used for comparison. [ABSTRACT FROM AUTHOR]

Object detection and recognition are challenging computer vision tasks receiving great attention due to the large number of applications. This work focuses on the detection/recognition of products in supermarket shelves; this framework has a number of practical applications such as providing additional product/price information to the user or guiding visually impaired customers during shopping. The automatic creation of planograms (i.e., actual layout of products on shelves) is also useful for commercial analysis and management of large stores. Although in many object detection/recognition contexts it can be assumed that training images are representative of the real operational conditions, in our scenario such assumption is not realistic because the only training images available are acquired in well-controlled conditions. This gap between the training and test data makes the object detection and recognition tasks far more complex and requires very robust techniques. In this paper we prove that good results can be obtained by exploiting color and texture information in a multi-stage process: pre-selection, fine-selection and post processing. For fine-selection we compared a classical Bag of Words technique with a more recent Deep Neural Networks approach and found interesting outcomes. Extensive experiments on datasets of varying complexity are discussed to highlight the main issues characterizing this problem, and to guide toward the practical development of a real application. [ABSTRACT FROM AUTHOR]

Current state-of-the-art approaches for human hand detection have achieved great success by making good use of multiscale and contextual information but still remain unsatisfactory for hand detection, especially in complex scenarios. The main reason is that there are some parts similar to human hands, such as wrists, faces and feet. Simply using contextual information makes it difficult to address these problems. In this paper, we propose a Context Attention Feature Pyramid Network (CA-FPN) for human hand detection. In this method, a novel Context Attention Module (CAM) is inserted into the feature pyramid networks. The CAM is designed to capture relative contextual information for hands and build long-range dependencies around hands. Our CA-FPN can achieve state-of-the-art results on two public hand detection datasets: the Oxford and Vision for Intelligent and Applications (VIVA) datasets. Furthermore, the inference time of our CA-FPN is approximately 8.5 FPS on one TITAN X GPU, indicating that it can be used in real-time applications. Besides, the CAM helps improve head detection on the HollywoodHeads dataset, demonstrating its robustness in other detection tasks. The code has been made available at https://github.com/IC-LAB/CA-FPN. • A context attention module to utilize dilated and typical convolutions. • The proposed module with FPN achieves top performance on challenging hand detection. • The proposed module can be applied to other vision tasks, such as head detection. [ABSTRACT FROM AUTHOR]

Abstract: A novel segment confidence-based binary segmentation (SCBS) for cursive handwritten words is presented in this paper. SCBS is a character segmentation strategy for off-line cursive handwriting recognition. Unlike the approaches in the literature, SCBS is an unordered segmentation approach. SCBS is repetition of binary segmentation and fusion of segment confidence. Each repetition generates only one final segmentation point. The binary segmentation module is a contour tracing algorithm to find a segmentation path to divide a segment into two segments. A set of segments before binary segmentation is called pre-segments, and a set of segments after binary segmentation is called post-segments. SCBS uses over-segmentation technique to generate suspicious segmentation points on pre-segments. On each suspicious segmentation point, binary segmentation is performed and the highest fusion value is recorded. If the highest fusion value is greater than the one of pre-segments, the suspicious segmentation point becomes the final segmentation point for the iteration. If not, no more segmentation is required. Segment confidence is obtained by fusing mean character, lexical and shape confidences. The proposed approach has been evaluated on local and benchmark (CEDAR) databases. [Copyright &y& Elsevier]

Abstract: Texture recognition have received tremendous attentions in the past decades, due to its wide applications in computer vision and pattern recognition. For various applications, formulating texture features in distributional forms can sometimes provide meaningful representation than in numerical forms. In this paper, a generalized probabilistic decision-based neural network (GPDNN), based on a novel methodology for the measurement of the difference between two distributions, is proposed for texture recognition. Based on a two-layer pyramid-type network structure, the proposed GPDNN receives texture data via 2-D grid input nodes, and outputs the classification and/or retrieval results at the top layer node. Our prototype system demonstrates a successful utilization of GPDNN to the texture recognition on 40 texture images selected from the MIT Vision Texture (VisTex) database. Regarding the performance, experiment results show that (1) based on the proposed distribution difference measurement method, the texture retrieval accuracy is improved from 77% to 82% by comparing with some recently published leading methods, and (2) the proposed GPDNN has significant improvements in classification accuracy from 82.2% to 90.1% and retrieval accuracy from 79.9% to 88.6% by comparing with traditional approaches. [ABSTRACT FROM AUTHOR]

Abstract: The polygonal approximation is an important topic in the area of pattern recognition, computer graphics and computer vision. This paper firstly proposes a new computational energy function to properly express the objective of the polygonal approximation problem based on competitive Hopfield neural network (CHNN), and then proposes a stochastic CHNN (SCHNN) by introducing stochastic dynamics into the CHNN to help the network escape from local minima. In order to further improve the performance of the SCHNN, a multi-start strategy or re-start mechanism is introduced. The multi-start strategy or re-start mechanism super-imposed on the SCHNN is characterized by alternating phases of cooling and reheating the stochastic dynamics, thus provides a means to achieve an effective dynamic or oscillating balance between intensification and diversification during the search. The proposed multi-start SCHNN (MS-SCHNN) is tested on a set of benchmark problems and several large size test instances. Simulation results show that the proposed MS-SCHNN is better than or competitive with several typical neural network algorithms such as CHNN and transiently chaotic neural network, metaheuristic algorithms such as genetic algorithms, and 12 commonly referred state-of-the-art algorithms specifically developed for the polygonal approximation. Furthermore, the chain codes and results of the proposed algorithm for the large size curves are also provided. [ABSTRACT FROM AUTHOR]

• This method achieves the speed-accuracy trade-off. • The paper predicts the region of interest by perceiving the motion trend of target. • The influence of the background is reduced by feeding the target ROI. • The appearance changes of target are adapted by updating the reference dynamically. • Temporal stability is improved by the adaptive updating strategy of reference. Video object segmentation (VOS) is a research hotspot in the field of computer vision. Traditional video object segmentation methods based on deep learning have some problems such as difficulty in adapting to the change of object appearance and low segmentation speed. In this manuscript, we propose a robust VOS method based on motion-aware region of interest (ROI) prediction and adaptive reference updating. Firstly, based on the historical movement trajectory of target region to perceive motion trend dynamically, we predict the motion-aware ROI of target object in the current frame and use it as the input of segmentation network. Then, in order to adapt to the appearance changes of target in the video, the adaptive updating strategy of reference is given to dynamically update the reference frame during the segmentation process. Finally, VOS Siamese network is designed for fast segmentation. Experiments on three public benchmark datasets, DAVIS-2016 and DAVIS-2017, show that the proposed method performs better than the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• Monitoring the driver behavior is used for decreasing the risk of traffic accidents. • The driver behavior can be deduced from vehicle characteristics during driving. • Deep learning methods can use for analyzing driver behavior. Driver behavior monitoring system as Intelligent Transportation Systems (ITS) have been widely exploited to reduce the traffic accidents risk. Most previous methods for monitoring the driver behavior are rely on computer vision techniques. Such methods suffer from violation of privacy and the possibility of spoofing. This paper presents a novel yet efficient deep learning method for analyzing the driver behavior. We have used the driving signals, including acceleration, gravity, throttle, speed, and Revolutions Per Minute (RPM) to recognize five types of driving styles, including normal, aggressive, distracted, drowsy, and drunk driving. To take the advantages of successful deep neural networks on images, we learn a 2D Convolutional Neural Network (CNN) on images constructed from driving signals based on recurrence plot technique. Experimental results confirm that the proposed method can efficiently detect the driver behavior. [ABSTRACT FROM AUTHOR]

• Learning new embedding patterns. • Customizing solutions for suggested transform domains and attacks. • Enhancing security and robustness by diffusing watermark in a wide area of images. Due to the rapid growth of machine learning tools and specifically deep networks in various computer vision and image processing areas, applications of Convolutional Neural Networks for watermarking have recently emerged. In this paper, we propose a deep end-to-end diffusion watermarking framework (ReDMark) which can learn a new watermarking algorithm in any desired transform space. The framework is composed of two Fully Convolutional Neural Networks with residual structure which handle embedding and extraction operations in real-time. The whole deep network is trained end-to-end to conduct a blind secure watermarking. The proposed framework simulates various attacks as a differentiable network layer to facilitate end-to-end training. The watermark data is diffused in a relatively wide area of the image to enhance security and robustness of the algorithm. Comparative results versus recent state-of-the-art researches highlight the superiority of the proposed framework in terms of imperceptibility, robustness and speed. The source codes of the proposed framework are publicly available at Github 1 1 https://github.com/MahdiShAhmadi/ReDMark/tree/master/. [ABSTRACT FROM AUTHOR]