Search

Machine vision based planar grasping detection is challenging due to uncertainty about object shape, pose, size, etc. Previous methods mostly focus on predicting discrete gripper configurations, and may miss some ground-truth grasp postures. In this paper, a pixel-level grasp detection method is proposed, which uses deep neural network to predict pixel-level gripper configurations on RGB images. Firstly, a novel Oriented Arrow Representation model (OAR-model) is introduced to represent the gripper configuration of parallel-jaw and three-fingered gripper, which can partly improve the applicability to different grippers. Then, the Adaptive Grasping Attribute model (AGA-model) is proposed to adaptively represent the grasping attribute of objects, for resolving angle conflicts in training and simplifying pixel-level labeling. Lastly, the Adaptive Feature Fusion and Grasp Aware Network (AFFGA-Net) is proposed to predict pixel-level OAR-models on RGB images. AFFGA-Net improves the robustness in unstructured scenarios by using Hybrid Atrous Spatial Pyramid and Adaptive Decoder connected in sequence. On the public Cornell dataset and actual objects, our structure achieves 99.09\% and 98.0\% grasp detection accuracy respectively. In over 2,400 robotic grasp trials, our structure achieves average success rate of 98.77\% in single-object scenarios and 93.69\% in cluttered scenarios. Moreover, AFFGA-Net completes a grasp detection pipeline within 15 ms.

The Tsetlin Machine (TM) is a recent machine learning algorithm with several distinct properties, such as interpretability, simplicity, and hardware-friendliness. Although numerous empirical evaluations report on its performance, the mathematical analysis of its convergence is still open. In this article, we analyze the convergence of the TM with only one clause involved for classification. More specifically, we examine two basic logical operators, namely, the "IDENTITY"- and "NOT" operators. Our analysis reveals that the TM, with just one clause, can converge correctly to the intended logical operator, learning from training data over an infinite time horizon. Besides, it can capture arbitrarily rare patterns and select the most accurate one when two candidate patterns are incompatible, by configuring a granularity parameter. The analysis of the convergence of the two basic operators lays the foundation for analyzing other logical operators. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained state-of-the-art performance on several pattern recognition problems., Comment: 23 pages, 10 figures

The analog circuit design of the memristive neural network (MNN), which can automatically perform the online learning algorithm, is an open question. In this paper, a memristive self-learning neuron circuit for implementing the online least mean square (LMS) algorithm is designed. Extending on the designed neuron circuit, the circuit implementation of the monolayer and multilayer neural network is proposed. The proposed neural network can automatically converge the output to the set target according to the input. The application-level validations of the circuits are done using pattern recognition and license plate detection. The performances of the designed MNN circuits and the effect of memristive variation are analyzed through PSpice simulations. The learning accuracy of the proposed circuit for license plate detection can reach 93%. Circuit simulation results reveal that the proposed MNN circuits can accelerate the training speed and have the tolerance to the variations of the memristor.

Hard landing is a typical flight safety incident, and interpretability plays an important role in flight safety research. However, existing studies failed to provide good interpretability of the reasons for hard landing incidents and suffer from low prediction accuracy. To address the above problems, in this paper we propose CurveCluster+, a curve clustering method based on quick access recorder (QAR) data for hard landing risk evaluation. Specifically, we first conduct an in-depth analysis on hard landing flights by comparing key QAR parameter curves with the group behavior, based on which we establish a two-level hierarchical classification of hard landing incidents according to the hard landing patterns. Then we extract curve-level features from key QAR parameters through interpolation and resampling. After that we turn the classic K-means clustering into a semi-supervised algorithm by incorporating some expert experience and apply it on the curve-level features to automatically recognize the hard landing patterns. Finally, we propose a risk evaluation model based on the clustering results to discover high-risk flights from normal ones. We evaluate our method on a QAR dataset of 37,943 Airbus 320 aircraft flights. The results show that compared with other state-of-the-art data-driven methods, CurveCluster+ provides strong interpretability of hard landing incidents and exhibits good performance in recognizing hard landing patterns (the overall accuracy of our method reaches up to 92.99%). Moreover, it only requires a handful of hard landing samples to discover high-risk flights from tremendous normal landing flights, which is critical for flight safety warnings.

Road traffic state estimation is an essential component of intelligent transportation systems (ITSs). However, road traffic state data collected by traffic detectors are often incomplete, which can cause problems across a variety of transportation applications, such as traffic state prediction and pattern recognition. We present GA-GAN (Graph Aggregate Generative Adversarial Network), consisting of graph sample and aggregate (GraphSAGE) and a generative adversarial network (GAN), to impute missing road traffic state data. Instead of using the original road network structure, which presents the spatial information to process a graph operation, we reconstruct the road network according to the correlation coefficients of road historical data. We utilize GraphSAGE to aggregate the temporal-spatial information from the neighbors of each road in the reconstructed road network. GAN is used to generate complete traffic state data from the extracted temporal-spatial information to achieve traffic state data imputation. To illustrate the efficient performance of the model, experiments are conducted on traffic data collected from California and Seattle, Washington, showing that the proposed model outperforms state-of-the-art methods.

Feature selection is a basic task for pattern recognition and classification. It enhances the performance of the classification algorithms with the help of removing the redundant features. Thanks to eliminating irrelevant features, the computational time is decreased. Thus, intensive works have been carried out in this area. This paper proposes a new standard error-based artificial bee colony (SEABC) algorithm for the feature selection problem, which is developed by integrating standard error-based new solution search mechanisms into the original artificial bee colony algorithm. The SEABC algorithm is used for feature selection. Shannon entropy function is used to serve as the objective function of the SEABC algorithm. Thirteen datasets are used from UCI machine learning datasets. Features are selected according to Shannon conditional entropy values and then a threshold process is implemented to find their best relevant subset. Support Vector Machines (SVMs) and k-Nearest Neighbor (KNN) are used as the optimal classifiers. The proposed SEABC algorithm is compared with genetic algorithm (GA), particle swarm optimization (PSO), ABC, improved ABC (I-ABC), Gbest-guided ABC (GABC), and PS-ABC algorithms. In general, it is observed that the SEABC algorithm achieves better classification results than other well-known algorithms.

Decision tree is a widely used non-parametric technique in machine learning, data mining and pattern recognition. It is simple to understand and interpret, however it faces challenges such as handling higher dimensional and class imbalanced datasets, over-fitting and instability. To overcome some of these issues, vertical partitioning approaches like serial partitioning, theme based partitioning are used in the literature. A vertical partitioning approach divides the feature set into subsets of features (blocks) and makes use of these subsets for subsequent tasks. In this work, we use the ideas of music rhythm tree to propose a novel vertical partitioning technique. It orders the features based on the average correlation strength of the features before partitioning the feature set. The proposed method is proved to be superior by showing an average of 13.8 % , 6 % , 9.8 % , 19.7 % , 9.4 % , and 29.4 % higher classification accuracy over C4.5, Random Forest, Bagging, Adaboost, an ensemble technique and a vertical partitioning technique respectively. Our empirical results on 15 datasets demonstrate that the proposed vertical partitioning method is more stable and better in handling class-imbalanced data. Finally, some popular statistical tests are conducted to validate the statistical significance of the results of the proposed method.

The interest on graph matching has not stopped growing since the late seventies. The basic idea of graph matching consists of generating graph representations of different data or structures and compare those representations by searching correspondences between them. There are manifold techniques that have been developed to find those correspondences and the choice of one or another depends on the characteristics of the application of interest. These applications range from pattern recognition (e.g. biometric identification) to signal processing or artificial intelligence. One of the aspects that make graph matching so attractive is its ability to facilitate data analysis, and medical imaging is one of the fields that can benefit from this in a greater extent. The potential of graph matching to find similarities and differences between data acquired at different points in time shows its potential to improve diagnosis, follow-up of human diseases or any other of the clinical scenarios that require comparison between different datasets. In spite of the large amount of papers that were published in this field to the date there is no survey paper of graph matching for clinical applications. This survey aims to fill this gap.

Bearing failure often occurs in a doubly fed induction generator (DFIG). The fault diagnosis method based on current signals has been attracted much attention. This paper proposes different discrete digital models and its measure functions employing random theory. Firstly, based on raw current signals with the probability density function (PDF), a distributed discrete digital model is proposed; to avoid finding the PDF in the raw current signals, a discrete digital model of moment feature, and a discrete digital model of raw data are proposed. Then, six measurement functions are proposed as features of the discrete digital model for pattern recognition and condition monitoring of bearing. Finally, the effectiveness of the current method is demonstrated by comparing different signal processing methods.

Fish classification (FC) is an expansively studied problem in the domains of image segmentation, pattern recognition, and information retrieval. It has been applied in a countless number of domains including target marketing. Meanwhile, governments are obliged to maintain the fish supply and balance between the ecosystem, commercial, agriculture field, marine scientists, and industrial arena of fish including the nutrition and canning factories. The various FC techniques performance is compared relying on the availability of preprocessing and feature extraction methods, the number of extracted features and classification accuracy, the number of fish families/species recognized. This survey also reviewed the use of Databases such as Fish4-Knowledge (F4K), knowledge database, and Global Information System (GIS) on Fishes and other FC databases. The study on preprocessing methods features extraction techniques and classifiers are gathered from recent works to enhance the understanding of the characteristics of preprocessing methods, features extraction techniques, and classifiers to guide future research directions and compensate for current research gaps.

Endowing ubiquitous robots with cognitive capabilities for recognizing emotions, sentiments, affects, and moods of humans in their context is an important challenge, which requires sophisticated and novel approaches of emotion recognition. Most studies explore data-driven pattern recognition techniques that are generally highly dependent on learning data and insufficiently effective for emotion contextual recognition. In this article, a hybrid model-based emotion contextual recognition approach for cognitive assistance services in ubiquitous environments is proposed. This model is based on: 1) a hybrid-level fusion exploiting a multilayer perceptron (MLP) neural-network model and the possibilistic logic and 2) an expressive emotional knowledge representation and reasoning model to recognize nondirectly observable emotions; this model exploits jointly the emotion upper ontology (EmUO) and the n-ary ontology of events HTemp supported by the NKRL language. For validation purposes of the proposed approach, experiments were carried out using a YouTube dataset, and in a real-world scenario dedicated to the cognitive assistance of visitors in a smart devices showroom. Results demonstrated that the proposed multimodal emotion recognition model outperforms all baseline models. The real-world scenario corroborates the effectiveness of the proposed approach in terms of emotion contextual recognition and management and in the creation of emotion-based assistance services.

One of the major reasons of death worldwide is Cardiovascular diseases. One of its type is Arrhythmia in which normal rhythm of heart is varied due to damage in heart muscles and electrolyte imbalance. To study cardiovascular disease, Electrocardiogram (ECG) signal is used that plays a significant role in identifying Arrhythmia. By using a combination of Multivariate Empirical Mode Decomposition (MEMD) and Artificial Neural Network (ANN), a hybrid technique is proposed in this research work to detect and classify Arrhythmia. MEMD, a promising technique that is used nowadays for denoising of multichannel signals, is used in this work. Two main features, i.e., RR interval and Heart Rate are extracted from the ECG signal for the detection of Arrhythmia. Using these features, Arrhythmia is classified into Tachycardia and Bradycardia. Classification is done using pattern recognition of ANN in which Multilayer feed forward neural network is used and trained through Back Propagation Algorithm. Results that are achieved using proposed technique is compared with previous existing techniques as in 2016 (Discrete Wavelet Transform) DWT and ANN gave 87.01% and in 2018 (Empirical Mode Decomposition) EMD and (Linear Discriminant Analysis) LDA gave 87% accuracy. As compare to these techniques, the proposed technique results show that it is an efficient technique that gives better accuracy of 89.8%.

Facial Attribute Classification (FAC) has attracted increasing attention in computer vision and pattern recognition. However, state-of-the-art FAC methods perform face detection/alignment and FAC independently. The inherent dependencies between these tasks are not fully exploited. In addition, most methods predict all facial attributes using the same CNN network architecture, which ignores the different learning complexities of facial attributes. To address the above problems, we propose a novel deep multi-task multi-label CNN, termed DMM-CNN, for effective FAC. Specifically, DMM-CNN jointly optimizes two closely-related tasks (i.e., facial landmark detection and FAC) to improve the performance of FAC by taking advantage of multi-task learning. To deal with the diverse learning complexities of facial attributes, we divide the attributes into two groups: objective attributes and subjective attributes. Two different network architectures are respectively designed to extract features for two groups of attributes, and a novel dynamic weighting scheme is proposed to automatically assign the loss weight to each facial attribute during training. Furthermore, an adaptive thresholding strategy is developed to effectively alleviate the problem of class imbalance for multi-label learning. Experimental results on the challenging CelebA and LFWA datasets show the superiority of the proposed DMM-CNN method compared with several state-of-the-art FAC methods.

T-spherical fuzzy set (TSFS) is a generalized version of the spherical fuzzy set (SFS) and picture fuzzy set (PFS) to manage awkward and unpredictable information in realistic decision issues. TSFS deals with yes, abstinence, no, and refusal type of fuzzy information. This manuscript aims to observe the drawbacks of some existing dice similarity measures (DSMs) and to propose some new DSMs in the environment of TSFSs. The validation of the new DSMs is proved. The defined DSMs are further extended to introduce some generalized DSMs (GDSMs) and their special cases are studied. Additionally, the TOPSIS method using the entropy measures (EMs) based on TSFSs is also explored and verified with the help of some examples. The proposed new GDSMs and TOPSIS method are applied to the problem of building material recognition, medical diagnosis, clustering, and the results obtained are investigated. A comparison of the new theory is established where the advancement of the proposed DSMs is elaborated under some conditions. The advantages of the new DSMs and the drawbacks of the previous DSMs of IFSs, PyFSs, and PFSs have been studied because of their applicability. The article is comprehensively summarized, and some possible future directions are stated.

In computer-based search systems, similarity plays a key role in replicating the human search process. Indeed, the human search process underlies many natural abilities such as image recovery, language comprehension, decision making, or pattern recognition. The search for images consists of establishing a correspondence between the available image and that sought by the user, by measuring the similarity between the images. Image search by content is generaly based on the similarity of the visual characteristics of the images. The distance function used to evaluate the similarity between images depends notonly on the criteria of the search but also on the representation of the characteristics of the image. This is the main idea of a content-based image retrieval (CBIR) system. In this article, first, we constructed type-2 beta fuzzy membership of descriptor vectors to help manage inaccuracy and uncertainty of characteristics extracted the feature of images. Subsequently, the retrieved images are ranked according to the novel similarity measure, noted type-2 fuzzy nearness measure (IT2FNM). By analogy to Type-2 Fuzzy Logic and motivated by near sets theory, we advanced a new fuzzy similarity measure (FSM) noted interval type-2 fuzzy nearness measure (IT-2 FNM). Then, we proposed three new IT-2 FSMs and we have provided mathematical justification to demonstrate that the proposed FSMs satisfy proximity properties (i.e. reflexivity, transitivity, symmetry, and overlapping). Experimental results generated using three image databases showing consistent and significant results.

In recent years, researchers have made a great success on the automatic classification and detection of malware utilizing machine learning methods. However, most machine learning based approaches over rely on the training samples such that a new malware family not belonging to the training set cannot be identified. To address such issue, we propose a soft relevance value (s-value), a new evaluating way of feature soft relevance that uses the mixed distance criterion to assess classified results. Specifically, we leverage the mixed distance criterion from pattern recognition to distinguish testing samples as a new family which is not labeled in training set. Finally, we evaluate how s-value can be used to distinguish and classify a new malware family with the malware datasets from the Research Prediction Competition of Microsoft Malware Classification Challenge and Windows (Kaggle). The experimental results show that, the train-ing time is approximately 12 hours, while the prediction time is only ∼0.5 second. Comparing against the Kaggle winner, our time costs for training and pprediction only occupy 16.7% and 3.8% of the winner.s time costs, respectively. The accuracy of classifying malware reaches 99.8%. Such results indicates that our proposed s-value achieves a balance in accuracy, training and prediction time, and outperforms the state-of-the-art machine learning based malware detection approaches. Besides, our method is able to identify new malware families that are not included in the training set.

Feature selection is important in both machine learning and pattern recognition. Successfully selecting informative features can significantly increase learning accuracy and improve result comprehensibility. Various methods have been proposed to identify informative features from high-dimensional data by removing redundant and irrelevant features to improve classification accuracy. In this article, we systematically survey existing sparse learning models for feature selection from the perspectives of individual sparse feature selection and group sparse feature selection, and analyze the differences and connections among various sparse learning models. Promising research directions and topics on sparse learning models are analyzed.

This article investigates hardware implementation of hierarchical temporal memory (HTM), a brain-inspired machine learning algorithm that mimics the key functions of the neocortex and is applicable to many machine learning tasks. Spatial pooler (SP) is one of the main parts of HTM, designed to learn the spatial information and obtain the sparse distributed representations (SDRs) of input patterns. The other part is temporal memory (TM) which aims to learn the temporal information of inputs. The memristor, which is an appropriate synapse emulator for neuromorphic systems, can be used as the synapse in SP and TM circuits. In this article, a memristor-based SP (MSP) circuit structure is designed to accelerate the execution of the SP algorithm. The presented MSP has properties of modeling both the synaptic permanence and the synaptic connection state within a single synapse, and on-device and parallel learning. Simulation results of statistic metrics and classification tasks on several real-world datasets substantiate the validity of MSP.

Background: Hand tremor is one of the consequences of MS disease degrading quality of patient’s life. Recently DBS is used as a prominent treatment to reduce this effect. Evaluation of this approach has significant importance because of the prevalence rate of disease.Objective: The purpose of this study was the nonlinear analysis of tremor signal in order to evaluate the quantitative effect of DBS on reducing MS tremor and differentiating between them using pattern recognition algorithms.Material and Methods:In this paper, nine features were extracted from the tremor signal. Through statistical analysis, the significance level of each feature was examined. Finally, tremor signals were categorized by SVM, weighted KNN and NN classifiers. The performance of methods was compared with an ROC graph.Results:The results have demonstrated that dominant frequency, maximum amplitude and energy of the first IMF, deviation of the direct path, sample entropy and fuzzy entropy have the potential to create a significant difference between the tremor signals. The classification accuracy rate of tremor signals in three groups for Weighted KNN, NN and SVM with Gaussian and Quadratic kernels resulted in 95.1%, 93.2%, 91.3% and 88.3%, respectively.conclusion: Generally, nonlinear and nonstationary analyses have a high potential for a quantitative and objective measure of MS tremor. Weighted KNN has shown the best performance of classification with the accuracy of more than 95%. It has been indicated that DBS has a positive influence on reducing the MS tremor. Therefore, DBS can be used in the objective improvement of tremor in MS patients.

Semantics and motion are two cues of essence for the success in video salient object detection. Most existing deep-learning based approaches extract semantic features by the use of only one fully convolutional network with simple stacked encoders. They simulate motion patterns of video objects with two consecutive frames being simultaneously fed into a convolutional LSTM network or a weights-sharing fully convolutional network. However, such approaches have the shortcomings of producing a coarse predicted saliency map or requiring significant computational overheads. In this paper, we present a novel approach with cascaded fully convolutional networks involving motion attention (abbreviated as CFCN-MA), to achieve real-time saliency detection in videos. Our key idea is to construct twofold fully convolutional networks in order to gain a saliency map from coarse to fine. We devise an optical flow-based motion attention mechanism to improve the prediction accuracy of the initial fully convolutional networks, using the popular FlowNet2-SD model that is efficient and effective for motion pattern recognition of distinctive objects in videos. This method can obtain a fine saliency map with a refined region of interest. Moreover, we propose a means for calculating attention-guided intersection-over-union loss (shortnamed as AIoU) to supervise the CFCN-MA model in learning a saliency map with both clear edge and complete structure. Our approach is evaluated on three popular benchmark datasets, namely DAVIS, ViSal and FBMS. Experimental results demonstrate that our method outperforms many state-of-the-art techniques while meeting the real-time demand at 27 fps.

This paper introduces the pipeline to extend the largest dataset in egocentric vision, EPIC-KITCHENS. The effort culminates in EPIC-KITCHENS-100, a collection of 100 hours, 20M frames, 90K actions in 700 variable-length videos, capturing long-term unscripted activities in 45 environments, using head-mounted cameras. Compared to its previous version (Damen in Scaling egocentric vision: ECCV, 2018), EPIC-KITCHENS-100 has been annotated using a novel pipeline that allows denser (54% more actions per minute) and more complete annotations of fine-grained actions (+128% more action segments). This collection enables new challenges such as action detection and evaluating the “test of time”—i.e. whether models trained on data collected in 2018 can generalise to new footage collected two years later. The dataset is aligned with 6 challenges: action recognition (full and weak supervision), action detection, action anticipation, cross-modal retrieval (from captions), as well as unsupervised domain adaptation for action recognition. For each challenge, we define the task, provide baselines and evaluation metrics. Published version Research at Bristol is supported by Engineering and Physical Sciences Research Council (EPSRC) Doctoral Training Program (DTP), EPSRC Fellowship UMPIRE (EP/T004991/1). Research at Catania is sponsored by Piano della Ricerca 2016-2018 linea di Intervento 2 of DMI, by MISE - PON I&C 2014-2020, ENIGMA project (CUP: B61B19000520008) and by MIUR AIM - Attrazione e Mobilita Internazionale Linea 1 - AIM1893589 - CUP E64118002540007.

This study presents an efficient sparse learning-based pattern recognition framework to recognize the discrete states of three emotions-happy, angry, and neutral emotion-using electroencephalogram (EEG) signals. In affective computing with massive spatiotemporal brainwave signals, a large number of features can be extracted to capture various information from multivariate brain data. However, it is often a challenge to model high-dimensional features efficiently in consideration of the intrinsic structure, such as channel location, feature group, time epoch, etc. In this study, features were extensively extracted from EEG signals and were applied on a structured sparse learning model to perform feature selection and classification simultaneously. An efficient ADMM-based algorithm with a closed-form solution was developed to solve the sparse group model. Experimental results show that the proposed method is capable of selecting a small number of important neural features to discriminate the three emotion states with high classification accuracy. With greatly enhanced interpretability and efficiency to learn neural signatures of brain activity from high-dimensional-feature, low-sample-size brain imaging data, the presented computational framework is promising for handling emotion recognition problems with high-dimensional brain imaging data.

Detecting the evolution of large-area landscape patterns using long-term remote-sensing images is helpful in supporting research on the relationship between landscape patterns and ecological processes, as well as the development of ecological process simulations and spatiotemporal interaction models. However, detection methods have generally been developed as separate applications, each with a separate type of landscape pattern change; remote-sensing images are acquired at epochal timesteps. Consequently, in practical applications, many omission changes for some types of pattern changes and inaccurate evolution time are presented in the detected map. In this article, state-and-evolution detection models (SEDMs) are promoted to obtain complete information about the evolution of landscape patterns based on yearly land cover data. In the proposed framework, we first define the major categories of landscape pattern changes to comprehensively reveal the characteristics of landscape pattern changes associated with real change cases. Next, a morphological rule-based pattern recognition approach is proposed for quantitative discrimination among these categories. This approach is then applied in annual land cover data to continuously detect landscape pattern evolution processes and evolution time. Finally, the detected evolution time in different evolution processes is applied to measure the timestep between two disparate types. The performances of the SEDMs are presented by Landsat-derived land cover evolution in Shanxi, China. The detected results are indirectly verified by the land cover conversion matrix and connect index, indicating strong robustness and generalization ability of the SEDMs.

Despite a historical focus on prosthetics, the incorporation of electromyography (EMG) sensors into less obtrusive wearable designs has recently gained attention as a potential human–computer interaction scheme for general consumer use. Because consumers are more used to wrist-worn devices, this article presents a comprehensive and systematic investigation of the feasibility of hand gesture recognition using EMG signals recorded at the wrist. A direct comparison of signal and information quality is conducted between concurrently recorded wrist and forearm signals. Both signals were collected simultaneously from 21 subjects while they performed a selection of 17 different single-finger gestures, multifinger gestures, and wrist gestures. Wrist EMG signals yielded consistently higher ( $p ) signal quality metrics than forearm signals for gestures that involved fine finger movements, while maintaining comparable quality for wrist gestures. Similarly, the performance of both individual state-of-the-art EMG features and a standard feature set was found to be significantly better when using wrist signals for single and multifinger gestures, and comparable for wrist gestures. Classifiers trained and tested using wrist EMG signals achieved average accuracy levels of 92.1% for single-finger gestures, 91.2% for multifinger gestures, and 94.7% for the conventional wrist gestures. In conclusion, this article clearly demonstrates the feasibility of using wrist EMG signals for hand gesture recognition. Results highlight not only the promise of this approach, but also the viability of incorporating prior knowledge from the prosthetics field in the design of wrist-based EMG pattern recognition systems.

Higher-order fuzzy decision-making methods have become powerful tools to support decision-makers in solving their problems effectively by reflecting uncertainty in calculations better than crisp sets in the last 3 decades. Fermatean fuzzy set proposed by Senapati and Yager, which can easily process uncertain information in decision making, pattern recognition, medical diagnosis et al., is extension of intuitionistic fuzzy set and Pythagorean fuzzy set by relaxing the restraint conditions of the support for degrees and support against degrees. In this paper, we focus on the similarity measures of Fermatean fuzzy sets. The definitions of the Fermatean fuzzy sets similarity measures and its weighted similarity measures on discrete and continuous universes are given in turn. Then, the basic properties of the presented similarity measures are discussed. Afterward, a decision-making process under the Fermatean fuzzy environment based on TOPSIS method is established, and a new method based on the proposed Fermatean fuzzy sets similarity measures is designed to solve the problems of medical diagnosis. Ultimately, an interpretative multi-criteria decision making example and two medical diagnosis examples are provided to demonstrate the viability and effectiveness of the proposed method. Through comparing the different methods in the multi-criteria decision making and the medical diagnosis application, it is found that the new method is as efficient as the other methods. These results illustrate that the proposed method is practical in dealing with the decision making problems and medical diagnosis problems.

In semiconductor foundries, wafer map defect analysis is crucial to prevent yield excursion. However, traditional manual inspection can hardly meet the high throughput demand. Deep learning based automatic defect detection shows promising efforts to achieve high accuracy and efficiency, yet the current approaches’ performance is limited by the imbalanced dataset and lack of interpretability. In this paper, we propose a Variational Autoencoder Enhanced Deep Learning Model (VAEDLM) for wafer defect imbalanced classification. It is light-weighted and effective in wafer defect pattern recognition on imbalanced dataset. It used variational autoencoder and decoder to generate similar wafer defect maps and a refined deep convolutional neural network for feature learning. We demonstrate the method using an authentic wafer map dataset, WM-811K. The performance is not only significantly improved after data augmentation, but it also beats the state-of-the art methods, reaching 99.19% accuracy, 99.10% recall, 99.23% precision, 99.96% AUC and 99.16% for F1-score. It clearly demonstrates the method’s efficacy to deal with the imbalanced defect pattern. Our study using saliency map and t-SNE further leads to enhanced interpretability.

This study aimed to clarify whether the pattern recognition involved in scoring proliferation fractions can be trained by abstract computerized images of virtual tissues. Twenty computer-generated images with randomly distributed blue or red dots were scored by 12 probands (all co-workers or collaborators of the Institute of Pathology, University of Bonn). Afterward, the probands underwent a training phase during which they received an immediate feedback on the actual rate of positivity after each image. Finally, the initial testing series was rescored. In a second round with 15 different probands, 20 Ki-67 immunohistochemistry images of tonsil tissue were scored, followed by the same training phase with computer-generated images, before the immunohistochemistry slides were scored again. Paired t-tests were used to compare the differences in mean rates pre- and post-training. Concerning computerized images, untrained probands scored the percentages of positive dots with a mean deviation from the true rates of 8.2%. Following training, the same testing series was scored significantly better with a mean deviation of 4.9% (mean improvement 3.3%, p

As a key component of a mechanical system, the extraction and accurate identification of rolling bearing fault feature information are of great importance to guarantee the normal operation of the mechanical system. Aiming at that the extraction of rolling bearing fault features and traditional support vector machine parameters affects the overall accuracy of pattern classification, the improved CEEMDAN (complete ensemble empirical mode decomposition with adaptive noise) time-domain energy entropy-based model for fault pattern recognition is proposed. The ICEEMDAN method is developed to decompose the signal to obtain the IMF component series. Then, the particular IMF components are selected according to the trend of correlation coefficient and variance contribution rate; meanwhile, the information entropy (power spectral entropy, singular spectral entropy, and time-domain energy entropy) of the screened IMF components is calculated to construct the feature vector sets, respectively. Finally, the feature vector sets are input into the PSO-SVM (particle swarm optimization-support vector machine) based model for training and pattern recognition. The effectiveness of the proposed method of the improved CEEMDAN time-domain energy entropy and PSO-SVM model is testified through numerical simulation and experiments on rolling bearing datasets. The comparison proceeded with the other mainstream intelligent recognition techniques indicates the superiority of the method with the diagnostic accuracy of 100% as for the final validation.

Purpose. In order to resolve the situation of high missed diagnosis rate and high misdiagnosis rate of the pathological analysis of the gastrointestinal endoscopic images by experts, we propose an automatic polyp detection algorithm based on Single Shot Multibox Detector (SSD). Method. In the paper, SSD is based on VGG-16, the fully connected layer is changed to a convolutional layer, and four convolutional layers with successively decreasing scales are added as a new network structure. In order to verify the practicability, it is not only compared with manual polyp detection but also with Mask R-CNN. Results. Multiple experimental results show that the mean Average Precision (mAP) of the SSD network is 95.74%, which is 12.4% higher than the manual detection and 5.7% higher than the Mask R-CNN. When detecting a single frame of image, the detection speed of SSD is 8.41 times that of manual detection. Conclusion. Based on the traditional pattern recognition algorithm and the target detection algorithm using deep learning, we select a variety of algorithms to identify and classify polyps to achieve efficient detection results. Our research demonstrates that deep learning has a lot of room for development in the field of gastrointestinal image recognition.

This work proposes a hybrid 3D Convolutional Neural Network and Restricted Boltzmann Machine (Hybrid 3DCNN-RBM) architecture tailored for gas concentration estimation. The immense success of deep learning in computer vision and natural language processing inspired us to design a deep-learning-based gas concentration estimation network. The proposed network is a joint 3DCNN and RBM network. It uses raw time-series gas sensor array data and provides the concentration of each gas in a mixture of gases. This raw time series data is considered similar to video signal comparing sensor array response as an image, which varies with time. We first time utilized 3DCNN for sensor array data processing and developed a joint network with RBM to design an end-to-end gas concentration estimation model. Although several pattern recognition-based methods have been applied to estimate the gas concentration of the gas mixtures, the effectiveness of these techniques enormously depends on the hand-created feature engineering. Nevertheless, the experiments show that the proposed method is an efficient method for estimating gas concentration in the gas mixture for sensor array data.

To explore the adoption effect of improved neural network blade pattern in corona virus disease (COVID)-19, comparative analysis is implemented. First, the following hypotheses are proposed. I: in addition to the confirmed cases and deaths, people suspected of being infected are also involved in the spread of the epidemic. II: patients who have been cured may also develop secondary infections, so it is considered that there is still a link between cured cases and the spread of the epidemic. III: only the relevant data of the previous day is used to predict the epidemic prevention and control of the next day. Then, the epidemic data from February 1st to February 15th in X province were selected as the control. The combined neural network model is used for prevention and control prediction, and the prediction results of the traditional neural network model are compared. The results show that the predictions of the daily new cases by the five neural network models have little difference with the actual value, and the trend is basically consistent. However, there are still differences in some time nodes. The errors of neural network 1 on the 6th and network 3 on the 13th are large. The accuracy of the combined neural network prediction model is high, and there is little difference between the result and the actual value at each time node. The prediction of the cumulative number of diagnoses per day of the five neural network models is also analyzed, and the results are relatively ideal. In addition, the accuracy of the combined neural network prediction model is high, and the difference between the result and the actual value at each time node is relatively small. It is found that the standard deviations of neural networks 2 and 3 are relatively high through the comparison of the deviations. The deviation means of the five models were all relatively low, and the mean deviation and standard deviation of the combined neural network model are the lowest. It is found that the accuracy of prediction on the epidemic spread in this province is good by comparing the performance of each neural network model. Regarding various indicators, the prediction accuracy of the combined neural network model is higher than that of the other four models, and its performance is also the best. Finally, the MSE of the improved neural network model is lower compared with the traditional neural network model. Moreover, with the change of learning times, the change trend of MSE is constant (P

With the continuous scaling down of process nodes, standard cells become much smaller and cell counts are dramatically increased. Pin accessibility becomes one of the major issues causing design rule violations (DRVs). To tackle this problem, many recent works apply machine-learning-based techniques to predict whether a local region has DRV or not by regarding global routing (GR) congestion and local pin density as the main features during the training process. Empirically, however, DRV occurrence is not necessary to be strongly correlated with the two features in advanced nodes. In this article, we propose the first work of deep-learning-based DRV prediction using pin pattern as our major feature to directly identify whether a DRV will exist or not due to bad pin accessibility of the given pin pattern. Unlike most of the existing models that can only be used for DRV prediction, the proposed models can be applied to guide detailed placement for pin accessibility optimization during physical design. Experimental results show that the proposed models are greatly superior than those of previous studies in terms of all quantitative metrics. Additionally, the numbers of DRVs can be dramatically reduced by applying the proposed model-guided detailed placement flow.

Three-dimensional Morphable Models (3DMMs) are powerful statistical tools for representing the 3D shapes and textures of an object class. Here we present the most complete 3DMM of the human head to date that includes face, cranium, ears, eyes, teeth and tongue. To achieve this, we propose two methods for combining existing 3DMMs of different overlapping head parts: i. use a regressor to complete missing parts of one model using the other, ii. use the Gaussian Process framework to blend covariance matrices from multiple models. Thus we build a new combined face-and-head shape model that blends the variability and facial detail of an existing face model (the LSFM) with the full head modelling capability of an existing head model (the LYHM). Then we construct and fuse a highly-detailed ear model to extend the variation of the ear shape. Eye and eye region models are incorporated into the head model, along with basic models of the teeth, tongue and inner mouth cavity. The new model achieves state-of-the-art performance. We use our model to reconstruct full head representations from single, unconstrained images allowing us to parameterize craniofacial shape and texture, along with the ear shape, eye gaze and eye color., 18 pages, 18 figures, submitted to Transactions on Pattern Analysis and Machine Intelligence (TPAMI) on the 9th of October as an extension paper of the original oral CVPR paper : arXiv:1903.03785

Spatio-temporal information processing is fundamental in both brain functions and AI applications. Current strategies for spatio-temporal pattern recognition usually involve explicit feature extraction followed by feature aggregation, which requires a large amount of labeled data. In the present study, motivated by the subcortical visual pathway and early stages of the auditory pathway for motion and sound processing, we propose a novel brain-inspired computational model for generic spatio-temporal pattern recognition. The model consists of two modules, a reservoir module and a decision-making module. The former projects complex spatio-temporal patterns into spatially separated neural representations via its recurrent dynamics, the latter reads out neural representations via integrating information over time, and the two modules are linked together using known examples. Using synthetic data, we demonstrate that the model can extract the frequency and order information of temporal inputs. We apply the model to reproduce the looming pattern discrimination behavior as observed in experiments successfully. Furthermore, we apply the model to the gait recognition task, and demonstrate that our model accomplishes the recognition in an event-based manner and outperforms deep learning counterparts when training data is limited.

Conventional surveillance systems for preventing accidents and incidents do not identify 95% thereof after 22 min when one person monitors a plurality of closed circuit televisions (CCTV). To address this issue, while computer-based intelligent video surveillance systems have been studied to notify users of abnormal situations when they happen, it is not commonly used in real environment because of weakness of personal information leaks and high power consumption. To address this issue, intelligent video surveillance systems based on small devices have been studied. This paper suggests implement an intelligent video surveillance system based on embedded modules for intruder detection based on information learning, fire detection based on color and motion information, and loitering and fall detection based on human body motion. Moreover, an algorithm and an embedded module optimization method are applied for real-time processing. The implemented algorithm showed performance of 88.51% for intruder detection, 92.63% for fire detection, 80% for loitering detection and 93.54% for fall detection. The result of comparison before and after optimization about the algorithm processing time showed 50.53% of decrease, implying potential real-time driving of the intelligent image monitoring system based on embedded modules.

An elastic neural network is developed and evolved towards effectively re-configurable hardware in fully parallel on chip. The original prototype of DiaNet is organized as a symmetrical bisection neural network, which is feasible to be partitioned into arbitrary pieces of neural networks (NNs) without redundancy. To prevent the depth explosion in implementing complex tasks (complicated pattern recognition for instance), the evolution of DiaNets is investigated in this work. By using the I/O layer integration technology which enables all neurons in the hidden layer of DiaNet to receive inputs, the number of layers is reduced to 8.8 % of DiaNet prototype. In this manner, the DiaNet topology is feasible to implement complex NNs without the risk of depth explosion. Moreover, the skip connection technology is proposed to avoid the gradient vanishing due to deep learning, which is significant to DiaNets especially. Compared with the LeNet5 model as state-of-the-art, the evolved DiaNet topology achieves the parameter reduction of 90.86 % for MNIST recognition with the negligible loss of accuracy. To reduce hardware utilization, the sensitivity to the decline of computational precision and bit-width is investigated to suggest the guideline for efficient hardware implementations. Finally, the effectiveness of DiaNet is verified by the proposed re-configurable architecture on FPGA with the power reduction of 10.8 % compared to state-of-the-art implementations.

Image binarization of uneven lighted images, using thresholding techniques, is still a challenging task. Adaptive thresholding methods are the widely adopted approaches for binarization of uneven lighting images. However, the efficacy of these adaptive thresholding methods is highly sensitive to the criteria function used for measuring the bimodal property of the gray level distribution of a local region. In this paper, we propose Gaussian Mixture Model (GMM) which is based on adaptive thresholding for binarizing uneven lighting images. The proposed GMM based criteria function efficiently partitioning the uneven light images into bimodal and unimodal subimages with low uneven light effect. At first, the bimodal subimages are binarized using Otsu’s thresholding approach, followed by unimodal subimages being thresholded using the bilinear interpolation of neighbouring thresholds of bimodal subimages. Next a fast Expectation Maximization(EM) algorithm is developed to reduce the computational complexity of the GMM. Experimental results on different uneven light images demonstrate that the proposed adaptive thresholding outperforms the other considered methods with an avg. misclassification error of 1.68 % and an average computation time of 1.50 seconds. The computational time can be further reduced by a specially purposed hardware and parallel processing of each subimages for real time applications.

Perceptual video hashing represents video perceptual content by compact hash. The binary hash is sensitive to content distortion manipulations, but robust to perceptual content preserving operations. Currently, boundary between sensitivity and robustness is often ambiguous and it is decided by an empirically defined threshold. This may result in large false positive rates when received video is to be judged similar or dissimilar in some circumstances, e.g., video content authentication. In this paper, we propose a novel perceptual hashing method for video content authentication based on maximized robustness. The developed idea of maximized robustness means that robustness is maximized on condition that security requirement of hash is first met. We formulate the video hashing as a constrained optimization problem, in which coefficients of features offset and robustness are to be learned. Then we adopt a stochastic optimization method to solve the optimization. Experimental results show that the proposed hashing is quite suitable for video content authentication in terms of security and robustness.

The problem of developing universal classifiers of biomedical data, in particular those that characterize the presence of a large number of parameters, inaccuracies and uncertainty, is urgent. Many studies are aimed at developing methods for analyzing these data, among them there are methods based on a neural network (NN) in the form of a multilayer perceptron (MP) using GA. The question of the application of evolutionary algorithms (EA) for setting up and learning the neural network is considered. Theories of neural networks, genetic algorithms (GA) and decision trees intersect and penetrate each other, new developed neural networks and their applications constantly appear. An example of a problem that is solved using EA algorithms is considered. Its goal is to develop and research a classifier for the diagnosis of breast cancer, obtained by combining the capabilities of the multilayer perceptron using the genetic algorithm (GA) and the CART decision tree. The possibility of improving the classifiers of biomedical data in the form of NN based on GA by applying the process of appropriate preparation of biomedical data using the CART decision tree has been established. The obtained results of the study indicate that these classifiers show the highest efficiency on the set of learning and with the minimum reduction of Decision Trees; increasing the number of contractions usually degrades the simulation result. On two datasets on the test set, the simulation accuracy was »83–87%. The experiments carried out have confirmed the effectiveness of the proposed method for the synthesis of neural networks and make it possible to recommend it for practical use in processing data sets for further diagnostics, prediction, or pattern recognition

Semantic information in the human brain is organized into multiple networks, but the fine-grain relationships between them are poorly understood. In this study, we compared semantic maps obtained from two functional magnetic resonance imaging experiments in the same participants: one that used silent movies as stimuli and another that used narrative stories. Movies evoked activity from a network of modality-specific, semantically selective areas in visual cortex. Stories evoked activity from another network of semantically selective areas immediately anterior to visual cortex. Remarkably, the pattern of semantic selectivity in these two distinct networks corresponded along the boundary of visual cortex: for visual categories represented posterior to the boundary, the same categories were represented linguistically on the anterior side. These results suggest that these two networks are smoothly joined to form one contiguous map.

This paper presents a Function-as-a-Service (FaaS) approach for deploying managed cloud functions onto heterogeneous cloud infrastructures. Current FaaS systems, such as AWS Lambda, allow domain-specific functionality, such as AI, HPC and image processing, to be deployed in the cloud while abstracting users from infrastructure and platform concerns. Existing approaches, however, use a single type of resource configuration to execute all function requests. In this paper, we present a novel FaaS approach that allows cloud functions to be effectively executed across heterogeneous compute resources, including hardware accelerators such as GPUs and FPGAs. We implement heterogeneous scheduling to tailor resource selection to each request, taking into account performance and cost concerns. In this way, our approach makes use of different processor types and quantities (e.g. 2 CPU cores), uniquely suited to handle different types of workload, potentially providing improved performance at a reduced cost. We validate our approach in three application domains: machine learning, bio-informatics, and physics, and target a hardware platform with a combined computational capacity of 24 FPGAs and 12 CPU cores. Compared to traditional FaaS, our approach achieves a cost improvement for non-uniform traffic of up to 8.9 times, while maintaining performance objectives.

Quantification and recognition of geochemical patterns are extremely important for geochemical prospecting and can facilitate a better understanding of regional metallogenesis. Recognition of such patterns with deep learning (DL) algorithms has attracted considerable attention, as these algorithms can generally extract high-level geochemical features and thus create models in which geochemical patterns are fully exploited. These DL algorithms are usually constructed to be compatible and consistent with the underlying data, but their interpretability and pertinent physical constraints (such as granitic intrusions related to magmatic-hydrothermal mineralization) are generally overlooked. This paper introduces a physically constrained variational autoencoder (VAE) architecture to identify geochemical patterns associated with tungsten polymetallic mineralization. We first identify physical constraints from geological characteristics and metallogenic regulation via the methods of fry analysis, standard deviation ellipses, and fractal analysis to reveal the controlling function of granitic intrusions on mineralization. Subsequently, we construct the physical constraints based on the nonlinear controlling function and add the geological constraints into the VAE loss function as a penalty term. After optimization of the network parameters, the well-trained physically constrained VAE architecture can recognize the geochemical anomaly patterns that the conventional VAE cannot. These extracted geochemical anomaly patterns generally show a strong spatial relationship with the granitic intrusions. In addition, the performance measures involving the receiver operating characteristic curve and success-rate further indicate that the generalization accuracy of the conventional VAE can be enhanced via physics-based regularization. These results suggest that the proposed physically constrained VAE, which integrates physical knowledge into the VAE loss function, not only improves the geochemical pattern recognition performance but also demonstrates consistency with geological and physical domain knowledge.

The 3D CAD shapes in current 3D benchmarks are mostly collected from online model repositories. Thus, they typically have insufficient geometric details and less informative textures, making them less attractive for comprehensive and subtle research in areas such as high-quality 3D mesh and texture recovery. This paper presents 3D Furniture shape with TextURE (3D-FUTURE): a richly-annotated and large-scale repository of 3D furniture shapes in the household scenario. At the time of this technical report, 3D-FUTURE contains 20,240 clean and realistic synthetic images of 5,000 different rooms. There are 9,992 unique detailed 3D instances of furniture with high-resolution textures. Experienced designers developed the room scenes, and the 3D CAD shapes in the scene are used for industrial production. Given the well-organized 3D-FUTURE, we provide baseline experiments on several widely studied tasks, such as joint 2D instance segmentation and 3D object pose estimation, image-based 3D shape retrieval, 3D object reconstruction from a single image, and texture recovery for 3D shapes, to facilitate related future researches on our database., Project Page: https://tianchi.aliyun.com/specials/promotion/alibaba-3d-future

Optical marker-based motion capture (MoCap) remains the predominant way to acquire high-fidelity articulated body motions. We introduce DeMoCap, the first data-driven approach for end-to-end marker-based MoCap, using only a sparse setup of spatio-temporally aligned, consumer-grade infrared-depth cameras. Trading off some of their typical features, our approach is the sole robust option for far lower-cost marker-based MoCap than high-end solutions. We introduce an end-to-end differentiable markers-to-pose model to solve a set of challenges such as under-constrained position estimates, noisy input data and spatial configuration invariance. We simultaneously handle depth and marker detection noise, label and localize the markers, and estimate the 3D pose by introducing a novel spatial 3D coordinate regression technique under a multi-view rendering and supervision concept. DeMoCap is driven by a special dataset captured with 4 spatio-temporally aligned low-cost Intel RealSense D415 sensors and a 24 MXT40S camera professional MoCap system, used as input and ground truth, respectively.

Multiple Object Tracking (MOT) has attracted increasing interests in recent years, which plays a significant role in video analysis. MOT aims to track the specific targets as whole trajectories and locate the positions of the trajectory at different times. These trajectories are usually applied in Action Recognition, Anomaly Detection, Crowd Analysis and Multiple-Camera Tracking, etc. However, existing methods are still a challenge in complex scene. Generating false (impure, incomplete) tracklets directly affects the performance of subsequent tasks. Therefore, we propose a novel architecture, Siamese Bi-directional GRU, to construct Cleaving Network and Re-connection Network as trajectory post-processing. Cleaving Network is able to split the impure tracklets as several pure sub-tracklets, and Re-connection Network aims to re-connect the tracklets which belong to same person as whole trajectory. In addition, our methods are extended to Multiple-Camera Tracking, however, current methods rarely consider the spatial-temporal constraint, which increases redundant trajectory matching. Therefore, we present Position Projection Network (PPN) to convert trajectory position from local camera-coordinate to global world-coodrinate, which provides adequate and accurate temporal-spatial information for trajectory association. The proposed technique is evaluated over two widely used datasets MOT16 and Duke-MTMCT, and experiments demonstrate its superior effectiveness as compared with the state-of-the-arts.