Search

In order to improve the efficiency of antenna optimization design, a surrogate model is often used to replace the full-wave electromagnetic simulation software. Broad learning system (BLS) provides an alternative method for deep structure, aiming to overcome the drawback of excessive time-consuming training process, however, usually not with satisfactory accuracy. In order to further improve the performance of the model, master-apprentice (MA) behavior is proposed in this paper, using the current BLS training results as the priori knowledge, which are taken as fixed features to the next BLS hidden layer for further training. Each MA behavior forms a double BLS structure, which is composed of two parts, the models trained before and after are called master BLS (MBLS) and apprentice BLS (ABLS) respectively. These two subsystems together constitute a master-apprentice BLS (MABLS). Two antenna examples, rectangular microstrip antenna (RMSA) and WLAN dual-band monopole antenna (DBMA), and 10 UCI regression datasets are employed to demonstrate the effectiveness of the proposed model.

Sign Language is the linguistic system adopted by the Deaf to communicate. The lack of fully-fledged Automatic Sign Language (ASLR) technologies contributes to the numerous difficulties that deaf individuals face in the absence of an interpreter, such as in private health appointments or in emergency situations. A challenging problem in the development of reliable ASLR systems is that sign languages do not rely only on manual gestures but also on facial expressions and other non-manual markers. This paper proposes to adopt Facial Action Coding System to encode sign language facial expressions. However, the state-of-the-art of Action Unit (AU) recognition models is mostly targeted to classify two dozen of AUs, typically related to the expression of emotions. We adopted Brazilian Sign Language (Libras) as our case study and we identified more than one hundred of AUs (with a great intersection with other sign languages). We then implemented and evaluated a novel AU recognition model architecture that combines SqueezeNet and geometric-based features. Our model obtained 88% of accuracy for 119 classes. Combined with the state-of-the-art of gesture recognition, our model is ready to improve sign disambiguation and to advance ASLR.

Batik is a hereditary cultural heritage that has high aesthetic value and deep philosophy. Currently, Indonesian batik has various types of different motifs and patterns, which are spread in Indonesia with their names and meanings. Batik classification uses Convolutional Neural Network as a pattern recognition method, especially batik image classification. The method used is a literature study, looking at studies from several journals regarding the Convolutional Neural Network Algorithm in Classification and providing conclusions about the usefulness of the algorithm. Analysis This literature study analyzes each journal from previous research related to the Convolutional Neural Network Algorithm in classifying Batik. The results of the analysis, conducted a discussion to better know the characteristics and application of Convolutional Neural Network in the classification of Batik. After discussing, this analysis ends with conclusions about the Convolutional Neural Network algorithm in classifying Batik. Based on previous studies, it can be seen that the convolution neural network can work well for image classification with large datasets. By evaluating the method that has been described by considering the architecture and the level of accuracy, namely getting an accuracy level of 100% with an image size of 128 x 128 and regarding the classification of batik, it shows that image size, image quality, image patterns affect the batik classification process.

Low-level details and high-level semantics are both essential to the semantic segmentation task. However, to speed up the model inference, current approaches almost always sacrifice the low-level details, leading to a considerable decrease in accuracy. We propose to treat these spatial details and categorical semantics separately to achieve high accuracy and high efficiency for real-time semantic segmentation. For this purpose, we propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral Segmentation Network (BiSeNet V2). This architecture involves the following: (i) A detail branch, with wide channels and shallow layers to capture low-level details and generate high-resolution feature representation; (ii) A semantics branch, with narrow channels and deep layers to obtain high-level semantic context. The detail branch has wide channel dimensions and shallow layers, while the semantics branch has narrow channel dimensions and deep layers. Due to the reduction in the channel capacity and the use of a fast-downsampling strategy, the semantics branch is lightweight and can be implemented by any efficient model. We design a guided aggregation layer to enhance mutual connections and fuse both types of feature representation. Moreover, a booster training strategy is designed to improve the segmentation performance without any extra inference cost. Extensive quantitative and qualitative evaluations demonstrate that the proposed architecture shows favorable performance compared to several state-of-the-art real-time semantic segmentation approaches. Specifically, for a $$2048\times 1024$$ input, we achieve 72.6% Mean IoU on the Cityscapes test set with a speed of 156 FPS on one NVIDIA GeForce GTX 1080 Ti card, which is significantly faster than existing methods, yet we achieve better segmentation accuracy. The code and trained models are available online at https://git.io/BiSeNet .

Automated text detection and analysis holds incredible potential for research in higher education. It is challenging because higher education institutes produce an enormous amount and variety of texts, letters, articles, books, reports etc. Futuristic E-learning based education replaces the difficulty of understanding the semantic meaning of the learning content from videos which is most prominent source used by the leaners to acquire knowledge. Therefore, Content Based Video retrieval has become the challenging research area under pattern recognition and computer vision in higher education through Massive Open Online Courses (MOOCs). Text plays a dynamic role in understanding the true meaning of behavior of the video. Hence, it is challenging to detect and identify the text in video due to variable complex background, low contrast, blur, poor illumination, font size, font-style, occlusions. The traditional approach of end-to-end convolution neural network (CNN) performs satisfactory in detecting video text. However, it is also important to deal with the video size, therefore, we have adopted Map Reduce technique to store the video content and utilize it efficiently by parallel computing. Followed by this, we employed novel approach to clean up the video frames to feed to neural network model based on region proposal network (RPN) with CNN by finding appropriate anchor ratios to extract the text candidates. Finally, we train our model with extracted frames to predict for the test videos. The proposed method is evaluated on ICDAR Video text benchmark datasets and few publicly available test datasets to achieve high recall.

Due to the low computational complexity, the two-phase test sample representation method shows outstanding advantages in the field of real-time face recognition. However, its first phase does not fully consider the imbalance of the determined K-nearest training samples. This defect is not well reflected and will seriously affect the recognition accuracy of the second phase. In this paper, we explore the above issues and propose to incorporate the unselected training samples into the modeling process. The proposed method not only allows the unselected samples to perform the final representation but also makes the selected nearest training samples play a more significant role than the unselected ones in classification. It is efficiently solved with an analytical solution. The rationales and probability interpretation of the proposed method are presented to further guarantee its rationality and effectiveness. Extensive experiments on diverse face databases are conducted to demonstrate the superior recognition performance in comparison with other competing classifiers.

Current available supervised classifiers cannot generalize across various domains due to distribution mismatch among them. Domain adaptation and transfer learning algorithms are proposed to tackle domain shift problem that originates from different data collection conditions. In this paper, we propose a transfer learning framework called iterative joint classifier and domain adaptation for visual transfer learning (ICDAV), which utilizes the balanced maximum mean discrepancy to better transfer knowledge across domains. Also, for learning a robust classifier against domain shift, a set of graph manifold regularizer and modified joint probability maximum mean discrepancy are simultaneously exploited to capture the domain structures and adapt the distribution of projected samples during the model learning process. Variety of experiments on several public datasets indicates that our approach achieves remarkable performance on visual domain adaptation and transfer learning tasks.

A novel photonics-based multivariate pattern recognition technique is presented to segregate bison meat samples based on muscle type, ageing, and retail display period. The technique uses color attributes obtained from visible to near-infrared hyperspectral images (400–1000 nm) to predict the stability of bison muscle samples. Unsupervised and supervised classification methods were implemented with an aim to discriminate muscle samples based on muscle type, ageing period, and retail display period. The wavelength region from 500 to 690 nm which is associated with the a∗ value in the CIE Lab color space was found to be significantly important for the classification of muscle samples over the storage period. Partial least squares discriminant analysis (PLS-DA) demonstrated classification accuracies from 0.88 to 0.94 for the classification of muscle type, ageing period and retail display followed by development of classification maps. For the estimation of color changes in the muscle samples over the storage and retail display period, a∗ value was predicted with an R2 of calibration of 0.89, and R2 of cross-validation of 0.88. Conclusively, the wavelength range from 550 to 690 nm can significantly contribute to sorting and predicting freshness of bison muscle samples based on muscle type, color stability and storage period.

In this article, a fault diagnosis scheme based on the data-driven approach of hypersonic vehicles (HVs) is studied. First, the fault features are obtained by wavelet packet translation (WPT) processing. Second, an improved distance evaluation technique (DET) based on Spearman correlation analysis is used to select features and reduce dimensions. The designed enhanced WPT-DET (EWPT-DET) method can extract sensitive features based on self-defined attention coefficients. Third, the fault pattern recognition process is achieved by support vector regression (SVR) with genetic algorithm optimization. The SVR classifier with high dimensional linear fitting ability is very suitable for HVs’ reaction control system with sensor faults. The method is further used in locating and diagnosing multisensor fusion faults. Moreover, the fault occurrence time of single sensor timing faults is judged. Finally, simulation studies are provided to illustrate the enhanced performance of the proposed approach.

The u-serrated immunodeposition pattern in direct immunofluorescence (DIF) microscopy is a recognizable feature and confirmative for the diagnosis of epidermolysis bullosa acquisita (EBA). Due to unfamiliarity with serrated patterns, serration pattern recognition is still of limited use in routine DIF microscopy. The objective of this study was to investigate the feasibility of using convolutional neural networks (CNNs) for the recognition of u-serrated patterns that can assist in the diagnosis of EBA. The nine most commonly used CNNs were trained and validated by using 220,800 manually delineated DIF image patches from 106 images of 46 different patients. The data set was split into 10 subsets: nine training subsets from 42 patients to train CNNs and the last subset from the remaining four patients for a validation data set of diagnostic accuracy. This process was repeated 10 times with a different subset used for validation. The best-performing CNN achieved a specificity of 89.3% and a corresponding sensitivity of 89.3% in the classification of u-serrated DIF image patches, an expert level of diagnostic accuracy. Experiments and results show the effectiveness of CNN approaches for u-serrated pattern recognition with a high accuracy. The proposed approach can assist clinicians and pathologists in recognition of u-serrated patterns in DIF images and facilitate the diagnosis of EBA.

Due to the high complexity and dynamics of the semiconductor manufacturing process, various process abnormality could result in wafer map defects in many work stations. Thus, wafer map pattern recognition (WMPR) in the semiconductor manufacturing process can help operators to troubleshoot root causes of the out-of-control process and then accelerate the process adjustment. This article proposes a novel deep neural network (DNN), two-dimensional principal component analysis-based convolutional autoencoder (PCACAE) for wafer map defect recognition. First, a new convolution kernel based on conditional two-dimensional principal component analysis is developed to construct the first convolutional block of PCACAE. Second, a convolutional autoencoder is cascaded by considering the nonlinearity of data representation. The second convolutional block of PCACAE is constructed based on the encoding part. Finally, the pretrained PCACAE is fine-tuned to obtain the final classifier. PCACAE is successfully applied for feature learning and recognition of wafer map defects. The experimental results on a real-world case demonstrate that PCACAE is superior to other well-known convolutional neural networks (e.g., GoogLeNet, PCANet) on WMPR.

Weeding is beneficial to the growth of the crops in field. At present, weeding in China mainly relies on chemical herbicide spraying on a large area, which leads to environmental pollution. Combined with digital image processing and pattern recognition technology, weed species identification in wheat seedling stage in field is of great significance to realize the variable spraying of herbicide, reduce the cost and protect the ecological environment. Weed species identification in field by machine vision is one of the challenging and hard topics because of the diversity and changeability of the weed in field. A weed species recognition approach is proposed combining modified Grabcut, adaptive fuzzy dynamic K-means algorithms and sparse representation classification (SRC). First, the original weed images are enhanced and noise is suppressed using filtering technique, and in the segmentation phase, each weed image is coarsely segmented by the modified GrabCut algorithm to remove most of background of the original image captured in the field, which can reduce the computing cost and recognition time. The original weed image is segmented by adaptive fuzzy dynamic K-means. Finally the weed species is recognized by SRC. Compared with the other weed recognition methods, the proposed method integrated the advantages of three approaches, (1) the improved Grabcut method does not require human interaction and can automatically segment the background, (2) the dynamic K-means algorithm introduces fitness function to evaluate clustering, which reduces the dependence of traditional K-means clustering algorithm on the initial value of clustering center to a certain extent, and avoids the problems such as dead zone center and center redundancy caused by local extremum, (3) SRC is utilized to classify the weed species. To test the proposed method, a lot of experiments are carried on the wheat weed image dataset. The results validate that the proposed method is effective for the weed species recognition, which can be used as a preliminary step for precision applying pesticide.

Outlier or anomaly detection is an important branch of data analysis that becomes a crucial task in many application domains. Data objects which significantly dissimilar and inconsistent from the rest of the data objects are referred to as an outlier. In this paper, a new approach, called LDBAD (Local Density-Based Abnormal Detector), is proposed to discover useful irregular patterns hidden in the collected data sets. This method aims to find local abnormal data objects, which are characterized through three proposed measurements: local distance, local density, and Influenced outlierness degree. The performance of the proposed approach is evaluated on flow pattern experiments along a 180 degrees sharp bend channel with and without a T-shaped spur dike. Flow velocity components are collected using 3D velocimeter Vectrino. The analysis shows that the novel outlier detection method is effective and applicable to find outlier objects. Moreover, some feed-forward neural network velocity prediction models are created to demonstrate the necessity and advantages of outlier detection in flow pattern experiments. The results show that the accuracy of created models has been increased by removing outliers from the measurements.

Our research is devoted to answering whether randomisation-based learning can be fully competitive with the classical feedforward neural networks trained using backpropagation algorithm for classification and regression tasks. We chose extreme learning as an example of randomisation-based networks. The models were evaluated in reference to training time and achieved efficiency. We conducted an extensive comparison of these two methods for various tasks in two scenarios: $$\bullet$$ ∙ using comparable network capacity and $$\bullet$$ ∙ using network architectures tuned for each model. The comparison was conducted on multiple datasets from public repositories and some artificial datasets created for this research. Overall, the experiments covered more than 50 datasets. Suitable statistical tests supported the results. They confirm that for relatively small datasets, extreme learning machines (ELM) are better than networks trained by the backpropagation algorithm. But for demanding image datasets, like ImageNet, ELM is not competitive to modern networks trained by backpropagation; therefore, in order to properly address current practical needs in pattern recognition entirely, ELM needs further development. Based on our experience, we postulate to develop smart algorithms for the inverse matrix calculation, so that determining weights for challenging datasets becomes feasible and memory efficient. There is a need to create specific mechanisms to avoid keeping the whole dataset in memory to compute weights. These are the most problematic elements in ELM processing, establishing the main obstacle in the widespread ELM application.

Hierarchical reinforcement learning (HRL) is a promising method, which decomposes complex tasks into a series of sub-tasks. However, at present, most HRL methods have slow convergence speed and are difficult to be widely applied to such scenarios in real life. In this paper, we propose an efficient hierarchical reinforcement learning algorithm with fuzzy rules (HFR), a novel framework for integrating human prior knowledge with hierarchical policy network, which can effectively accelerate the optimization of policy. The model presented in this paper uses fuzzy rules to represent the human prior knowledge, making the rules trainable because of the derivability of the fuzzy rules. In addition, a switch module that adaptively adjusts the decision-making frequency of the upper-level policy is proposed to solve the limitation of manual tuning. Experiment results demonstrate that HFR has a faster convergence rate than the current state-of-the-art HRL algorithms, especially in complex scenarios, such as robot control tasks.

Dimensionality reduction is vital in many fields, such as computer vision and pattern recognition. This paper proposes an unsupervised dimensionality reduction algorithm based on multi-local linear regression. The algorithm first divides the high-dimensional data into many localities. Under the criterion of local homeomorphism, the continuous dependency relationship of the high-dimensional data is maintained in each locality in the low-dimensional space. At the same time, due to the overlap of locality divisions, that is, each data may belong to multiple localities. Therefore, the algorithm performs a multi-local linear prediction on each target data point, to better capture the internal geometric structure of the data. Finally, to coordinate the predictions of the target data points by each locality, we require that the variance between the predictions of each locality to the same target point should be as small as possible. We perform experiments on synthetic and real datasets. Compared with the existing advanced algorithms, the experimental results show that the proposed algorithm has good feasibility.

In the field of pattern recognition, the problem of offline character recognition has long been a fascinating challenge. Several studies demonstrate that the process of image recognition is a little computationally expensive. The objective of this study includes an effective and reliable technique for the offline identification of Latin font characters produced by computers with a less number feature dimension. For which an effective representation technique for the character images is needed. The framework uses some machine learning techniques, SVM, random forest, RNN, and CNN, such as a standard recognition classifier. This study uses four feature extraction technologies to assess the efficiency of the proposed framework on a particular Chars74kFnt dataset. The extracted feature set originally introduced here is of 528 dimensions. Results show that CNNs deep neural network classifier outperforms the others without compromising efficiency. Later on, this dimension is reduced from 528 to just 220 by applying the Maximum Relevance Minimum Redundancy (mRMR) feature selection process. The results of these tests are shown in separate tables and also analyzed.

Both regression modeling and clustering methodologies have been extensively studied as separate techniques. There has been some activity in using regression-based algorithms to partition a data set into clusters for classical data; we propose one such algorithm to cluster interval-valued data. The new algorithm is based on the k-means algorithm of MacQueen (1967) and the dynamical partitioning method of Diday and Simon (1976), with the partitioning criteria being based on establishing regression models for each sub-cluster. This also depends on distance measures between the underlying regression models for each sub-cluster. Several types of simulated data sets are generated for several different data structures. The proposed k-regressions algorithm consistently out-performs the k-means algorithm. Elbow plots are used to identify the total number of clusters K in the partition. The new method is also applied to real data.

In many machine learning applications and algorithms, the algorithm performance and accuracy are highly dependent on the metric used to measure the distance between different samples. Therefore, learning a distance metric specific to the data can improve these algorithms’ performance. This paper proposes an unrestricted deep metric learning framework based on neural networks’ interaction for learning metrics in latent space. The proposed method is inspired by generative neural nets (GANs), in which two neural nets are working together to learn true data distribution. In our method, one network plays the role of a supervisor for another network, a feature learning auto-encoder. Its task is to learn transformation to latent space in which data have more meaningful distance and separability. i.e., the supervisor gets the output of the auto-encoder and sends feedback to modify its weights. They interact with each other interleavingly. Several experiments were conducted on four datasets, such as MNIST, GISETTE, Winnipeg Cropland Classification (WCC), and swarm behavior, from different application domains, to evaluate the proposed method’s performance. The results show that we can force auto-encoder to learn label information to project data into a latent space with better separability by using our approach. In addition to better class discrimination, the proposed method is far faster than normal auto-encoders during feature learning and has much less training time in the classification phase.

Representation learning has been widely applied in real-world recommendation systems to capture the features of both users and items. Existing grocery recommendation methods only represent each user and item by single deterministic points in a low-dimensional continuous space, which limit the expressive ability of their embeddings, resulting in recommendation performance bottlenecks. In addition, existing representation learning methods for grocery recommendation only consider the items (products) as independent entities, neglecting their other valuable side information, such as the textual descriptions and the categorical data of items. In this paper, we propose the Variational Bayesian Context-Aware Representation (VBCAR) model for grocery recommendation. VBCAR is a novel variational Bayesian model that learns distributional representations of users and items by leveraging basket context information from historical interactions. Our VBCAR model is also extendable to leverage side information by encoding contextual features into representations based on the inference encoder. We conduct extensive experiments on three real-world grocery datasets to assess the effectiveness of our model as well as the impact of different construction strategies for item side information. Our results show that our VBCAR model outperforms the current state-of-the-art grocery recommendation models while integrating item side information (especially the categorical features with the textual information of items) results in further significant performance gains. Furthermore, we demonstrate through analysis that our model is able to effectively encode similarities between product types, which we argue is the primary reason for the observed effectiveness gains.

Augmented Reality (AR) is more than an added value for Cultural Heritage (CH); it is vital for its sustainability, promotion and dissemination, increasing accessibility in CH even during difficult periods of time, like the Covid-19 pandemic. In order to be meaningful and engaging, an AR application should have the following characteristics: easiness of use, high-quality representations and compatibility. This paper presents a marker-less mobile AR application for the display and inspection of high-resolution 3D cultural assets, overlayed on a particular location in the real-world scene. Instead of predefined markers, an image captured by the user is exploited as a pattern for real-time feature matching, pose estimation and scene augmentation. Our approach is based on pure computer vision and photogrammetric techniques, implemented using native C++ and Java code for Android mobile platforms. It is built with the use of the OpenCV library and the OpenGL ES graphics API without any dependencies of AR Software Development Kits (SDKs). Therefore, it supports cross-vendor portability regarding mobile model devices and hardware specifications. The evaluation of the developed application examines the performance of various matching techniques and the overall responsiveness of processing and 3D rendering on mid-range and low-end smartphones. The results showcase the reliability and responsiveness of the pattern recognition as well as the potential of the 3D graphics engine to render and overlay complex 3D models balancing between visual quality and time. The proposed methodology is applied to the Ciborium of the church of St. Charalabos, located at St. Stephen’s Monastery in Meteora, Greece.

The news media shape public opinion, and often, the visual bias they contain is evident for careful human observers. This bias can be inferred from how different media sources portray different subjects or topics. In this paper, we model visual political bias in contemporary media sources at scale, using webly supervised data. We collect a dataset of over one million unique images and associated news articles from left- and right-leaning news sources, and develop a method to predict the image’s political leaning. This problem is particularly challenging because of the enormous intra-class visual and semantic diversity of our data. We propose two stages of training to tackle this problem. In the first stage, the model is forced to learn relevant visual concepts that, when joined with document embeddings computed from articles paired with the images, enable the model to predict bias. In the second stage, we remove the requirement of the text domain and train a visual classifier from the features of the former model. We show this two-stage approach that relies on an auxiliary task leveraging text, facilitates learning and outperforms several strong baselines. We present extensive quantitative and qualitative results analyzing our dataset. Our results reveal disparities in how different sides of the political spectrum portray individuals, groups, and topics.

Shoulder disarticulation amputees account for a small portion of upper‐limb amputees, thus little emphasis has been devoted to developing functional prosthesis for this cohort of amputees. In this study, shoulder girdle recognition was investigated with acquired data from electrophysiological (electromyography [EMG]) and low frequency contraction (accelerometer [Acc]) signals from both amputee and non‐amputee participants. The contribution of this study is based around the contrast of the classification accuracy (CA) for different sensor configurations using a unique set of signal features. It was seen that the fusion of the EMG‐Acc produced an enhancement in the CA in the range of 10%–20%, depending on which windowing parameters were considered. From this, it was seen that the best combination of a windowing scheme and classifier would likely be for the 350 ms and spectral regression discriminant analysis, with a fusion of the EMG‐Acc information. The results have thus provided evidence that the two sensors can be combined and used in practice for prosthesis control. Taking a holistic view on the study, the authors conclude by providing a framework on how the shoulder motion recognition could be combined with neuromuscular reprogramming to contribute towards easing the cognitive burden of amputees during the prosthesis control process.

Background No simple staging system has emerged for basal cell carcinomas (BCCs), since they do not follow the TNM process, and practitioners failed to agree on simple clinical or pathological criteria as a basis for a classification. Operational classification of BCCs is required for decision‐making, trials and guidelines. Unsupervised clustering of real cases of difficult‐to‐treat BCCs (DTT‐BCCs; part 1) has demonstrated that experts could blindly agree on a five groups classification of DTT‐BCCs based on five patterns of clinical situations. Objective Using this five patterns to generate an operational and comprehensive classification of BCCs. Method Testing practitioner's agreement, when using the five patterns classification to ensure that it is robust enough to be used in the practice. Generating the first version of a staging system of BCCs based on pattern recognition. Results Sixty‐two physicians, including 48 practitioners and the 14 experts who participated in the generation of the five different patterns of DTT‐BCCs, agreed on 90% of cases when classifying 199 DTT‐BCCs cases using the five patterns classification (part 1) attesting that this classification is understandable and usable in practice. In order to cover the whole field of BCCs, these five groups of DTT‐BCCs were added a group representing the huge number of easy‐to‐treat BCCs, for which sub‐classification has little interest, and a group of very rare metastatic cases, resulting in a four‐stage and seven‐substage staging system of BCCs. Conclusion A practical classification adapted to the specificities of BCCs is proposed. It is the first tumour classification based on pattern recognition of clinical situations, which proves to be consistent and usable. This EADO staging system version 1 will be improved step by step and tested as a decision tool and a prognostic instrument.

Federated learning (FL) is an emerging distributed artificial intelligence (AI) algorithm. It can train a global model with multiple participants and at the same time ensure the privacy of the participants’ data. Thus, FL provides a solution for the problems faced by data silos. Existing federated learning algorithms face two significant challenges when dealing with (1) non-independent and identically distributed (non-IID) data, and (2) data with noise or without preprocessing. To address these challenges, a novel federated learning approach based on the confidence of federated Kalman filters is proposed and is referred to as FedCK in this paper. Firstly, this paper proposes a deep Generative Adversarial Network with an advanced auxiliary classifier as a pre-training module. The Non-IID increases the discreteness of the parameters of local models, it is difficult for FL to aggregate an excellent global model. The pre-training module proposed in this paper can deeply mine hidden features and increase the correlation between local model parameters. Secondly, a federated learning framework based on Federated Kalman Filter (FKF) is proposed in this paper. Because the general federation average aggregation algorithm cannot identify the model parameters with noise. This paper uses the idea of FKF to propose a set of adaptive confidence to improve the fault tolerance of FL. Experiments carried out on the MNIST, CIFAR-10 and SVHN datasets demonstrate that FedCK has better robustness and accuracy than classical federated learning methods.

With the rapid development of Human-computer interaction, automatic emotion recognition based on multichannel electroencephalography (EEG) signals has attracted much attention in recent years. However, many existing studies on EEG-based emotion recognition ignore the correlation information between different EEG channels and cannot fully capture the contextual information of EEG signals. In this paper, a novel multi-feature fusion network is proposed, which consists of spatial and temporal neural network structures to learn discriminative spatio-temporal emotional information to recognize emotion. In this experiment, two common types of features, time-domain features (Hjorth, Differential Entropy, Sample Entropy) and frequency domain features (Power Spectral Density), are extracted. Then, to learn the spatial and contextual information, a convolution neural network, inspired by GoogleNet with inception structure, was adopted to capture the intrinsic spatial relationship of EEG electrodes and contextual information, respectively. Fully connected layers are used for feature fusion, instead of the SoftMax function, SVM is selected to classify the high-level emotion features. Finally, to evaluate the proposed method, we conduct leave-one-subject-out EEG emotion recognition experiments on the DEAP dataset, and the experiment results show that the proposed method achieves excellent performance and average emotion recognition accuracies of 80.52% and 75.22% in the valence and arousal classification tasks of the DEAP database, respectively.

Crowd counting plays a significant role in crowd monitoring and management, which suffers from various challenges, especially in crowd-scale variations and background interference issues. Therefore, we propose a method named depth and edge auxiliary learning for still image crowd density estimation to cope with crowd-scale variations and background interference problems simultaneously. The proposed multi-task framework contains three sub-tasks including the crowd head edge regression, the crowd density map regression and the relative depth map regression. The crowd head edge regression task outputs distinctive crowd head edge features to distinguish crowd from complex background. The relative depth map regression task perceives crowd-scale variations and outputs multi-scale crowd features. Moreover, we design an efficient fusion strategy to fuse the above information and make the crowd density map regression generate high-quality crowd density maps. Various experiments were conducted on four main-stream datasets to verify the effectiveness and portability of our method. Experimental results indicate that our method can achieve competitive performance compared with other superior approaches. In addition, our proposed method improves the counting accuracy of the baseline network by $$15.6\%$$ .

In recent years, the development of deep neural networks is in full gear in the fields of computer vision, natural language processing, and others. However, the existence of adversarial examples brings risks to the completion of these tasks, which is also a huge obstacle to implement deep learning applications in the real world. In order to solve the aforementioned problems and improve the robustness of neural networks, a novel defense network based on generative adversarial networks (GANs) is proposed. First, we use generators to eliminate disturbances of adversarial samples and utilize multi-scale discriminators to classify images of different scales to better assist the generator to produce high-quality images. Then, we utilize salient feature extraction model to extract salient maps of both clean examples and adversarial samples, thus improving the denoising effect of the generator by reducing the difference between salient images. The proposed method can guide the generation networks to accurately remove the invisible disturbance and to restore the adversarial samples to clean samples, which not only improves the success rate of classification, but also achieves satisfactory defense effect. Extensive experiments are conducted to compare the defense effect of our proposed method with other defense methods against various attacks. Results show that our method has strong defensive capabilities against the tested attack methods.

The distance measure between intuitionistic fuzzy sets (IFSs) is a concept of very contemporary interest among the researchers in the field of decision-makings, such as pattern recognition, medical diagnosis, and multiattribute decision-making (MADM) problems. Consequently, diverse distance measures are developed and used in determining the similarity and dissimilarity between IFSs. In the existing methods, the distance measures are calculated based on the geometry of the IFSs. However, the IFSs hold information about the elements in a set. As such, some of the existing distance measures are misleading and unreasonable. Hence, in this paper, a nonlinear distance formula is devised to follow the problem definition. Further, by explicitly proving the distance properties, it is being established that the distance formula is a distance measure. Further, theories for the construction of distance measures are developed. The convex combination of two distance measures is also a distance measure is being proved explicitly. Furthermore, based on the proposed distance measures, similarity measures have been developed. Aside from that, an intriguing idea has been introduced, namely, that an infinite number of distance measures can be constructed from a given pair of distance measures. Additionally, the proposed distance and similarity measures are applied to a variety of problems, including medical diagnosis, pattern recognition, and a MADM problem in COVID-19 face mask selection, where the legitimacy and applicability of the proposed advanced distance measure is demonstrated.

Alcohol impairment in traffic crashes is a critical safety concern. Alcohol impairment in non-motorist crashes has been rising in recent years. However, there is not much research focused on cyclis...

Medical image segmentation can effectively overcome human perception with strong personal limitations. The superpixel/voxel segmentation method has strong adaptability and computational efficiency. It can effectively separate the diseased tissue from normal cells or bone and muscle, which is widely used. This paper proposes a superpixel/voxel medical image segmentation algorithm based on regional interlinked value and block (region) merging, which can segment the two-dimensional bone image and three-dimensional brain image. By computing the regional interlinked value, the proposed method can overcome the problem of the initial setting block size in the traditional superpixel/voxel segmentation method. Next, the blocks with the same features are merged. To segment the superpixel/voxel medical image, the final distance with the intensity feature, the location feature, and the gradient feature is considered. Compared with most state-of-the-art algorithms, the proposed method has strong robustness and efficiency, which provides a solid foundation for further image segmentation.

Deep neural networks have demonstrated remarkable recognition results on video classification, however great improvements in accuracies come at the expense of large amounts of computational resources. In this paper, we introduce LiteEval for resource efficient video recognition. LiteEval is a coarse-to-fine framework that dynamically allocates computation on a per-video basis, and can be deployed in both online and offline settings. Operating by default on low-cost features that are computed with images at a coarse scale, LiteEval adaptively determines on-the-fly when to read in more discriminative yet computationally expensive features. This is achieved by the interactions of a coarse RNN and a fine RNN, together with a conditional gating module that automatically learns when to use more computation conditioned on incoming frames. We conduct extensive experiments on three large-scale video benchmarks, FCVID, ActivityNet and Kinetics, and demonstrate, among other things, that LiteEval offers impressive recognition performance while using significantly less computation for both online and offline settings.

Visual urban perception has recently attracted a lot of research attention owing to its importance in many fields. Traditional methods for visual urban perception mostly need to collect adequate training instances for newly-added perception attributes. In this paper, we consider a novel formulation, zero-shot learning, to free this cumbersome curation. Based on the idea of different images containing similar objects are more likely to possess the same perceptual attribute, we learn the semantic correlation space formed by objects semantic information and perceptual attributes. For newly-added attributes, we attempt to synthesize their prototypes by transferring similar object vector representations between the unseen attributes and the training (seen) perceptual attributes. For this purpose, we leverage a deep semantic-aware network for zero-shot visual urban perception model. It is a new two step zero-shot learning architecture, which includes supervised visual urban perception step for training attributes and zero-shot prediction step for unseen attributes. In the first step, we highlight the important role of semantic information and introduce it into supervised deep visual urban perception framework for training attributes. In the second step, we use the visualization techniques to obtain the correlations between semantic information and visual perception attributes from the well trained supervised model, and learn the prototype of unseen attributes and testing images to predict perception score on unseen attributes. The experimental results on a large-scale benchmark dataset validate the effectiveness of our method.

Most recommender systems are essentially a contextually accurate matching between users and items with similarities. Thus, similarity is particularly important to the recommendation process. Furthermore, the highest goal of similarity is to simulate the subjective human feeling of similarity, i.e., to simulate objective feature engineering in a way that is as consistent with subjective feeling as possible. By studying the subjective cognition of similarity, we found that the process could be divided into two stages, namely, perception and comprehension. Perception has fuzziness in that deterministic data cannot accurately describe subjective perception and judge emotional tendencies. Second, comprehension has gradations such that a linear model easily underfits the similarity. To address these two problems, we proposed a new implicit social recommendation algorithm based on multilayer fuzzy perception similarity. An extensive experimental study conducted on benchmark datasets showed that the proposed algorithm is very competitive with some of the traditional recommendation algorithms and state-of-the-art neural network algorithms, especially in terms of the obtained rankings.

Person re-identification (ReID) is mainly aimed at establishing correct identity correspondence among moving person collected by multiple cameras. Extending labeled data sets with pseudo-labels is one of the common methods of ReID. However, single evaluation standards and fixed screening pseudo-label methods make pseudo-labels gradually weaken their update rate. Based on that, we propose a semi-supervised adaptive stepwise learning (SSAS) method for accelerating the update of pseudo-labels. Using the concept of Kullback–Leibler divergence, a more global pseudo-label update idea (GPLU) is proposed, an evaluation criterion of pseudo-labels is designed to satisfy two conditions: The first is to use simple tracklets as pseudo-label data in the early stage, and the second is to gradually add complex and diverse tracklets as pseudo-label data in the iterative process. Our proposed adaptive pseudo-label screening strategy steadily improves the recognition accuracy of ReID. In addition, we conduct extensive experiments on canonical data sets and the evaluation results suggest the superiority of our method.

Computer assisted cardiotocography holds a great promise in minimizing human errors thereby improving perinatal outcome. Despite exponential growth (Moore's law) in computing power for decades, this promise remains unrealized. The systematic analyses of studies on computerized cardiotocography offer little guide to future. This analytical review presents a more qualitative discussion of available evidence as well as concepts regarding the development and acceptance of computerized cardiotocography. To begin with, a workable approach would be for computer algorithms to follow the most scientific visual cardiotocography interpretation frameworks incorporating multiple fetal heart rate parameters and uterine contractions. This ability could be studied and form the basis for regulation of computer algorithms. Addition of background risk factors would be another step. This may take form of familiar multi-tier systems or new alternative strategies like the fetal reserve index. "Machine learning" will remain challenging because of complex variability in fetal-maternal conditions, labor characteristics and clinical intervention changing the outcomes. Randomized controlled trials of adequate size may remain very rare. However, prospective and retrospective testing of computer algorithms with careful qualitative and comparative approach would help clinicians and hospital managers in their decisions. Singular parameters like the popular "deceleration area" and "deceleration capacity" have poor predictive value for fetal acidemia or hypoxic injury. Scientific pattern-recognition of important fetal heart rate parameters like decelerations seems crucial for visual as well as computerized cardiotocography. Success of computerizedcardiotocography depends on team effort by the obstetricians with in-depth practical knowledge/experience and skilled artificial intelligence (AI) specialists.

The images captured in an outdoor environment have the possibility of being degraded in the presence of air particles that scatters and absorbs light. This degradation of images introduces pixel distortion, blurring, contrast attenuation that results in poor visibility. The foggy images also restrict the efficiency associated with the computer vision systems that concentrates on pattern recognition, surveillance and target tracking. In this paper, sparse coding and improved dark channel prior-based deep CNN (SP-IDCP-DCNN) model is proposed for enhancing the visibility of foggy Images that attributes towards efficiency improvement of computer vision systems. In specific, sparse coding is inherited with the merits of regularization and dropout layer for minimizing the overfitting problem achieved based on dimension reduction. This sparse coding is used for generating ridgelet-based components through the utilization of Ricker Wavelet functions. It included an adaptive modification process in the dark channel prior computer for significantly minimizing the need of generating artifacts involved in the process of restoring the image’s visibility. The sparse coding is also responsible for potentially extracting sparse-coded features that attributes towards image visibility enhancement. The simulation results of the proposed SP-IDCP-DCNN Model confirmed its predominance over the existing CNN-based image visibility enhancement schemes in terms of mean opinion score and fog reduction factor.

The understanding of the relationships between pattern recognition and mathematical abilities is important for students in their learning of mathematics and mathematics teachers in the design of th...

From a real-world perspective, missing information is an ordinary scenario in data stream. Generally, missing data generate diverse problems in recognizing the pattern of data (i.e., clustering and classification). Particularly, missing data in data stream is a challenging topic. With imbalanced data, the problem of missing data greatly affects pattern recognition. As a solution to all these issues, this study puts forward an adaptive technique with fuzzy-based information decomposition method, which simultaneously solves the problem of incomplete data and overcomes the imbalanced data stream in a dataset. The main purpose of the proposed fuzzy adaptive imputation approach (FAIA) is to represent the effect of missing values in imbalance data stream and handle the missing data problem in imbalance data stream. FAIA is a single pass method. It considers adaptive selection of intervals based on all observed instances by using the interrelationship of attributes to identify correct interval for computing missing instances. Here, the interrelationship of two attributes means one attribute’s value of an instance depends on another attribute’s value of the same instance. In FAIA, after measuring all interval distances from a certain missing value, the least distance is selected for this missing value. Synthetic data of minority class are generated using the same process of missing value imputation for balancing data that is called oversampling. Instances of the datasets are divided into the chunks in data stream to balance data without any ensemble of previous chunks because missing values may misguide the future chunks. To demonstrate the performance of FAIA, the experiment is divided into three parts: missing data imputation, imbalanced information for offline data for data stream, and imbalanced information with missing value for offline data. Eleven numerical datasets with different dimensions from various repositories are considered for the computing performance of missing data imputation and imbalanced data without data stream. Four different datasets are also used to measure the performance of imbalanced data stream. In maximum measuring cases, the proposed method outperforms.

Pythagorean fuzzy set (PFS) is a more flexible and effective way than intuitionistic fuzzy set (IFS) to seize indeterminacy. In this context, the main aim is to develop a number of new diverse types of PFS similarity measures which not only satisfy the well-known axioms, but also conquer the division-by-zero problem successfully. Moreover, the developed measures are based on two concepts of t-norm and s-norm together with the distance measure between PFSs. In order for further clarifying the role of proposed PFS similarity measures, we assess here two aspects of comparison: the microscopy aspect and the macroscopy aspect. The latter aspect allows us to know how the results are actually obtained on the basis of structural form of similarity measures, and the former aspect enables us to judge about the results of similarity measures without considering how they have been concluded. We then investigate a number of desirable properties of proposed PFS similarity measures, and show their effectiveness compared to the existing ones by encountering both of existing and newly constructed measures in some case studies concerning pattern recognition and medical diagnosis.

Internet of Things (IoT) in the agriculture field provides crops-oriented data sharing and automatic farming solutions under single network coverage. The components of IoT collect the observable data from different plants at different points. The data gathered through IoT components, such as sensors and cameras, can be used to be manipulated for a better farming-oriented decision-making process. This work proposes a system that observes the crops’ growth and leaf diseases continuously for advising farmers in need. To provide analytical statistics on plant growth and disease patterns, the proposed framework uses machine learning (ML) techniques, such as support vector machine (SVM) and convolutional neural network (CNN). This framework produces efficient crop condition notifications to terminal IoT components which are assisting in irrigation, nutrition planning, and environmental compliance related to the farming lands. In this regard, this work proposes ensemble classification and pattern recognition for crop monitoring system (ECPRC) to identify plant diseases at the early stages. The proposed ECPRC uses ensemble nonlinear SVM (ENSVM) for detecting leaf and crop diseases. In addition, this work performs comparative analysis between various ML techniques, such as SVM, CNN, naive Bayes, and $K$ -nearest neighbors. In this experimental section, the results show that the proposed ECPRC system works optimally compared to the other systems.

Aerial inspection of agricultural regions can provide crucial information to safeguard from numerous obstacles to efficient farming. Farmland anomalies such as standing water, weed clusters, hamper the farming practices, which causes improper use of farm area and disrupts agricultural planning. Monitoring of farmland and crops through Internet-of-Things (IoT)-enabled smart systems has potential to increase the efficiency of modern farming techniques. Unmanned Aerial Vehicle (UAV)-based remote sensing is a powerful technique to acquire farmland images on a large scale. Visual data analytics for automatic pattern recognition from the collected data is useful for developing Artificial intelligence (AI)-assisted farming models, which holds great promise in improving the farming outputs by capturing the crop patterns, farmland anomalies and providing predictive solutions to the inherent challenges faced by farmers. In this work, we propose a deep learning framework AgriSegNet for automatic detection of farmland anomalies using multiscale attention semantic segmentation of UAV acquired images. The proposed model is useful for monitoring of farmland and crops to increase the efficiency of precision farming techniques.

In autonomous drone racing, a drone flies through a gate track at high speed. Some solutions involve the use of camera localisation to control the drone. However, effective localisation is very demanding in processing time, which may compromise the flight speed. To address the latter, we propose a deep learning-based method for camera localisation that processes a small sequence of images of the scene at high-frequency operation. Our solution is a compact convolutional neural network based on the Inception network that uses a sequence of grey-scale images rather than colour images as input; we have called this network ‘GreySeqNet’. Our approach aims at leveraging the localisation process using a small stack of consecutive images fed as input to the network. To save computational effort, we explore the use of grey images instead of colour images, thus saving convolutional layers. We have conducted experiments in a simulated environment to measure the performance of GreySeqNet in different race tracks with variations in gate’s height and position. According to the results obtained in several test runs, our method achieves a camera pose estimation at an average operation frequency of 83 Hz running on GPU and 26 Hz on CPU, with an average camera pose error of 31 cm.

Machine visual intelligence has exploded in recent years. Large-scale, high-quality image and video datasets significantly empower learning-based machine vision models, especially deep-learning models. However, images and videos are usually compressed before being analyzed in practical situations where transmission or storage is limited, leading to a noticeable performance loss of vision models. In this work, we broadly investigate the impact on the performance of machine vision from image and video coding. Based on the investigation, we propose Just Recognizable Distortion (JRD) to present the maximum distortion caused by data compression that will reduce the machine vision model performance to an unacceptable level. A large-scale JRD-annotated dataset containing over 340,000 images is built for various machine vision tasks, where the factors for different JRDs are studied. Furthermore, an ensemble-learning-based framework is established to predict the JRDs for diverse vision tasks under few- and non-reference conditions, which consists of multiple binary classifiers to improve the prediction accuracy. Experiments prove the effectiveness of the proposed JRD-guided image and video coding to significantly improve compression and machine vision performance. Applying predicted JRD is able to achieve remarkably better machine vision task accuracy and save a large number of bits.

Aggregating temporal features from other frames is verified to be very effective for video object detection to overcome the challenges in still images, such as occlusion, motion blur, and rare pose. Currently, proposal-level feature aggregation dominates this direction. However, there are two main problems for the holistic proposal-level feature aggregation. First, the object proposals generated by the region proposal network ignore the useful context information around the object which is proved to be helpful for object classification. Second, the traditional proposal-level feature aggregation regards the proposal as a whole without considering the important object structure information, which makes the similarity comparison between two proposals less effective when occlusion or pose misalignment occurs on proposal objects. To deal with these problems, we propose the Context and Structure Mining Network to better aggregate features for video object detection. In our method, we first encode the spatial-temporal context information into object features in a global manner, which can benefit the object classification. In addition, the holistic proposal is divided into several patches to capture the structure information of the object, and cross patch matching is conducted to alleviate the pose misalignment between objects in target and support proposals. Moreover, an importance weight is learned for each target proposal patch to indicate how informative this patch is for the final feature aggregation, by which the occluded patches can be neglected. This enables the aggregation module to leverage the most important and informative patches to obtain the final feature aggregation. The proposed framework outperforms all the latest state-of-the-art methods on the ImageNet VID dataset with a large margin. This project is publicly available https://github.com/LiangHann/Context-and-Structure-Mining-Network-for-Video-Object-Detection .

Exploiting the prior information is fundamental for image reconstruction in computational hyperspectral imaging (CHI). Existing methods usually unfold the 3D signal as a 1D vector and then handle the prior information among different dimensions in an indiscriminative manner, which inevitably ignores the high-dimensionality nature of the hyperspectral image (HSI) and thus results in poor reconstruction performance. In this paper, we propose a high-order tensor optimization based reconstruction method to boost the quality of CHI. Specifically, we first propose an adaptive dimension-discriminative low-rank tensor recovery (ADLTR) model to exploit the high-dimensionality prior of HSI faithfully. In the ADLTR model, we utilize the 3D tensors as the basic elements to fundamentally preserve the structure information in the spatial and spectral dimensions, introduce a dimension-discriminative low-rankness model to fully characterize the prior in the basic elements, and propose a weight estimation strategy by adaptively exploiting the diversity in each dimension. Then, we develop an optimization framework for the CHI reconstruction by integrating the structure prior in ADLTR with the system imaging principle, which is finally solved via the alternating minimization scheme. Extensive experiments on both synthetic and real data demonstrate that our method outperforms state-of-the-art methods.

The explosively developed era of big-data compels the increasing demand of nonvolatile memory with high efficiency and excellent storage properties. Herein, we fabricated a high-speed photoelectric multilevel memory device for neuromorphic computing. The novel two-dimensional (2D) MoSSe with a unique Janus structure was employed as the channel, and the stack of Al2O3/black phosphorus quantum dots (BPQDs)/Al2O3 was adopted as the dielectric. The storage performance of the resulting memory could be verified by the endurance and retention tests, in which the device could remain stable states of programming and erasing even after 1, 000 cycles and 1, 000 s. The multibit storage could be realized through both different voltage amplitudes and pulse numbers, which could achieve 6 bits (64 distinguishable levels) under pulse width of 50 ns. Furthermore, our memory device also could realize the simulations of synapses in human brain with optical and electric modulations synergistically, such as excitatory post-synaptic current (EPSC), long-term potentiation/depression (LTP/LTD), and spike-timing-dependent plasticity (STDP). Neuromorphic computing was successfully achieved through a high recognition of handwritten digits up to 92.5% after 103 epochs. This research is a promising avenue for the future development of efficient memory and artificial neural network systems.

Some tools such as entropy, divergence measures and similarity measures are applied to real-world phenomena like decision-making, robotics, pattern recognition, clustering, expert and knowledge-based system and medical diagnosis. An intuitionistic fuzzy set (IFS) comprises of membership function and non-membership function, but neutrality function is missing in IFS. Therefore, picture fuzzy set (PFS) is an excellent tool to handle such situations when there are answers like yes, no, abstain and refusal. PFS is the generalization of fuzzy set (FS) and intuitionistic fuzzy set (IFS) and shows better adaptation to various real-world problems. To draw conclusions for these problems, based on discrimination between two probability distributions, tools such as divergence measure play a crucial role. The aim of this study is to propose a divergence measure for picture fuzzy sets with its validity proof and to deliberate its key properties. Besides, the newly developed divergence measure is applied to decision-making in machine learning such as pattern recognition, medical diagnosis and clustering using numerical illustrations. To validate the proposed method and to check its effectiveness, expediency and legitimacy, a comparative analysis is given and also the superiority of the divergence measure is tested over the existing methods by comparing their results.

Community detection in complex networks has been revisited with graph deep learning recently and has attracted great attention. It is often challenging to uncover underlying communities on attributed networks because of the complexity and diversity of graph-structured data. A recent prominent graph deep learning model is graph convolutional network (GCN), which effectively integrates network topology and attribute information in graph representation learning. However, most GCN-based community detection methods are semi-supervised and require a considerable amount of labeled data for training. Here, we propose a weakly-supervised learning method based on GCN for community detection in attributed networks. Our new method integrates the techniques of GCN and label propagation and the latter constructs a balanced label set to uncover underlying community structures with topology and attribute information. The experiments on various real-world networks give a comparison view to evaluate the proposed method. The experimental result demonstrates the proposed method performs more efficiently with a comparative performance over current state-of-the-art community detection algorithms.