Search

Facial expression recognition based on residual networks is important for technologies related to space human-robot interaction and collaboration but suffers from low accuracy and slow computation in complex network structures. To solve these problems, this paper proposes a multiscale feature fusion attention lightweight wide residual network. The network first uses an improved random erasing method to preprocess facial expression images, which improves the generalizability of the model. The use of a modified depthwise separable convolution in the feature extraction network reduces the computational effort associated with the network parameters and enhances the characterization of the extracted features through a channel shuffle operation. Then, an improved bottleneck block is used to reduce the dimensionality of the upper layer network feature map to further reduce the number of network parameters while enhancing the network feature extraction capability. Finally, an optimized multiscale feature lightweight attention mechanism module is embedded to further improve the feature extractability of the network for human facial expressions. The experimental results show that the accuracy of the model is 73.21%, 98.72%, and 95.21% on FER2013, CK+ and JAFFE, respectively, with a covariance of 10.14 M. Compared with other networks, the model proposed in this paper has faster computing speed and better accuracy at the same time. [ABSTRACT FROM AUTHOR]

The recognition of terrain and outdoor complex environments based on vision sensors is a key technology in practical robotics applications, and forms the basis of autonomous navigation and motion planning. While traditional machine learning methods can be applied to outdoor terrain recognition, their recognition accuracy is low. In order to improve the accuracy of outdoor terrain recognition, methods based on deep learning are widely used. However, the network structure of deep learning methods is very complex, and the number of parameters is large, which cannot meet the actual operating requirements of of unmanned systems. Therefore, in order to solve the problems of poor real-time performance and low accuracy of deep learning algorithms for terrain recognition, this paper proposes the efficient EfferDeepNet network for pixel level terrain recognition in order to realize global perception of outdoor environment. First, this method uses convolution kernels with different sizes in the depthwise separable convolution (DSC) stage to extract more semantic feature information. Then, an attention mechanism is introduced to weight the acquired features, focusing on the key local feature areas. Finally, in order to avoid redundancy due to a large number of features and parameters in the model, this method uses a ghost module to make the network more lightweight. In addition, to solve the problem of pixel level terrain recognition having a negative effect on image boundary segmentation, the proposed method integrates an enhanced feature extraction network. Experimental results show that the proposed EfferDeepNet network can quickly and accurately perform global recognition and semantic segmentation of terrain in complex environments. [ABSTRACT FROM AUTHOR]

In order to categorize feature classes by capturing subtle differences, hyperspectral images (HSIs) have been extensively used due to the rich spectral-spatial information. The 3D convolution-based neural networks (3DCNNs) have been widely used in HSI classification because of their powerful feature extraction capability. However, the 3DCNN-based HSI classification approach could only extract local features, and the feature maps it produces include a lot of spatial information redundancy, which lowers the classification accuracy. To solve the above problems, we proposed a spatial attention network (SATNet) by combining 3D OctConv and ViT. Firstly, 3D OctConv divided the feature maps into high-frequency maps and low-frequency maps to reduce spatial information redundancy. Secondly, the ViT model was used to obtain global features and effectively combine local-global features for classification. To verify the effectiveness of the method in the paper, a comparison with various mainstream methods on three publicly available datasets was performed, and the results showed the superiority of the proposed method in terms of classification evaluation performance. [ABSTRACT FROM AUTHOR]

In recent years, many deep learning algorithms based on seismic signals have been proposed to solve the moving target recognition problem in unattended ground sensor systems. Despite the excellent performance of these deep networks, most of them can only be deployed on cloud-based devices and cannot be deployed on low-power hardware devices due to the large network size. Second, since seismic signals are affected by the terrain, employing only seismic signals as reconnaissance means for unattended ground sensors cannot achieve multiterrain-type adaptability. In response, this paper proposes an MFC-TinyNet method facing a multiterrain. The method adds depthwise separable convolutional layers to the network, which effectively reduces the size of the network while keeping the target recognition accuracy constant, and solves the problem that the model is difficult to deploy on low-power hardware. It also uses the Mel-frequency spectrum feature extraction method to fuse sound and seismic signals to improve the accuracy of the model's moving target recognition on a multiterrain. Experiments demonstrate that the method can combine the two advantages of the small network model and multiterrain applicability. [ABSTRACT FROM AUTHOR]

At present, there are some problems in underwater low light image, such as low contrast, blurred details, color distortion. In the process of low illumination image enhancement, there are often problems such as artifacts, loss of edge details and noise amplification in the enhanced image. In this paper, we propose an underwater low-light enhancement algorithm based on U-shaped generative adversarial network, combined with bright channel prior and attention mechanism, to address the problems. For the problems of uneven edges and loss of details that occurred in traditional enhanced images, we propose a two-channel fusion technique for the input channel. Aiming at the problems of brightness, texture and color distortion in enhanced images, we propose a feature extraction technique based on the attention mechanism. For the problems of noise in enhanced output images, we propose a multi-loss function to constrain the network. The method has a wide range of applications in underwater scenes with large depth. This method can be used for target detection or biological species identification in underwater low light environment. Through the enhancement experiment of underwater low light image, the proposed method effectively solves the problems of low contrast, blurred details, color distortion, etc. of underwater low light image. Finally, we performed extensive comparison experiments and completed ablation experiments on the proposed method. The experimental results show that the proposed method is optimal in human visual experience and underwater image quality evaluation index. [ABSTRACT FROM AUTHOR]

In close-range air combat, highly reliable trajectory prediction results can help pilots to win victory to a great extent. However, traditional trajectory prediction methods can only predict the precise location that the target aircraft may reach, which cannot meet the requirements of high-precision, real-time trajectory prediction for highly maneuvering targets. To this end, this paper proposes an attention-based convolution long sort-term memory (AttConvLSTM) network to calculate the arrival probability of each space in the reachable area of the target aircraft. More specifically, by segmenting the reachable area, the trajectory prediction problem is transformed into a classification problem for solution. Second, the AttConvLSTM network is proposed as an efficient feature extraction method, and combined with the multi-layer perceptron (MLP) to solve this classification problem. Third, a novel loss function is designed to accelerate the convergence of the proposed model. Finally, the flight trajectories generated by experienced pilots are used to evaluate the proposed method. The results indicate that the mean absolute error of the proposed method is no more than 45.73m, which is of higher accuracy compared to other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Multimodal tasks based on attention mechanism and language face numerous problems. Based on multimodal hierarchical attention mechanism and genetic neural network, this paper studies the application of image segmentation algorithm in data completion and 3D scene reconstruction. The algorithm refers to the process of concentrating attention that humans subjectively pay attention to and calculates the difference between each pixel in the genetic neural network test image in the color space and the average value of the target image, which solves the problem of static feature maps and dynamic feature maps of image sequences. In addition, in view of the problem that the number of attention enhancement feature extraction modules is too large and the parameters are too large, the recursive mechanism is used as the feature extraction branch, and new model parameters are not added when the network depth is increased. The simulation results show that the accuracy of the improved image saliency detection algorithm based on the attention mechanism reaches 89.7%, and the difference between the average value of the single-point pixel and the target image is reduced to 0.132, which further promotes the practicability and reliability of the image segmentation model. [ABSTRACT FROM AUTHOR]

Dimension reduction is a preprocessing step in machine learning for eliminating undesirable features and increasing learning accuracy. In order to reduce the redundant features, there are data representation methods, each of which has its own advantages. On the other hand, big data with imbalanced classes is one of the most important issues in pattern recognition and machine learning. In this paper, a method is proposed in the form of a cost-sensitive optimization problem which implements the process of selecting and extracting the features simultaneously. The feature extraction phase is based on reducing error and maintaining geometric relationships between data by solving a manifold learning optimization problem. In the feature selection phase, the cost-sensitive optimization problem is adopted based on minimizing the upper limit of the generalization error. Finally, the optimization problem which is constituted from the above two problems is solved by adding a cost-sensitive term to create a balance between classes without manipulating the data. To evaluate the results of the feature reduction, the multi-class linear SVM classifier is used on the reduced data. The proposed method is compared with some other approaches on 21 datasets from the UCI learning repository, microarrays and high-dimensional datasets, as well as imbalanced datasets from the KEEL repository. The results indicate the significant efficiency of the proposed method compared to some similar approaches. [ABSTRACT FROM AUTHOR]

Recently, deep learning has become a promising technique for constructing gesture recognition classifiers from surface electromyography (sEMG) signals in human–computer interaction. In this paper, we propose a gesture recognition method with sEMG signals based on a deep multi-parallel convolutional neural network (CNN), which solves the problem that traditional machine learning methods may lose too much useful information during feature extraction. CNNs provide an efficient way to constrain the complexity of feedforward neural networks by weight sharing and restriction to local connections. Sophisticated feature extraction is to be avoided and hand gestures are to be classified directly. A multi-parallel and multi-convolution layer convolution structure is proposed to classify hand gestures. Experiment results show that in comparison with five traditional machine learning methods, the proposed method could achieve higher accuracy. [ABSTRACT FROM AUTHOR]

To extract more accurate and abundant features of corn disease and solve the problems of rough classification and low recognition accuracy, the attention mechanism is introduced into the field of corn disease recognition. The corn disease recognition model (AT-AlexNet) is proposed based on an attention mechanism. The network was based on AlexNet, and the new down-sampling attention module was constructed to enhance the foreground response of the disease; the Mish activation function was introduced to improve the nonlinear expression of the network; the new module of the full connection layer was designed to reduce the network parameters. In the experiment of the enhanced corn disease datasets, the average recognition accuracy of the attention-based network model AT-AlexNet is 99.35%. The recognition accuracy of using the Mish activation function is 0.65% higher than that of the ReLu activation function. The experiments show that compared with other identification methods, the proposed method has better classification performance for corn diseases. [ABSTRACT FROM AUTHOR]

To solve the problems of low accuracy, low real-time performance, poor robustness and others caused by the complex environment, this paper proposes a face mask recognition and standard wear detection algorithm based on the improved YOLO-v4. Firstly, an improved CSPDarkNet53 is introduced into the trunk feature extraction network, which reduces the computing cost of the network and improves the learning ability of the model. Secondly, the adaptive image scaling algorithm can reduce computation and redundancy effectively. Thirdly, the improved PANet structure is introduced so that the network has more semantic information in the feature layer. At last, a face mask detection data set is made according to the standard wearing of masks. Based on the object detection algorithm of deep learning, a variety of evaluation indexes are compared to evaluate the effectiveness of the model. The results of the comparations show that the mAP of face mask recognition can reach 98.3% and the frame rate is high at 54.57 FPS, which are more accurate compared with the exiting algorithm. [ABSTRACT FROM AUTHOR]

Recently, deep recurrent neural networks have achieved great success in various machine learning tasks, and have also been applied for sound event detection. The detection of temporally overlapping sound events in realistic environments is much more challenging than in monophonic detection problems. In this paper, we present an approach to improve the accuracy of polyphonic sound event detection in multichannel audio based on gated recurrent neural networks in combination with auditory spectral features. In the proposed method, human hearing perception-based spatial and spectral-domain noise-reduced harmonic features are extracted from multichannel audio and used as high-resolution spectral inputs to train gated recurrent neural networks. This provides a fast and stable convergence rate compared to long short-term memory recurrent neural networks. Our evaluation reveals that the proposed method outperforms the conventional approaches. [ABSTRACT FROM AUTHOR]

Graph-based semi-supervised dimension reduction can use the inherent graph structure of samples to propagate label information, and has become a hot research field in machine learning. However, most current methods have strict linear constraints and cannot handle data sampled from nonlinear manifolds; and rely on predefined graphs, which cannot capture the local structure information of data and cannot handle complex non-Gaussian data. To address these issues, this paper proposes a new locality-preserved flexible dimension reduction framework, called Semi-supervised Flexible Adaptive Graph Embedding (SFAG), which learns the embedding space that can preserve the local neighborhood structure by constructing a k 1 -nearest neighbor graph over labeled samples. Then, another k 2 -nearest neighbor graph is constructed on all samples to adaptively construct the optimal graph, clustering labeled and unlabeled embedding sample points with neighborhood relations into the same sub-manifold sharing the same label information. Last but not least, the hard linear projection constraint is relaxed by adding residual terms to obtain not only the nonlinear embedding of the training samples, but also the linear projection matrix applied directly to the out-of-sample. In addition, two different semi-supervised dimension reduction methods for adaptive construction of optimal graphs are proposed based on the SFAG framework. Several evaluation experiments validate the effectiveness of our method in exploring manifold structures and classification tasks. • A novel semi-supervised adaptive flexible dimension reduction method is proposed. • The local and global structure of the data is explored by constructing kNN graphs. • The hard linear constraints on the projection are relaxed by adding a residual term. • An algorithm is designed to solve the norm-constrained problem of feature extraction. [ABSTRACT FROM AUTHOR]

The current demographic development will lead to changing working conditions. The paper at hand presents methods for the investigation of expert knowledge within the company and how to document these information, widely independent from individuals, within a technology based learning environment, using virtual reality (VR) technologies and other media. Expert knowledge is often coded as tacit knowledge [1] and can be hardly verbalized by experts. The extraction of tacit knowledge requires methods that are based on stories. Beyond facts these stories also include tacit knowledge driven by emotions. One of the narrative methods presented in the paper will be the »triad interview« [2]. The documentation of these interviews within texts is not very sustainable and other methods for the documentation of expert knowledge are required. Especially in the field of maintenance that is focused in the paper, virtual reality is a suitable method for the documentation as technicians are familiar with drawings and visual content representations. Virtual reality learning environment can visualize working processes and keep the narrative structure that is required for transferring tacit knowledge. Besides, interactions as a typical characteristic of virtual reality applications allow the design of problem solving tasks. The paper presents the »triad interview« and the »working process analysis« [3] that are used for extracting the relevant information of a working process. For the didactical design of the learning tasks the method of the »complete action« [4] is applied. The paper will present the design of the several phases and their benefit for the learning process. Finally the results and experiences from an evaluation study within the organization will be presented. [ABSTRACT FROM AUTHOR]

Learning useful representations from original features is a key issue in classification tasks. Stacked autoencoders (SAEs) are easy to understand and realize, and they are powerful tools that learn deep features from original features, so they are popular for classification problems. The deep features can further combine the original features to construct more representative features for classification. However, existing SAEs do not consider the original features within the network structure and during training, so the deep features have low complementarity with the original features. To solve the problem, this paper proposes an embedded stacked group sparse autoencoder (ESGSAE) for more effective feature learning. Different from traditional stacked autoencoders, the ESGSAE model considers the complementarity between the original feature and the hidden outputs by embedding the original features into hidden layers. To alleviate the impact of the small sample problem on the generalization of the proposed ESGSAE model, an L 1 regularization-based feature selection strategy is designed to further improve the feature quality. After that, an ensemble model with support vector machine (SVM) and weighted local discriminant preservation projection (w_LPPD) is designed to further enhance the feature quality. Based on the designs above, an embedded stacked group sparse autoencoder ensemble with L 1 regularization and manifold reduction is proposed to obtain deep features with high complementarity in the context of the small sample problem. At the end of this paper, several representative public datasets are used for verification of the proposed algorithm. The results demonstrate that the ESGSAE ensemble model with L 1 regularization and manifold reduction yields superior performance compared to other existing and state-of-the-art feature learning algorithms, including some representative deep stacked autoencoder methods. Specifically, compared with the original features, the representative feature extraction algorithms and the improved autoencoders, the algorithm proposed in this paper can improve the classification accuracy by up to 13.33%, 7.33%, and 9.55%, respectively. The data and codes can be found in: https://share.weiyun.com/Jt7qeORm • A hybrid feature is embedded into the training process to construct a novel deep model. • A group sparsity constraint is introduced to obtain the sparse representations. • The ESGSAE ensemble model is constructed to obtain high complementary features. • A three-step feature learning mechanism is realized. [ABSTRACT FROM AUTHOR]

Relation classification plays an important role in the field of natural language processing (NLP). Previous research on relation classification has verified the effectiveness of using convolutional neural network (CNN) and recurrent neural network (RNN). In this paper, we proposed a model that combine the RNN and CNN (RCNN), which will Give full play to their respective advantages: RNN can learn temporal and context features, especially long-term dependency between two entities, while CNN is capable of catching more potential features. We experiment our model on the SemEval-2010 Task 8 dataset 1 , and the result shows that our method is superior to most of the existing methods. [ABSTRACT FROM AUTHOR]

Clinical narrative text includes information related to a patient's medical history such as chronological progression of medical problems and clinical treatments. A chronological view of a patient's history makes clinical audits easier and improves quality of care. In this paper, we propose a clinical Problem-Action relation extraction method, based on clinical semantic units and event causality patterns, to present a chronological view of a patient's problem and a doctor's action. Based on our observation that a clinical text describes a patient's medical problems and a doctor's treatments in chronological order, a clinical semantic unit is defined as a problem and/or an action relation. Since a clinical event is a basic unit of the problem and action relation, events are extracted from narrative texts, based on the external knowledge resources context features of the conditional random fields. A clinical semantic unit is extracted from each sentence based on time expressions and context structures of events. Then, a clinical semantic unit is classified into a problem and/or action relation based on the event causality patterns of the support vector machines. Experimental results on Korean discharge summaries show 78.8% performance in the F1-measure. This result shows that the proposed method is effectively classifies clinical Problem-Action relations. [ABSTRACT FROM AUTHOR]

The zero-shot image classification (ZSIC) is designed to solve the classification problem when the sample is very small, or the category is missing. A common method is to use attribute or word vectors as a priori category features (auxiliary information) and complete the domain transfer from training of seen classes to recognition of unseen classes by building a mapping between image features and a priori category features. However, feature extraction of the whole image lacks discrimination, and the amount of information of single attribute features or word vector features of categories is insufficient, which makes the matching degree between image features and prior class features not high and affects the accuracy of the ZSIC model. To this end, a spatial attention mechanism is designed, and an image feature extraction module based on this attention mechanism is constructed to screen critical features with discrimination. A semantic information fusion method based on matrix decomposition is proposed, which first decomposes the attribute features and then fuses them with the extracted word vector features of a dataset to achieve information expansion. Through the above two improvement measures, the classification accuracy of the ZSIC model for unseen images is improved. The experimental results on public datasets verify the effect and superiority of the proposed methods. [ABSTRACT FROM AUTHOR]

Existing panoptic segmentation networks based on multiscale methods do not distinguish shared features according to different scales, which leads to a loss of feature information and suboptimal results. To solve this problem, the authors improve on UPSNet (a unified panoptic segmentation network) by introducing a multiscale fusion module cascaded between layers into the semantic segmentation head of UPSNet. On the one hand, this enriches the feature information for differently scaled features by cascading the feature information of adjacent scales, and on the other hand, the feature information at each scale can be combined adaptively to achieve a better segmentation effect. Furthermore, a frequency domain attention mechanism is introduced into the backbone network of UPSNet to improve the feature extraction ability of the backbone network by learning more frequency domain feature information. The experimental results of the authors' improved network on the Common Objects in Context (COCO) and Cityscapes data sets show that compared with UPSNet, the performance is significantly improved: the panoptic quality is improved by 0.9% and 1.6%, respectively. [ABSTRACT FROM AUTHOR]

Machine learning becomes increasingly promising in specific emitter identification (SEI), particularly in feature extraction and target recognition. Traditional features, such as radio frequency (RF), pulse amplitude (PA), power spectral density (PSD), and etc., usually show limited recognition effects when only a slight difference exists in radar signals. Numerous two-dimensional features on transform domain, like various time-frequency representation and ambiguity function are used to augment information abundance, whereas the unacceptable computational burden usually emerges. To solve this problem, some artfully handcrafted features in transformed domain are proposed, like representative slice of ambiguity function (AF-RS) and compressed sensing mask (CS-MASK), to extract representative information that contributes to machine recognition task. However, most handcrafted features only utilizing neural network as a classifier, few of them focus on mining deep informative features from the perspective of machine cognition. Such feature extraction that is based on human cognition instead of machine cognition may probably miss some seemingly nominal texture information which actually contributes greatly to recognition, or collect too much redundant information. In this paper, a novel data-driven feature extraction is proposed based on machine cognition (MC-Feature) resort to saliency detection. Saliency detection exhibits positive contributions and suppresses irrelevant contributions in a transform domain with the help of a saliency map calculated from the accumulated gradients of each neuron to input data. Finally, positive and irrelevant contributions in the saliency map are merged into a new feature. Numerous experimental results demonstrate that the MC-feature can greatly strengthen the slight intra-class difference in SEI and provides a possibility of interpretation of CNN. [ABSTRACT FROM AUTHOR]

Distant supervision (DS) automatically annotates free text with relation mentions from existing knowledge bases (KBs), providing a way to alleviate the problem of insufficient training data for relation extraction in natural language processing (NLP). However, the heuristic annotation process does not guarantee the correctness of the generated labels, promoting a hot research issue on how to efficiently make use of the noisy training data. In this paper, we model two types of biases to reduce noise: (1) bias-dist to model the relative distance between points (instances) and classes (relation centers); (2) bias-reward to model the possibility of each heuristically generated label being incorrect. Based on the biases, we propose three noise tolerant models: MIML-dist, MIML-dist-classify, and MIML-reward, building on top of a state-of-the-art distantly supervised learning algorithm. Experimental evaluations compared with three landmark methods on the KBP dataset validate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

In order to effectively solve the problems of high error rate, long time consuming, and low accuracy of feature extraction in current painting image classification methods, a painting image classification method based on transfer learning and feature fusion was proposed. The global characteristics of the painting picture, such as color, texture, and form, are extracted. The SIFT method is used to extract the painting's local features, and the global and local characteristics are normalized and merged. The painting images are preliminarily classified using the result of feature fusion, the deterministic and nondeterministic samples are divided, and the estimated Gaussian model parameters are transferred to the target domain via a transfer learning algorithm to alter the distribution of nondeterministic samples, completing the painting image classification. Experimental results show that the proposed method has a low error rate and low feature extraction time and a high accuracy rate of painting image classification. [ABSTRACT FROM AUTHOR]

Online social media have aroused much research interest in recent years. In contrast to previous work that focused on the detection of emerging topics, this article undertakes the prediction of active users in online social events, which is so far rarely explored. This prediction task is formulated as a binary classification problem that built on real-world tweet streams, taking Ferguson event and New York Chockhold event as examples. Then, a comprehensive user feature system is designed to characterize the events’ online participants, which includes not only basic statistical characteristics and image-pixel-level features, but also some emotional features and personality features. Next, the Weighted Random Forest (Weighted-RF) classifier is adopted to solve the classification problem. Based on the user feature system and the classifier, the experience of a previous event can be archived and applied to the prediction of later similar events. Experimental results show that the Weighted-RF trained by samples of Ferguson event can effectively predict active users in NYC event, with an AUC value around 0.8392. Besides, the image-content based personality model provides a new tool for depicting user portraits, which further contributes to the quantitative analysis of online social events. [ABSTRACT FROM AUTHOR]

In many machine learning algorithms, a major assumption is that the training and the test samples are in the same feature space and have the same distribution. However, for many real applications this assumption does not hold. In this paper, we survey the problem where the training samples and the test samples are from different distributions. This problem can be referred as domain adaptation . The training samples, always with labels, are obtained from what is called source domains, while the test samples, which usually have no labels or only a few labels, are obtained from what is called target domains. The source domains and the target domains are different but related to some extent; the learners can learn some information from the source domains for the learning of the target domains. We focus on the multi-source domain adaptation problem where there is more than one source domain available together with only one target domain. A key issue is how to select good sources and samples for the adaptation. In this survey, we review some theoretical results and well developed algorithms for the multi-source domain adaptation problem. We also discuss some open problems which can be explored in future work. [ABSTRACT FROM AUTHOR]

Despite the benefits of presence, immersive virtual reality is still too limiting and ineffective to be widely adopted in people׳s everyday lives. One important reason for this is its inability to handle highly diverse 3D interaction tasks, such as object manipulation from different scales, perspectives, reference frames, and dimensions. Inspired by preliminary development of an immersive world builder tool, this paper offers an in-depth presentation of our solution to this problem, using a coordinated, tablet- and HMD-based, hybrid virtual environment (HVE) system. Wearing a non-occlusive HMD, the user is able to view and interact with a tablet mounted on the non-dominant forearm, which provides a multi-touch interaction surface, as well as an exocentric God view of the virtual world. To reduce transition gaps across 3D interaction tasks and interfaces, four coordination mechanisms are proposed, two of which were implemented, and one was evaluated in a user study featuring complex level-editing tasks. Based on subjective ratings, task performance, interview feedback, and video analysis, we found that having multiple seamless Interaction Contexts (ICs) with complementary benefits can lead to good performance and user experience, despite the complexity of learning and using the hybrid system. The results also suggest keeping 3DUI tasks synchronized across the ICs, as this can help users understand their relationships, smoothen within- and between-task IC transitions, and inspire more creative use of different interfaces. Finally, based on these findings, a four-step design process is proposed to aid the process of designing and implementing HVE systems, as well as applying them to appropriate task scenarios. [ABSTRACT FROM AUTHOR]

The encoder–decoder framework is prevalent in existing remote-sensing image captioning (RSIC) models. The appearance of attention mechanisms brings significant results. However, current attention-based caption models only build up the relationships between the local features without introducing the global visual feature and removing redundant feature components. It will cause caption models to generate descriptive sentences that are weakly related to the scene of images. To solve the problems, this article proposed a global visual feature-guided attention (GVFGA) mechanism. First, GVFGA introduces the global visual feature and fuses them with local visual features to build up their relationships between them. Second, an attention gate utilizing the global visual feature is proposed in GVFGA to filter out redundant feature components in the fused image features and provide more salient image features. In addition, to relieve the hidden state’s burden, a linguistic state (LS) is proposed to specifically provide textual features, making the hidden state only guiding visual–textual attention process. What’s more, to further refine the fusion of visual features and textual features, a LS-Guided Attention (LSGA) mechanism is proposed. It can also filter out the irrelevant information in the fused visual–textual feature with the help of an attention gate. The experimental results show that this proposed image captioning model can achieve better results on three RSIC datasets, UCM-Captions, Sydney-Captions, and RSICD datasets. [ABSTRACT FROM AUTHOR]

Given the prior information of the target, hyperspectral target detection focuses on exploiting spectral differences to separate objects of interest from the background, which can be treated as information retrieval (IR) task in machine learning (ML). Most traditional detection methods work in the original feature space and rely heavily on specific assumptions, which cannot guarantee effective extraction of features for the target and background in hyperspectral images (HSIs). Mass estimation (ME) is a base modeling mechanism that has been proven to effectively solve problems in IR and is not restricted by specific assumptions. In this article, we propose a novel target detection method through ensemble-based IR with ME (EIRME). By directly deriving the ordering from a sample set to rank data points, ME provides a simple and straightforward ranking measure to ensure that points similar to the given target are far away from dissimilar points. For the estimation of mass distribution, the proposed method utilizes a tree-structured mapping to generate a feature space, in which the separability of the target and background is further improved. In particular, to break through the technical difficulty that the direct migration of IR methods with mass measure cannot specifically meet the high-precision requirements of target detection in HSIs, we develop a specialized measurement, topological mass, which innovatively combines the mass measure with tree topology to quantify the spectral difference for detection output. Moreover, the IR with ME based on parallel measurements through ensemble trees provides a robust solution with better generalization capacity and higher precision for hyperspectral target detection, facilitating practical applications. Experimental results on benchmark HSI datasets prove that the specialized measurement that we developed successfully overcomes the drawbacks of the direct migration of IR methods with ME and exhibits unique advantages. In addition, comparisons with the most classic and advanced detection algorithms demonstrate the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Visual Question Answering (VQA) is a task to answer natural language questions tied to the content of visual images. Most recent VQA approaches usually apply attention mechanism to focus on the relevant visual objects and/or consider the relations between objects via off-the-shelf methods in visual relation reasoning. However, they still suffer from several drawbacks. First, they mostly model the simple relations between objects, which results in many complicated questions cannot be answered correctly, because of failing to provide sufficient knowledge. Second, they seldom leverage the harmony cooperation of visual appearance feature and relation feature. To solve these problems, we propose a novel end-to-end VQA model, termed Multi-modal Relation Attention Network (MRA-Net). The proposed model explores both textual and visual relations to improve performance and interpretability. In specific, we devise 1) a self-guided word relation attention scheme, which explore the latent semantic relations between words; 2) two question-adaptive visual relation attention modules that can extract not only the fine-grained and precise binary relations between objects but also the more sophisticated trinary relations. Both kinds of question-related visual relations provide more and deeper visual semantics, thereby improving the visual reasoning ability of question answering. Furthermore, the proposed model also combines appearance feature with relation feature to reconcile the two types of features effectively. Extensive experiments on five large benchmark datasets, VQA-1.0, VQA-2.0, COCO-QA, VQA-CP v2, and TDIUC, demonstrate that our proposed model outperforms state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Due to the actual operation of the wind turbine, the collected fault data sets are limited, and it is difficult to realize fault diagnosis through the correlation between variables. To solve this problem, a fault diagnosis method based on virtual expansion and spherical mapping model is proposed in this paper. Firstly, Hermite interpolation is applied to discrete wind power data samples to obtain an interpolation curve about the characteristics of the sample, and a synchronous sampling method is adopted for the interpolation curve to construct a virtual sample. Then, the features of the virtual sample are mapped to a three-dimensional space. Define the spherical data model and perform spherical fitting in a three-dimensional coordinate system. Finally, feature extraction is performed on the fitted spherical surface for training and testing extreme learning machine (ELM). The distribution law of fault data in the spherical model is summarized. Using the data generated based on the Bootstrap method as a control group, comparative experiments were carried out in back-propagation neural network (BP), Probabilistic Neural Network (PNN), General Regression Neural Network (GRNN), and Support Vector Machine (SVM), which verified the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Data-driven chatter detection techniques avoid complex physical modeling and provide the basis for industrial applications of cutting process monitoring. Among them, feature extraction is the key step of chatter detection, which can compensate for the accuracy disadvantage of machine learning algorithms to some extent if the extracted features are highly correlated with the milling condition. However, the classification accuracy of the current feature extraction methods is not satisfactory, and a combination of multiple features is required to identify the chatter. This limits the development of unsupervised machine learning algorithms for chattering detection, which further affects the application in practical processing. In this paper, the fractal feature of the signal is extracted by structure function method (SFM) for the first time, which solves the problem that the features are easily affected by process parameters. Milling chatter is identified based on k-means algorithm, which avoids the complex process of training model, and the judgment method of milling chatter is also discussed. The proposed method can achieve 94.4% identification accuracy by using only one single signal feature, which is better than other feature extraction methods, and even better than some supervised machine learning algorithms. Moreover, experiments show that chatter will affect the distribution of cutting bending moment, and it is not reliable to monitor tool wear through the polar plot of the bending moment. This provides a theoretical basis for the application of unsupervised machine learning algorithms in chatter detection. [ABSTRACT FROM AUTHOR]

The detection of coffee bean defects is the most crucial step prior to bean roasting. Existing defect detection methods used in the specialty coffee bean industry entail manual screening and sorting, require substantial human resources, and are not standardized. To solve these problems, this study developed a deep learning algorithm to detect defects in coffee beans. The results reveal that when the pooling layer was used to enhance features and reduce neural dimensionality, some of the coffee been features were lost or misclassified. Therefore, a novel dimensionality reduction method was adopted to increase the ability of feature extraction. The developed model also overcame the drawbacks of padding causing blurred image boundaries and the dead neurons causing impeding feature propagation. Images of eight types of coffee beans were used to train and test the proposed detection model. The proposed method was verified to reduce the bias when classifying defects in coffee beans. The detection accuracy rate of the proposed model was 95.2%. When the model was only used to detect the presence of defects, the accuracy rate increased to 100%. Thus, the proposed model is highly accurate in coffee bean defect detection in the classification of eight types of coffee beans. [ABSTRACT FROM AUTHOR]

Typically, classification is conducted on a dataset that consists of numerical features and target classes. For instance, a grayscale image, which is usually represented as a matrix of integers varying from 0 to 255, enables one to apply various classification algorithms to image classification tasks. However, datasets represented as binary features cannot use many standard machine learning algorithms optimally, yet their amount is not negligible. On the other hand, oversampling algorithms such as synthetic minority oversampling technique (SMOTE) and its variants are often used if the dataset for classification is imbalanced. However, since SMOTE and its variants synthesize new minority samples based on the original samples, the diversity of the samples synthesized from binary features is highly limited due to the poor representation of original features. To solve this problem, a preprocessing approach is studied. By converting binary features into numerical ones using feature extraction methods, succeeding oversampling methods can fully display their potential in improving the classifiers' performances. Through comprehensive experiments using benchmark datasets and real medical datasets, it was observed that a converted dataset consisting of numerical features is better for oversampling methods (maximum improvements of accuracy and F1-score were 35.11% and 42.17%, respectively). In addition, it is confirmed that feature extraction and oversampling synergistically contribute to the improvement of classification performance. [ABSTRACT FROM AUTHOR]

In recent decades, deep learning (DL) has become a rapidly growing research direction, redefining the state-of-the-art performances in a wide range of techniques, such as object detection and speech recognition. In the aircraft design, dynamics, and control field, many works hinge on the information-rich data-driven approach, which includes the fusion-based prognostic and health management, the airliner's flight safety monitoring, intelligent sensing, and flight control systems development. While DL provides great potentials to solve these data-driven problems, a systematic review and discussion as to how the DL has been/can be used for these problems are still missing in relation to the rapidly developing and widely used DL techniques. In this article, we aim to address this urgent issue to provide a timely overview of the state-of-the-art for applying DL to the aircraft design, dynamics, and control field. In particular, we briefly introduce five representative DL methods, i.e., deep neural network, deep autoencoder, deep belief network, convolutional neural network, and recurrent neural network. Mathematical definitions for each method are presented, and illustrative applications are also discussed. We then review the existing DL-based works that have appeared in the aircraft design, dynamics, and control field. The review efforts are divided into two major groups, i.e., the own-ship aircraft modeling, wherein the works have been/can be implemented online for the aircraft design/dynamics/control, and other airplanes research works, wherein DL-based schemes provide offline monitoring of the aircraft operation. We then summarize the data sources and DL architectures. Referring to the experiences of DL research works/techniques development in other related fields, future opportunities, challenges, and potential solutions for implementing DL in the aircraft design, dynamics, and control field are also discussed. [ABSTRACT FROM AUTHOR]

The trade-off between feature representation power and spatial localization accuracy is crucial for the dense classification/semantic segmentation of remote sensing images (RSIs). High-level features extracted from the late layers of a neural network are rich in semantic information, yet have blurred spatial details; low-level features extracted from the early layers of a network contain more pixel-level information but are isolated and noisy. It is therefore difficult to bridge the gap between high- and low-level features due to their difference in terms of physical information content and spatial distribution. In this article, we contribute to solve this problem by enhancing the feature representation in two ways. On the one hand, a patch attention module (PAM) is proposed to enhance the embedding of context information based on a patchwise calculation of local attention. On the other hand, an attention embedding module (AEM) is proposed to enrich the semantic information of low-level features by embedding local focus from high-level features. Both proposed modules are lightweight and can be applied to process the extracted features of convolutional neural networks (CNNs). Experiments show that, by integrating the proposed modules into a baseline fully convolutional network (FCN), the resulting local attention network (LANet) greatly improves the performance over the baseline and outperforms other attention-based methods on two RSI data sets. [ABSTRACT FROM AUTHOR]

A change and maintenance method is proposed based on the change propagation model and the procedure model information for solving data maintenance problem of 3D machining procedure model change and to help improve the flexibility of 3D machining procedure model and the reliability of the change result. Design Structure Matrix (DSM) is established by analyzing the relationships between machining features in the machining process route to obtain all possible propagation paths. On the basis of obtained paths, machining features that may be affected and machining procedure models related to machining features are stored by the structured method. Algorithms to solve the problems of adding, deleting and modifying machining features are proposed to realize the change and maintenance of 3D machining procedure model by combining machining procedure model's information in the machining process route. In the end, some numerical examples are given to explain both rationality and feasibility of the proposed approaches. [ABSTRACT FROM AUTHOR]