Search

This paper presents the prediction of aging state of oil impregnated insulating Kraft paper using textural properties. Under conditions of prolonged thermal stress, the insulating capacity of Kraft paper decreases due to carbonization and degradation of cellulose. To analyze these effects, a new accelerated aging kit is developed for the experiments. Four groups of samples are collected, and the morphology is examined. Texture feature is extracted for the samples using normalization and Gray-Level Co-occurrence Matrix (GLCM). Supervised (Support Vector Machine (SVM)) and unsupervised (K-means) machine intelligence methods are trained to classify images based on the feature. In addition, new samples are checked for feasibility of the prediction of aging state. The calculated training accuracy of SVM is 92.67%, and for k-means is 95.0%, on the other hand, the testing accuracy of SVM is 91.0%, and for k-means is 92.6%. Therefore, this machine intelligence-based methodology using image features is effective in classifying the aging state of the Kraft Insulating Paper. [ABSTRACT FROM AUTHOR]

Timely identification of the aging stage of the transformer can effectively ensure the safe operation of the transformer and prevent the aging fault of the transformer. This work proposes the aging diagnosis of transformer using the Raman spectroscopic recurrence plot combined with learning vector quantization (LVQ) neural network. Oil-paper insulation samples at different aging stages are obtained by accelerated thermal aging tests. Acquisition of Raman spectra of insulating oil is from 800 to 3000 cm−1 and converts to the Raman spectroscopic recurrence plot. The gray-level cooccurrence matrix (GLCM) is used to extract the characteristic information of the Raman spectroscopic recurrence plot. The optimal number of neurons in competitive layer of LVQ neural network is determined by the experimental method. A transformer aging diagnosis model based on the Raman spectroscopic recurrence plot is established. The results show that the diagnostic accuracy of the proposed diagnostic model reached 95%. The Raman spectroscopic recurrence plot has more abundant aging “fingerprint” characteristic information and the proposed diagnostic method shows potential for monitoring oil-paper insulation. [ABSTRACT FROM AUTHOR]

Detecting insulation performance through testing the Raman spectroscopy of transformer oil is noninvasive and shows great promise in on-site diagnosis. The information, stored in spectra, that may reflect aging states of insulation aims to be effectively unfolded and provides decision support as well. In this article, a transformation method based on intrinsic product arithmetic is used to transform the 1-D Raman spectral data into 2-D image data. With the help of CNN based on the dimensionality expanding method, diagnosis of oil–paper insulation with different aging states based on the Raman spectroscopy was realized. The network-based oil–paper insulation aging diagnosis model has an accuracy of 95% in ten aging categories of 60 test samples and 100% in four aging categories. The result shows the trained CNN was able to identify the differences of transformer oil samples at each aging stage through visualized Raman spectral image. This method provides a more accurate determination method for diagnosing oil-immersed transformers’ aging conditions; it also develops an available method for a more detailed evaluation of insulation aging levels. [ABSTRACT FROM AUTHOR]

Under long-term synergy effect of multi-factors, especially the thermal stress, insulating paper will be degraded and its insulation performance will decline due to carbonization and degradation of cellulose. This paper presents an optical approach for aging feature extraction of the insulating paper, where one of the image processing methods called texture analysis is utilized. By conducting laboratory accelerated thermal aging tests, insulating paper samples with different aging conditions for both Nomex and Kraft, evaluated with the aging time, are prepared. After taking optical microscopic images of insulating paper samples belong to different aging groups, up to 14 texture features are extracted using the gray-level co-occurrence matrix (GLCM). With different feature selection methods applied, several of them are finally selected to represent the aging condition of insulating. Numerical tests with both supervised and unsupervised algorithms, as well as a linear regression method verifies the validity of these features in characterizing the aging condition of the insulating paper. [ABSTRACT FROM PUBLISHER]

Recycled paper is extensively used worldwide. In the last decades, its market has expanded considerably. The increasing use of recycled paper in papermaking has led to the production of paper containing several types of impurities. Consequently, wastepaper mills are forced to implement quality control schemes for evaluating the incoming wastepaper stock, thus guarantying the specifications of the final product. The main objective of this work is to present a fast and reliable system for identifying different paper types. Therefore, undesirable paper types can be refused, improving the performance of the paper machine and the final quality of the paper manufactured. For this purpose, two fast techniques, i.e., Fourier transform midinfrared (FTIR) and reflectance near infrared (NIR), were applied to acquire the infrared spectra of the paper samples. Next, four processing multivariate methods, i.e., principal component analysis, canonical variate analysis (CVA), extended CVA (ECVA), and support vector machines (SVMs), were employed in the feature-extraction or dimension-reduction stage. Afterward, the k nearest neighbor (kNN) algorithm was used in the classification phase. Experimental results show the usefulness of the proposed methodology and the potential of both FTIR and NIR spectroscopic methods. Using the FTIR spectrum in association with SVM and kNN, the system achieved a maximum classification accuracy of 100%, whereas using the NIR spectrum in association with ECVA or SVM and kNN, the system achieved a maximum classification accuracy of 96.4%. [ABSTRACT FROM PUBLISHER]

Recognizing actions from still images has been popularly studied recently. In this paper, we model an action class as a flexible number of spatial configurations of body parts by proposing a new spatial sum–product network (SPN). First, we discover a set of parts in image collections via unsupervised learning. Then, our new spatial SPN is applied to model the spatial relationship and also the high-order correlations of parts. To learn robust networks, we further develop a hierarchical spatial SPN method, which models pairwise spatial relationship between parts inside subimages and models the correlation of subimages via extra layers of SPN. Our method is shown to be effective on two benchmark data sets. [ABSTRACT FROM PUBLISHER]

In this paper, we investigate a very challenging task of automatically generating presentation slides for academic papers. The generated presentation slides can be used as drafts to help the presenters prepare their formal slides in a quicker way. A novel system called PPSGen is proposed to address this task. It first employs the regression method to learn the importance scores of the sentences in an academic paper, and then exploits the integer linear programming (ILP) method to generate well-structured slides by selecting and aligning key phrases and sentences. Evaluation results on a test set of 200 pairs of papers and slides collected on the web demonstrate that our proposed PPSGen system can generate slides with better quality. A user study is also illustrated to show that PPSGen has a few evident advantages over baseline methods. [ABSTRACT FROM PUBLISHER]

This paper addresses classifying different common partial discharge (PD) types under different acoustic emission (AE) measurement conditions. Four types of PDs are considered for the multi-class classification problem, namely; PD from a sharp point to ground plane, surface discharge, PD from a void in the insulation, and PD from semi parallel planes. The collected AE signals are processed using pattern classification techniques to identify their corresponding PD types. The measurement conditions include the influences of various PD locations, oil temperatures, and having a barrier in the line-of-sight between the PD source and the AE sensor. A recognition rate of 94% is achieved when classifying the different PD types measured at the same conditions. In addition, it has been found that the different PD source locations, oil temperatures, and barrier insertion have an impact on the recognition rate. However, by including AE samples at these different conditions in the training process, a recognition rate around 90% for all cases is achieved. [ABSTRACT FROM AUTHOR]

The prototype O system seamlessly integrates Web searching and browsing on touch-enabled tablets using two-handed gesture interaction, much like handling and marking content on a piece of paper. The system also leverages contextual information to infer user intent and significantly improve search results over traditional keyword-based approaches, enabling users to go beyond websites' "walled garden"' and more fully explore information objects. [ABSTRACT FROM PUBLISHER]

The article reports on research into sketch-understanding systems for tablet personal computers (tablet PC). It discusses the work of the Sketch Understanding Group, of which the author is a member, at the Massachusetts Institute of Technology (MIT) on developing a "magic paper," or "intelligent paper," a paper-like surface that can convert sketches of physical systems into animated computer graphics. Sketch-understanding systems go beyond traditional computer-aided design (CAD) programs that produce diagrams, and beyond handwriting recognition systems.

Effectively evaluating the perceptual quality of dehazed images remains an under-explored research issue. In this paper, we propose a no-reference complex-valued convolutional neural network (CV-CNN) model to conduct automatic dehazed image quality evaluation. Specifically, a novel CV-CNN is employed that exploits the advantages of complex-valued representations, achieving better generalization capability on perceptual feature learning than real-valued ones. To learn more discriminative features to analyze the perceptual quality of dehazed images, we design a dual-stream CV-CNN architecture. The dual-stream model comprises a distortion-sensitive stream that operates on the dehazed RGB image, and a haze-aware stream on a novel dark channel difference image. The distortion-sensitive stream accounts for perceptual distortion artifacts, while the haze-aware stream addresses the possible presence of residual haze. Experimental results on three publicly available dehazed image quality assessment (DQA) databases demonstrate the effectiveness and generalization of our proposed CV-CNN DQA model as compared to state-of-the-art no-reference image quality assessment algorithms. [ABSTRACT FROM AUTHOR]

This paper focuses on the facial micro-expression (FME) generation task, which has potential application in enlarging digital FME datasets, thereby alleviating the lack of training data with labels in existing micro-expression datasets. Despite obvious progress in the image animation task, FME generation remains challenging because existing image animation methods can hardly encode subtle and short-term facial motion information. To this end, we present a facial-prior-guided FME generation framework that takes advantage of facial priors for facial motion generation. Specifically, we first estimate the geometric locations of action units (AUs) with detected facial landmarks. We further calculate an adaptive weighted prior (AWP) map, which alleviates the estimation error of AUs while efficiently capturing subtle facial motion patterns. To achieve smooth and realistic synthesis results, we use our proposed facial prior module to guide motion representation and generation modules in mainstream image animation frameworks. Extensive experiments on three benchmark datasets consistently show that our proposed facial prior module can be adopted in image animation frameworks and significantly improve their performance on micro-expression generation. Moreover, we use the generation technique to enlarge existing datasets, thereby improving the performance of general action recognition backbones on the FME recognition task. Our code is available at https://github.com/sysu19351158/FPB-FOMM. [ABSTRACT FROM AUTHOR]

Small moving objects at far distance always occupy only one or a few pixels in image and exhibit extremely limited visual features, which bring great challenges to motion detection. Highly evolved visual systems endow flying insects with remarkable ability to pursue tiny mates and prey, providing a good template to develop image processing method for small target motion detection. The insects’ excellent sensitivity to small moving objects is believed to come from a class of specific neurons called small target motion detectors (STMDs). However, existing STMD-based methods often experience performance degradation when coping with complex natural scenes. In this paper, we propose a bio-inspired visual system with spatio-temporal feedback mechanism (called Spatio-Temporal Feedback STMD) to suppress false positive background movement while enhancing system responses to small targets. Specifically, the proposed visual system is composed of two complementary subnetworks and a feedback loop. The first subnetwork is designed to extract spatial and temporal movement patterns of cluttered background by neuronal ensemble coding. The second subnetwork is developed to capture small target motion information where its output and signal from the first subnetwork are integrated together via the feedback loop to filter out background false positives in a recurrent manner. Experimental results demonstrate that the proposed spatio-temporal feedback visual system is more competitive than existing methods in discriminating small moving targets from complex natural environments. [ABSTRACT FROM AUTHOR]

Most researchers focus on designing accurate crowd counting models with heavy parameters and computations but ignore the resource burden during the model deployment. A real-world scenario demands an efficient counting model with low-latency and high-performance. Knowledge distillation provides an elegant way to transfer knowledge from a complicated teacher model to a compact student model while maintaining accuracy. However, the student model receives the wrong guidance with the supervision of the teacher model due to the inaccurate information understood by the teacher in some cases. In this paper, we propose a dual-knowledge distillation (DKD) framework, which aims to reduce the side effects of the teacher model and transfer hierarchical knowledge to obtain a more efficient counting model. First, the student model is initialized with global information transferred by the teacher model via adaptive perspectives. Then, the self-knowledge distillation forces the student model to learn the knowledge by itself, based on intermediate feature maps and target map. Specifically, the optimal transport distance is utilized to measure the difference of feature maps between the teacher and the student to perform the distribution alignment of the counting area. Extensive experiments are conducted on four challenging datasets, demonstrating the superiority of DKD. When there are only approximately 6% of the parameters and computations from the original models, the student model achieves a faster and more accurate counting performance as the teacher model even surpasses it. [ABSTRACT FROM AUTHOR]

This paper addresses digital staining and classification of the unstained white blood cell images obtained with a differential contrast microscope. We have data coming from multiple domains that are partially labeled and partially matching across the domains. Using unstained images removes time-consuming staining procedures and could facilitate and automatize comprehensive diagnostics. To this aim, we propose a method that translates unstained images to realistically looking stained images preserving the inter-cellular structures, crucial for the medical experts to perform classification. We achieve better structure preservation by adding auxiliary tasks of segmentation and direct reconstruction. Segmentation enforces that the network learns to generate correct nucleus and cytoplasm shape, while direct reconstruction enforces reliable translation between the matching images across domains. Besides, we build a robust domain agnostic latent space by injecting the target domain label directly to the generator, i.e., bypassing the encoder. It allows the encoder to extract features independently of the target domain and enables an automated domain invariant classification of the white blood cells. We validated our method on a large dataset composed of leukocytes of 24 patients, achieving state-of-the-art performance on both digital staining and classification tasks. [ABSTRACT FROM AUTHOR]

Quality monitoring is an essential element of defect detection in semiconductor manufacturing processes, but semiconductor companies use virtual metrology (VM) in addition to actual metrology to prevent productivity degradation due to the time and costs required to obtain measurements. Past VM studies aimed to predict average wafer measurement values via equipment sensor data and focused on achieving improved predictive performance by selecting or extracting important variables among high-dimensional variables such as equipment sensor data. However, the management of wafer chip quality requires not only average measurement values but also measurement value predictions for chips located at the edges, which are vulnerable to defects. In this paper, we therefore propose a graph attention (GAT) network-based VM model that predicts the measurement values of chips located at wafer edges by constructing graph data with measurement data (i.e., the measurement location information in wafers and the measurement values). To verify the performance of the proposed model, we conduct a comparative experiment with conventional machine learning methods. The experimental results show that the proposed VM model contributes to a predictive performance improvement in terms of the measurement values of chips located at wafer edges. [ABSTRACT FROM AUTHOR]

In the field of reversible data hiding (RDH), how to predict an image and embed a message into the image with smaller distortion are two important aspects. In this paper, we propose a novel and efficient RDH method by innovating an intelligent predictor and an adaptive embedding way. In the prediction stage, we first constructed a convolutional neural network (CNN) based predictor by reasonably dividing an image into four parts. In such a way, each part can be predicted by using the other three parts as the context for the improvement of the prediction performance. Compared with existing predictors, the proposed CNN predictor can use more neighboring pixels for the prediction by exploiting its multi-receptive fields and global optimization capacities. In the embedding stage, we also developed a prediction-error-ordering (PEO) based adaptive embedding strategy, which can better adapt image content and thus efficiently reduce the embedding distortion by elaborately and luminously applying background complexity to select and pair those smaller prediction errors for data hiding. With the proposed CNN prediction and embedding ways, the RDH method presented in this paper provides satisfactory results in improving the visual quality of data hidden images, e.g., the average PSNR value for the Kodak benchmark dataset can reach as high as 63.59 dB with an embedding capacity of 10,000 bits. Extensive experimental results have shown that the RDH method proposed in this paper is superior to those existing state-of-the-art works. [ABSTRACT FROM AUTHOR]

In the field of reversible data hiding (RDH), how to predict an image and embed a message into the image with smaller distortion are two important aspects. In this paper, we propose a novel and efficient RDH method by innovating an intelligent predictor and an adaptive embedding way. In the prediction stage, we first constructed a convolutional neural network (CNN) based predictor by reasonably dividing an image into four parts. In such a way, each part can be predicted by using the other three parts as the context for the improvement of the prediction performance. Compared with existing predictors, the proposed CNN predictor can use more neighboring pixels for the prediction by exploiting its multi-receptive fields and global optimization capacities. In the embedding stage, we also developed a prediction-error-ordering (PEO) based adaptive embedding strategy, which can better adapt image content and thus efficiently reduce the embedding distortion by elaborately and luminously applying background complexity to select and pair those smaller prediction errors for data hiding. With the proposed CNN prediction and embedding ways, the RDH method presented in this paper provides satisfactory results in improving the visual quality of data hidden images, e.g., the average PSNR value for the Kodak benchmark dataset can reach as high as 63.59 dB with an embedding capacity of 10,000 bits. Extensive experimental results have shown that the RDH method proposed in this paper is superior to those existing state-of-the-art works. [ABSTRACT FROM AUTHOR]

In recommendation tasks, we model user preferences by learning node representations (i.e., user and item embeddings) based on the observed user-item interaction data, which is a bipartite graph. Graph Neural Networks (GNNs) are widely used to refine the representations by exploring the topology of the graph: embeddings of neighbors are propagated to each node to reconstruct its embeddings. However, the propagation strategy in existing GNNs is empirical and defective: (1) a substantial proportion of links are missed in the sparse observed graph, which causes ineffective and biased propagation; and (2) the propagation weights are determined by a coarse pre-defined rule, which only takes the degree of nodes into consideration. In this paper, we propose a dense and data-driven propagation mechanism for GNNs. Considering the graph we use to propagate embeddings in recommendation tasks is extremely sparse, we complement it and use the predicted graph as the new propagation tool. We learn the propagation matrix from the data, and propose a Self-propagation Graph Neural Network (SGNN). Since it is very space- and time-consuming to maintain a large and dense propagation matrix, we factorize it for storing and updating. In this paper, we propose three methods to complete the sparse graph and construct the propagation matrix: (1) we complete the graph based on a recommendation model; (2) we measure the node distance based on spectral clustering; (3) we predict missing links of the graph based on predictive embeddings. In SGNN, the embeddings can be propagated to not only the observed neighbors, but also the potential yet unobserved neighbors, and the propagation weights are learned based on the connection strength. Comprehensive experiments on three real-world datasets demonstrate the effectiveness and efficiency of our proposed model: SGNN outperforms recent state-of-the-art GNNs significantly. Codes are available on https://github.com/Wenhui-Yu/LCFN. [ABSTRACT FROM AUTHOR]

Entity resolution (ER) identifies data instances referring to the same real-world entity and has received enormous research attention. In this paper, we examine the task of ER from a broader perspective, with its input extended from textual records, which are conventionally studied in the literature, to other modalities such as check-in sequences, GPS trajectories and surveillance video frames to generate new applications. Our goal in this paper is to design an effective model to uniformly support all these ER applications with different input formats. Technically, we fully exploit the semantic contexts of embedding vectors for the pair of input instances. In particular, we propose an integrated multi-context attention framework that takes into account self-attention, pair-attention and global-attention from three types of context. The idea can be further extended to incorporate attribute attention in order to support structured datasets. We conduct extensive experiments on a diverse class of entity resolutions tasks, including tasks on unstructured, structured and dirty textual records, check-in sequences, GPS trajectories and surveillance video frames. The experimental results verified the effectiveness and generality of our model. When compared with strong baselines in these applications, our model can achieve superior or comparative performance. [ABSTRACT FROM AUTHOR]

Macrovascular invasion (MaVI) is a major threat to survival in hepatocellular carcinoma (HCC), which should be treated as early as possible to ensure safety and efficacy. In this aspect, MaVI prediction can be helpful. However, MaVI prediction is difficult because of the inter-class similarity and intra-class variation of HCC in computed tomography (CT) images. Moreover, existing methods fail to include clinical priori knowledge associated with HCC, leading to incomprehensive information extraction. In this paper, we proposed a prior knowledge-aware fusion network (PKAFnet) to accurately achieve MaVI prediction in CT images. First, a perception module was presented to extract features related to tumor marginal heterogeneity in the graph domain, which contributed to rotation invariance and captured intensity variations of tumor margin. Second, a tumor segmentation network was built to obtain global information of a 3D tumor image and information associated with tumor internal heterogeneity in the image domain. Finally, multi-domain features associated with the tumor margin and tumor region were combined by using a multi-domain attentional feature fusion module. Thus, by incorporating MaVI-related prior knowledge, our PKAFnet can alleviate overfitting, which can improve the discriminative ability. The proposed PKAFnet was validated on a multi-center dataset, and remarkable performance was achieved in an independent testing set. Moreover, the interpretability of perception module and segmentation network were presented in our paper, which illustrated the effectiveness and credibility of PKAFnet. Therefore, the proposed method showed great application potential for MaVI prediction. [ABSTRACT FROM AUTHOR]

To avoid unexpected failures of engineering systems, sensors have been widely used to monitor the degradation process of the systems. A number of studies have been conducted to analyze the collected sensor signals and predict the failure time. However, the existing studies are usually restricted and cannot be adapted to different practical situations. In this paper, we propose a systematic method for degradation modeling and prognosis that can be widely applied in different scenarios. In particular, the proposed method is capable to handle one or multiple sensors, powerful to capture the nonlinear relations between sensor signals and the degradation process with few assumptions, generic to consider multiple failure modes, flexible to deal with unequally spaced sensor measurements or asynchronous signals, and easily understandable with little preprocessing required. The main idea is to predict the failure time of an in-service unit based on a subset of the nearest historical units, where features are extracted from each sensor to describe the progression of sensor signals and local linear regression models are constructed to establish the relation between failure time and the extracted features. The prediction variance is then used as the goodness-of-fit measure, based on which decision-level fusion and feature-level fusion are proposed to combine multiple sensors. A case study with two datasets on the degradation modeling of aircraft engines is conducted which shows satisfactory performance of the proposed method. Note to Practitioners—This paper aims at modeling the collected sensor signals to understand the degradation process of the monitored engineering systems and predict the failure time. The main idea is to measure the similarity of units and predict the failure time of an in-service unit based on a subset of the nearest historical units. The developed method is widely applicable in different practical situations such as multiple sensors, multiple failure modes, asynchronous signals, and missing data. Furthermore, the method requires little preprocessing. There are several steps involved for implementing the proposed method: 1) collecting the sensor signals for historical units and the in-service unit; 2) extracting features from each sensor signal; 3) constructing a local linear model to predict the failure time based on the extracted features, and obtaining the prediction variance on the in-service unit; and 4) combining the information of different sensors using the decision-level fusion or feature-level fusion, if each unit is monitored by multiple sensors. [ABSTRACT FROM AUTHOR]

Radio signal recognition has a wide application in future communication systems and the vehicular communication, whose core is the extraction of signal features such as electromagnetic fingerprints. With the rapid development of artificial intelligence technology, deep learning has made amazing breakthroughs in image recognition, speech recognition and other fields. Deep learning is applied to electromagnetic fingerprint extraction in this paper. Firstly, thousands of the downlink aircraft communications addressing and reporting system (ACARS) signals used for communication between civil aircraft and airport tower are collected and generated. Then a pre-transformation network suitable for electromagnetic signals is constructed to convert one-dimensional signals into two-dimensional feature maps, and afterwards the feature maps are input into the convolutional neural network (CNN) for classification. By adopting the attention modules, the classification results were improved by a few percentage points over the baseline with a little cost. The method proposed in this paper achieves an accuracy rate of 94.1% and can obtain the aircraft type in a shorter time than traditional method. Moreover, the robustness of the proposed model in response to additive Gaussian white noise (AWGN) and phase deviation is studied and tested. [ABSTRACT FROM AUTHOR]

This paper presents a novel methodology to design a secured custom reusable intellectual property (IP) core for the convolutional layer of convolutional neural network (CNN). Since the reusable IP cores used in system-on-chips (SoCs) of consumer electronics (CE) systems are susceptible to the hardware threat of IP counterfeiting. Therefore, this paper also presents the security of the proposed convolutional layer reusable IP core against the threat of IP counterfeiting using facial biometrics. This enables the integration of secured reusable IP cores in the SoCs of CE systems, thereby ensuring the safety of end consumers. In the proposed approach, the convolutional layer IP core is designed through high-level synthesis (HLS) process and secured by embedding secret biometric security information into the design during register allocation phase of the HLS process. The qualitative and quantitative analysis of the proposed approach exhibits significantly lower probability of coincidence (Pc) (up to 47% less) and higher tamper tolerance (1.93E+25) than recent approaches. Further, it offers robust security with zero design overhead. [ABSTRACT FROM AUTHOR]

This paper attacks the challenging problem of video retrieval by text. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described exclusively in the form of a natural-language sentence, with no visual example provided. Given videos as sequences of frames and queries as sequences of words, an effective sequence-to-sequence cross-modal matching is crucial. To that end, the two modalities need to be first encoded into real-valued vectors and then projected into a common space. In this paper we achieve this by proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own. Our novelty is two-fold. First, different from prior art that resorts to a specific single-level encoder, the proposed network performs multi-level encoding that represents the rich content of both modalities in a coarse-to-fine fashion. Second, different from a conventional common space learning algorithm which is either concept based or latent space based, we introduce hybrid space learning which combines the high performance of the latent space and the good interpretability of the concept space. Dual encoding is conceptually simple, practically effective and end-to-end trained with hybrid space learning. Extensive experiments on four challenging video datasets show the viability of the new method. Code and data are available at https://github.com/danieljf24/hybrid_space. [ABSTRACT FROM AUTHOR]

To solve the information explosion problem and enhance user experience in various online applications, recommender systems have been developed to model users’ preferences. Although numerous efforts have been made toward more personalized recommendations, recommender systems still suffer from several challenges, such as data sparsity and cold-start problems. In recent years, generating recommendations with the knowledge graph as side information has attracted considerable interest. Such an approach can not only alleviate the above mentioned issues for a more accurate recommendation, but also provide explanations for recommended items. In this paper, we conduct a systematical survey of knowledge graph-based recommender systems. We collect recently published papers in this field, and group them into three categories, i.e., embedding-based methods, connection-based methods, and propagation-based methods. Also, we further subdivide each category according to the characteristics of these approaches. Moreover, we investigate the proposed algorithms by focusing on how the papers utilize the knowledge graph for accurate and explainable recommendation. Finally, we propose several potential research directions in this field. [ABSTRACT FROM AUTHOR]

In this paper, a data assimilation network is proposed to tackle the challenges of domain generalization for person re-identification (ReID). Most of the existing research efforts only focus on single-dataset issues, and the trained models are difficult to generalize to unseen scenarios. This paper presents a distinctive idea to improve the generality of the model by assimilating three types of images: style-variant images, misaligned images and unlabeled images. The latter two are often ignored in the previous domain generalization ReID studies. In this paper, a non-local convolutional block attention module is designed for assimilating the misaligned images, and an attention adversary network is introduced to correct it. A progressive augmented memory is designed for assimilating the unlabeled images by progressive learning. Moreover, we propose an attention adversary difference loss for attention correction, and a labeling-guide discriminative embedding loss for progressive learning. Rather than designing a specific feature extractor that is robust to style shift as in most previous domain generalization work, we propose a data assimilation meta-learning procedure to train the proposed network, so that it learns to assimilate style-variant images. It is worth mentioning that we add an unlabeled augmented dataset to the source domain to tackle the domain generalization ReID tasks. Extensive experiments demonstrate that our approach significantly outperforms the state-of-the-art domain generalization methods. [ABSTRACT FROM AUTHOR]

Co-salient object detection (CoSOD) aims to detect common salient objects sharing the same attributes in an image group. The key issue of CoSOD is how to model the inter-saliency relations within an image group. The major limitation of previous methods is that they pre-define the group-to-one relations within an image group. In this paper, we propose a new concept of structural inter-saliency relations and solve the CoSOD with deep reinforcement learning framework. Firstly, we design a semantic relation graph (SRG) to model the structural inter-saliency relations. Then the feature selecting agent (FS-agent) aims to select the informative features, which can help the SRG effectively model structural inter-saliency relations. Finally, relation updating agent (RU-agent) progressively updates the SRG to focus on the co-salient relations like human decision-making process. Extensive experiments on co-saliency datasets show that because of well modeling inter-saliency relations in image group, our proposed method achieves superior performance compared to the state-of-the-art methods. We hope that this paper can motivate future research for visual co-analysis tasks. [ABSTRACT FROM AUTHOR]

Surface defect detection plays an important role in manufacturing and has aroused widespread interests. However, it is challenging as defects are highly similar to non-defects. To address this issue, this paper proposes a Region of Interest Attention (RoIA) network based on deep learning for automatically identifying surface defects. It consists of three parts: multi-level feature preservation (MFP) module, region proposal attention (RPA) module, and skip dense connection detection (SDCD) ones, where MFP is designed to differentiate defect features and texture information by feature reserved block, RPA is developed to locate the position of the defects by capturing global and local context information, and SDCD is proposed to better predict defect categories by propagating the fine-grained details from low-level feature map to high-level one. Experimental results conducted on three public datasets (e.g., NEU-DET, DAGM and Magnetic-Tile) demonstrate that the proposed method can significantly improve the detection performance than state-of-the-art ones and achieve an average defect detection accuracy of 99.49%. [ABSTRACT FROM AUTHOR]

Chemical mechanical polishing (CMP) planarizes the chip surface through a synergistic effect of chemical etching and abrasive polishing, and has become increasingly essential in modern semiconductor manufacturing. For better production yield, the industry seeks an accurate CMP model to identify, localize and control surface unevenness caused by the layout dependent effect (LDE) prior to fabrication, which provides valuable information for manufacturing-friendly designs, such as accurate hotspot prediction, design-rule optimization, and smart dummy fill. However, existing models are often overly coarse-grained and inaccurate: They are unable to utilize or analyze detailed topology information in a layout design, and can only characterize the post-CMP chip surface into a rough step-height model. The prediction resolution is also constrained by the featuring window size. In this paper, we propose a novel fine-grained and end-to-end feature-scale CMP modeling paradigm based on fully convolutional neural networks. The model directly analyzes the detailed layout topology without an explicit step of featuring parameter extraction, accurately predicts the rich chip surface micro-topography at a feature-scale precision. We designed and taped-out test chips under 28nm and 40nm processes. Experiments on silicon data demonstrated our model’s accuracy improvements of 65% to 76%, as well as its cross-process-node adaptability. [ABSTRACT FROM AUTHOR]

The vanilla Few-shot Learning (FSL) learns to build a classifier for a new concept from one or very few target examples, with the general assumption that source and target classes are sampled from the same domain. Recently, the task of Cross-Domain Few-Shot Learning (CD-FSL) aims at tackling the FSL where there is a huge domain shift between the source and target datasets. Extensive efforts on CD-FSL have been made via either directly extending the meta-learning paradigm of vanilla FSL methods, or employing massive unlabeled target data to help learn models. In this paper, we notice that in the CD-FSL task, the few labeled target images have never been explicitly leveraged to inform the model in the training stage. However, such a labeled target example set is very important to bridge the huge domain gap. Critically, this paper advocates a more practical training scenario for CD-FSL. And our key insight is to utilize a few labeled target data to guide the learning of the CD-FSL model. Technically, we propose a novel Generalized Meta-learning based Feature-Disentangled Mixup network, namely GMeta-FDMixup. We make three key contributions of utilizing GMeta-FDMixup to address CD-FSL. Firstly, we present two mixup modules – mixup-P and mixup-M that help facilitate utilizing the unbalanced and disjoint source and target datasets. These two novel modules enable diverse image generation for training the model on the source domain. Secondly, to narrow the domain gap explicitly, we contribute a novel feature disentanglement module that learns to decouple the domain-irrelevant and domain-specific features. By stripping the domain-specific features, we alleviate the negative effects caused by the domain inductive bias. Finally, we repurpose a new contrastive learning module, dubbed ConL. ConL prevents the model from only capturing category-related features via introducing contrastive loss. Thus, the generalization ability on novel categories is improved. Extensive experimental results on two benchmarks show the superiority of our setting and the effectiveness of our method. Code and models will be released. [ABSTRACT FROM AUTHOR]

Recent research advances in salient object detection (SOD) could largely be attributed to ever-stronger multi-scale feature representation empowered by the deep learning technologies. The existing SOD deep models extract multi-scale features via the off-the-shelf encoders and combine them smartly via various delicate decoders. However, the kernel sizes in this commonly-used thread are usually “fixed”. In our new experiments, we have observed that kernels of small size are preferable in scenarios containing tiny salient objects. In contrast, large kernel sizes could perform better for images with large salient objects. Inspired by this observation, we advocate the “dynamic” scale routing (as a brand-new idea) in this paper. It will result in a generic plug-in that could directly fit the existing feature backbone. This paper’s key technical innovations are two-fold. First, instead of using the vanilla convolution with fixed kernel sizes for the encoder design, we propose the dynamic pyramid convolution (DPConv), which dynamically selects the best-suited kernel sizes w.r.t. the given input. Second, we provide a self-adaptive bidirectional decoder design to accommodate the DPConv-based encoder best. The most significant highlight is its capability of routing between feature scales and their dynamic collection, making the inference process scale-aware. As a result, this paper continues to enhance the current SOTA performance. Both the code and dataset are publicly available at https://github.com/wuzhenyubuaa/DPNet. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel model for simultaneous stable co-saliency detection (CoSOD) and object co-segmentation (CoSEG). To detect co-saliency (segmentation) accurately, the core problem is to well model inter-image relations between an image group. Some methods design sophisticated modules, such as recurrent neural network (RNN), to address this problem. However, order-sensitive problem is the major drawback of RNN, which heavily affects the stability of proposed CoSOD (CoSEG) model. In this paper, inspired by RNN-based model, we first propose a multi-path stable recurrent unit (MSRU), containing dummy orders mechanisms (DOM) and recurrent unit (RU). Our proposed MSRU not only helps CoSOD (CoSEG) model captures robust inter-image relations, but also reduces order-sensitivity, resulting in a more stable inference and training process. Moreover, we design a cross-order contrastive loss (COCL) that can further address order-sensitive problem by pulling close the feature embedding generated from different input orders. We validate our model on five widely used CoSOD datasets (CoCA, CoSOD3k, Cosal2015, iCoseg and MSRC), and three widely used datasets (Internet, iCoseg and PASCAL-VOC) for object co-segmentation, the performance demonstrates the superiority of the proposed approach as compared to the state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]

Previous 2D saliency detection methods extract salient cues from a single view and directly predict the expected results. Both traditional and deep-learning-based 2D methods do not consider geometric information of 3D scenes. Therefore the relationship between scene understanding and salient objects cannot be effectively established. This limits the performance of 2D saliency detection in challenging scenes. In this paper, we show for the first time that saliency detection problem can be reformulated as two sub-problems: light field synthesis from a single view and light-field-driven saliency detection. This paper first introduces a high-quality light field synthesis network to produce reliable 4D light field information. Then a novel light-field-driven saliency detection network is proposed, in which a Direction-specific Screening Unit (DSU) is tailored to exploit the spatial correlation among multiple viewpoints. The whole pipeline can be trained in an end-to-end fashion. Experimental results demonstrate that the proposed method outperforms the state-of-the-art 2D, 3D and 4D saliency detection methods. Our code is publicly available at https://github.com/OIPLab-DUT/ESCNet. [ABSTRACT FROM AUTHOR]

Extracting robust and discriminative local features from images plays a vital role for long term visual localization, whose challenges are mainly caused by the severe appearance differences between matching images due to the day-night illuminations, seasonal changes, and human activities. Existing solutions resort to jointly learning both keypoints and their descriptors in an end-to-end manner, leveraged on large number of annotations of point correspondence which are harvested from the structure from motion and depth estimation algorithms. While these methods show improved performance over non-deep methods or those two-stage deep methods, i.e., detection and then description, they are still struggled to conquer the problems encountered in long term visual localization. Since the intrinsic semantics are invariant to the local appearance changes, this paper proposes to learn semantic-aware local features in order to improve robustness of local feature matching for long term localization. Based on a state of the art CNN architecture for local feature learning, i.e., ASLFeat, this paper leverages on the semantic information from an off-the-shelf semantic segmentation network to learn semantic-aware feature maps. The learned correspondence-aware feature descriptors and semantic features are then merged to form the final feature descriptors, for which the improved feature matching ability has been observed in experiments. In addition, the learned semantics embedded in the features can be further used to filter out noisy keypoints, leading to additional accuracy improvement and faster matching speed. Experiments on two popular long term visual localization benchmarks (Aachen Day and Night v1.1, Robotcar Seasons) and one challenging indoor benchmark (InLoc) demonstrate encouraging improvements of the localization accuracy over its counterpart and other competitive methods. [ABSTRACT FROM AUTHOR]

In this paper, a Multi-scale Contrastive Graph Convolutional Network (MC-GCN) method is proposed for unconstrained face recognition with image sets, which takes a set of media (orderless images and videos) as a face subject instead of single media (an image or video). Due to factors such as illumination, posture, media source, etc., there are huge intra-set variances in a face set, and the importance of different face prototypes varies considerably. How to model the attention mechanism according to the relationship between prototypes or images in a set is the main content of this paper. In this work, we formulate a framework based on graph convolutional network (GCN), which considers face prototypes as nodes to build relations. Specifically, we first present a multi-scale graph module to learn the relationship between prototypes at multiple scales. Moreover, a Contrastive Graph Convolutional (CGC) block is introduced to build attention control model, which focuses on those frames with similar prototypes (contrastive information) between pair of sets instead of simply evaluating the frame quality. The experiments on IJB-A, YouTube Face, and an animal face dataset clearly demonstrate that our proposed MC-GCN outperforms the state-of-the-art methods significantly. [ABSTRACT FROM AUTHOR]

Identifying the same persons across different views plays an important role in many vision applications. In this paper, we study this important problem, denoted as Multi-view Multi-Human Association (MvMHA), on multi-view images that are taken by different cameras at the same time. Different from previous works on human association across two views, this paper is focused on more general and challenging scenarios of more than two views, and none of these views are fixed or priorly known. In addition, each involved person may be present in all the views or only a subset of views, which are also not priorly known. We develop a new end-to-end deep-network based framework to address this problem. First, we use an appearance-based deep network to extract the feature of each detected subject on each image. We then compute pairwise-similarity scores between all the detected subjects and construct a comprehensive affinity matrix. Finally, we propose a Deep Assignment Network (DAN) to transform the affinity matrix into an assignment matrix, which provides a binary assignment result for MvMHA. We build both a synthetic dataset and a real image dataset to verify the effectiveness of the proposed method. We also test the trained network on other three public datasets, resulting in very good cross-domain performance. [ABSTRACT FROM AUTHOR]

The paper proposes a solution based on Generative Adversarial Network (GAN) for solving jigsaw puzzles. The problem assumes that an image is divided into equal square pieces, and asks to recover the image according to information provided by the pieces. Conventional jigsaw puzzle solvers often determine the relationships based on the boundaries of pieces, which ignore the important semantic information. In this paper, we propose JigsawGAN, a GAN-based auxiliary learning method for solving jigsaw puzzles with unpaired images (with no prior knowledge of the initial images). We design a multi-task pipeline that includes, (1) a classification branch to classify jigsaw permutations, and (2) a GAN branch to recover features to images in correct orders. The classification branch is constrained by the pseudo-labels generated according to the shuffled pieces. The GAN branch concentrates on the image semantic information, where the generator produces the natural images to fool the discriminator, while the discriminator distinguishes whether a given image belongs to the synthesized or the real target domain. These two branches are connected by a flow-based warp module that is applied to warp features to correct the order according to the classification results. The proposed method can solve jigsaw puzzles more efficiently by utilizing both semantic information and boundary information simultaneously. Qualitative and quantitative comparisons against several representative jigsaw puzzle solvers demonstrate the superiority of our method. [ABSTRACT FROM AUTHOR]

A smart grid integrates advanced sensors, efficient measurement methods, progressive control technologies, and other techniques and devices to achieve safe, efficient and economical operation of the grid system. However, the diversified and open environment of a smart grid makes energy and information of the smart grid vulnerable to malicious attacks. As a representative cyber-physical attack, the data integrity attack has an extremely severe impact on the grid operation for it can bypass the traditional detection mechanisms by adjusting the attack vector. In this paper, we first present the attack strategy against dynamic state estimation of power grid in the perspective of adversary and formulate the data integrity attack detection problem that has the characteristic of sequential decision making as a partially observable Markov decision process. Then, a deep reinforcement learning-based approach is proposed to detect against data integrity attacks, which utilizes the Long Short-Term Memory layer to extract the state features of previous time steps in determining whether the system is currently under attack. Moreover, the noisy networks are employed to ensure effective agent exploration, which prevents the agent from sticking to the non-optimal policy. The principle of a multi-step learning is adopted to increase the estimation accuracy of Q value. To address the sparse rewards problem, the prioritized experience replay is proposed to increase training efficiency. Simulation results demonstrated that the proposed detection approach surpasses the benchmarks in the comparison metrics: delay error rate and false rate. Note to Practitioners—In this paper, we present a deep reinforcement learning-based algorithm to defend against the data integrity attacks of smart grid. Most of the previous works discretized the system states and utilized the current state information to identify whether the system is under attack. For this reason, the detection policy may totally ignored the continuously changing characteristics of the grid states, which will lead to poor detection performance. Moreover, the attacked system states only accounts for a small part of the entire grid operation states, the probability of sampling the experience containing the attack state is extremely small, which limits the learning efficiency of previous RL-based detection approaches. In order to increase the accuracy of detection, we first present the attack strategy against power grid’s dynamic state estimation in the perspective of adversary and formulate the partially observable Markov decision process model of attack detection problem. Moreover, we propose a deep reinforcement learning-based detection approach combining the LSTM network to extract the system state features of the previous time steps to determine whether the system is currently being attacked. To address the sparse rewards problem, the prioritized experience replay is used to increase learning efficiency. The experiments demonstrate the effectiveness of proposed detection scheme compared with benchmarks in terms of detection delay as well as accuracy. In conclusion, the proposed detection scheme is helpful in defending against the data integrity attacks without obtaining the opponent’s strategy in advance and can be conveniently applied to the real-world security management system of smart grid. [ABSTRACT FROM AUTHOR]

To monitor process and identify the deviation as early as possible, data-driven methods have been applied for process monitoring and fault detection in semiconductor manufacturing. Although various fault detection and classification models had been discussed in the literature, however, little research has been devoted to feature selection from trace data that is important for process monitoring of natural variation. Additionally, the high-mix production mode with different recipes leads to process dynamic of wafer-to-wafer (W2W) variation which should also be identified for safeguarding false alarms and serving as a warning indicator. Therefore, this paper proposes a data-driven framework to identify the key features with respect to the W2W variation. In particular, the self-organizing map is used to annotate the grade of wafer variation among the in-line metrology data. Subsequently, the adaptive boosting (AdaBoost) is adopted to examine the effectiveness of every feature and its processing times, respectively. To validate the proposed framework, an empirical study from a semiconductor fabrication plant is conducted. The experimental results demonstrate that the key feature identification is of critical importance to build highly capable models for process monitoring. Through the dimensionality reduction technique, it has been illustrated that a smaller set of the identified key features are able to pinpoint the W2W variation of different wafer grades more clearly than the whole set of process features. Note to Practitioners— Process monitoring has become more difficult with the shrinking linewidth in semiconductor manufacturing. The challenges of analyzing equipment sensor or raw trace data for process monitoring in high-mix manufacturing processes are to incorporate subject-matter expert knowledge for setting control limit meticulously, to detect the subtle changes by analyzing the whole trace data profile, and to identify W2W variation for reducing false alarms. This paper proposes a data-driven framework for process monitoring by adopting data-driven approaches without recourse to domain judgement. Experimental results demonstrate that the proposed data-driven framework can effectively identify the key features via sensor readings and corresponding processing times, respectively. The engineers can make use of the extracted features to perform a predictive monitoring on metrology data for detection of potential process deterioration. [ABSTRACT FROM AUTHOR]

Surface defect is a serious problem in real-world manufacturing system and it is important to use vision-based recognition to ensure the surface quality of products. Currently, due to the ability of automatic feature extraction, deep learning models, such as convolutional neural network (CNN), have been widely used in this area. However, these CNN-based models may not solve a problem well - inter-class similarities and intra-class variations (ISIV), which affect their ability of feature extraction and thus influence their recognition performance. To address this problem, this paper introduces a graph guidance mechanism into CNN to improve the ability of feature extraction, called Graph guided Convolutional Neural Network (GCNN). Firstly, GCNN defines a graph by computing the similarities between training samples. Secondly, the graph is introduced into VGG11, a popular CNN structure, to increase the inter-class distances and decrease the intra-class distances between defect samples. Meanwhile, a learnable coefficient is introduced into the training process to balance the effect of graph guidance automatically. The experimental results on four famous surface defect datasets demonstrate that the graph guidance helps CNN models have better ability of feature extraction and thus achieve better performance. Compared with state-of-the-art models, the proposed method can achieve the best performance. Furthermore, the final discussion shows that the learnable coefficient can help the proposed model to gain better performance, and that the proposed model increases a little computation cost compared to its original CNN model. Note to Practitioners—This paper is motivated by the problem in real-world manufacturing process – inter-class similarities and intra-class variations. Most of current CNN-based models may not solve this problem well, which limits their applications in real-world system. This paper proposes a graph guidance mechanism and introduces it to the training of CNN. The proposal can improve the ability of feature extraction, and thus have better performance than those without the graph guidance. By applying the proposal to four examples, the results show that the proposal is more feasible and effective than the state-of-the-art models in surface defect recognition. Furthermore, the mechanism assists the training of CNN and is removed in the process of recognition, so it will not increase time and occupied memory in recognition, and meanwhile, only limited computation cost is increased in training process. [ABSTRACT FROM AUTHOR]

In manufacturing industries, it is of fundamental importance to detect anomalies in production in order to meet the required quality goals and to limit the number of defective products that are accidentally delivered to the customers. Nevertheless, monitoring systems currently employed in production are typically very simple and rely on a set of univariate control charts that fail to capture the multivariate and complex nature of real-world industrial systems. In such context, Machine Learning (ML)-based approaches for Anomaly Detection (AD) have proven to be extremely effective in increasing anomalies detectability and, in general, in enhancing monitoring procedures. However, industrial data are typically very complex and not suitable to be fed directly to classical ML-based AD tools making feature extraction procedures a necessary step that unfortunately may lead to information loss and low scalability. Deep Learning, has proven very effective at learning useful representations of complex data in an automatic way. In this paper, we propose an AD pipeline that makes use of convolutional autoencoders to extract useful features from two-dimensional, non-image, data. We test our approach on real world Optical Emission Spectroscopy data that are typical of semiconductor manufacturing and we achieve improved performance over classical monitoring methods. Note to Practitioners—Advanced monitoring is one of the most important task in the context of Industry 4.0. Some of the main issues in developing Machine Learning-based solutions in industrial environment are: (i) the lack of reliable tagged data; (ii) the complexity of data structures present in real-world scenarios. In this paper we investigate unsupervised anomaly detection for 2-dimensional data in manufacturing environment: we provide an approach that exploit Deep Learning-based architecture for handling the data at hand. We show the effectiveness of the proposed approach in a real world case study related to optical emission spectroscopy data in semiconductor manufacturing process providing satisfactory classification accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we focus on co-selection of instances and features in the semi-supervised learning scenario. In this context, co-selection becomes a more challenging problem as data contain labeled and unlabeled examples sampled from the same population. To carry out such semi-supervised co-selection, we propose a unified framework, called sCOs, which efficiently integrates labeled and unlabeled parts into the co-selection process. The framework is based on introducing both a sparse regularization term and a similarity preserving approach. It evaluates the usefulness of features and instances in order to select the most relevant ones, simultaneously. We propose two efficient algorithms that work for both convex and nonconvex functions. To the best of our knowledge, this paper offers, for the first time ever, a study utilizing nonconvex penalties for the co-selection of semi-supervised learning tasks. Experimental results on some known benchmark datasets are provided for validating sCOs and comparing it with some representative methods in the state-of-the art. [ABSTRACT FROM AUTHOR]

In the controller development process, it is required to verify whether control functions operate normally, without fault or anomaly. Many control functions with complex system structures require various actual tests and test data analysis for verification. This paper presents an anomaly detection algorithm to verify control functions and to more efficiently analyze test data of a hybrid control unit (HCU) of a hybrid electric vehicle (HEV). The anomaly detection algorithm automatically detects anomaly of control functions from the test data, instead of manual labor by engineers. The examined target HCU control functions in this paper are the engine clutch engagement/disengagement control function and the engine start cooperative control function, which are crucial functions of transmission mounted electric drive (TMED) HEVs. A data-driven approach using a one-class support vector machine (SVM) is used to make it easy to apply various control functions. Actual vehicle test data is examined by the algorithm to verify the control functions at an actual vehicle level. The developed anomaly detection algorithm demonstrates feasibility and effectiveness of the proposed algorithm in detecting not only prior-known anomalies but also prior-unknown anomalies. Because other vehicle control functions have similar characteristics with the target functions of this paper, this algorithm is expected to be successfully applied to other HCU control functions. [ABSTRACT FROM AUTHOR]

Since gait can be utilized to identify individuals from a far distance without their interaction and coordination, recently many gait recognition methods have been proposed. However, due to a real-world scenario of clothing changes, a degradation occurs for most of these methods. Thus in this paper, a more efficient gait recognition method is proposed to address the problem of clothing variances. First, part-based gait features are formulated from two different perspectives, i.e., the separated body parts that are more robust to clothing changes and the estimated human skeleton key-point regions. It is reasonable to formulate such features for cloth-changing gait recognition, because these two perspectives are both less vulnerable to clothing changes. Given that each feature has its own advantages and disadvantages, a more efficient gait feature is generated in this paper by assembling these two features together. Moreover, since local features are more discriminative than global features, in this paper more attention is focused on the local short-range features. Also, unlike most methods, in our method we treat the estimated key-point features as a set of word embeddings, and a transformer encoder is specifically used to learn the dependence of each correlative key-points. The robustness and effectiveness of our proposed method are certified by experiments on CASIA Gait Dataset B, and it has achieved the state-of-the-art performance on this dataset. [ABSTRACT FROM AUTHOR]

Spectre attacks can be catastrophic and widespread because they exploit common design flaws caused by the speculative capabilities in modern processors to leak sensitive data through side channels. Completely fixing the problem would require a redesign of the architecture for transient execution or the implementation of a new design on re-configurable hardware. However, such fixes cannot be backported to old machines with fixed hardware design. Completely replacing those machines will take a long time. Moreover, existing software patches may cause significant performance overhead. This paper proposes to detect Spectre by monitoring deviations in microarchitectural events using hardware performance counters with promising accuracy above 90 percent under a variety of workload conditions. However, the attacker may attempt to evade detection by slowing down the attack or mimicking benign programs. This paper thus compares different evasion strategies quantitatively and demonstrates that it is possible for the attacker to avoid detection when operating the attacks at a lower speed while maintaining a reasonable attack success rate. Then, we show that, in order to resist evasion, the original detector must be enhanced by randomly switching between a set of detectors using different features and sampling periods so we can keep the detection accuracy above 80 percent. [ABSTRACT FROM AUTHOR]

Spatial failure patterns in wafer defect maps are very useful for root cause analysis, which is essential for yield optimization in the manufacturing process. Especially, the scratch defect type is the most challenging pattern to recognize because the position, shape, size and curvature vary widely from one scratch to another. Discontinuity points within scratches also contribute to the low recognition rate, and such points are often hidden defective dies that become reliability threat. Previous studies on defect pattern identification show that the recognition rate for scratches is among the lowest in all patterns even if the overall accuracy is high. In this paper, we propose a novel scratch pattern recognition method. In this paper, image processing techniques are applied to recognize scratch patterns in wafer maps. The Hough transform is first employed to identify line segments, from which scratches can be reconstructed. In contrast to previous machine-learning based methods, there is no need to train a complicated prediction model so that the computation time is small. The method is validated by using wafer maps from six real products and public data. Experimental results show that the proposed method achieves high recall, precision and accuracy in all cases. [ABSTRACT FROM AUTHOR]

At present, many studies have shown that partitioning the gait sequence and its feature map can improve the accuracy of gait recognition. However, most models just cut the feature map at a fixed single scale, which loses the dependence between various parts. So, our paper proposes a structure called Part Feature Relationship Extractor (PFRE) to discover all of the relationships between each parts for gait recognition. The paper uses PFRE and a Convolutional Neural Network (CNN) to form the RPNet. PFRE is divided into two parts. One part that we call the Total-Partial Feature Extractor (TPFE) is used to extract the features of different scale blocks, and the other part, called the Adjacent Feature Relation Extractor (AFRE), is used to find the relationships between each block. At the same time, the paper adjusts the number of input frames during training to perform quantitative experiments and finds the rule between the number of input frames and the performance of the model. Our model is tested on three public gait datasets, CASIA-B, OU-LP and OU-MVLP. It exhibits a significant level of robustness to occlusion situations, and achieves accuracies of 92.82% and 80.26% on CASIA-B under BG # and CL # conditions, respectively. The results show that our method reaches the top level among state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The content based image retrieval aims to find the similar images from a large scale dataset against a query image. Generally, the similarity between the representative features of the query image and dataset images is used to rank the images for retrieval. In early days, various hand designed feature descriptors have been investigated based on the visual cues such as color, texture, shape, etc. that represent the images. However, the deep learning has emerged as a dominating alternative of hand-designed feature engineering from a decade. It learns the features automatically from the data. This paper presents a comprehensive survey of deep learning based developments in the past decade for content based image retrieval. The categorization of existing state-of-the-art methods from different perspectives is also performed for greater understanding of the progress. The taxonomy used in this survey covers different supervision, different networks, different descriptor type and different retrieval type. A performance analysis is also performed using the state-of-the-art methods. The insights are also presented for the benefit of the researchers to observe the progress and to make the best choices. The survey presented in this paper will help in further research progress in image retrieval using deep learning. [ABSTRACT FROM AUTHOR]

Smoke detection in video is a challenging task because of the irregular shape of smoke, its complex motion state, which is affected by temperature, wind and other external factors, and background disturbances. Pixel-based foreground modeling method is a crucial step in many smoke detection systems and can be applied to efficiently focus on a certain object or a specific region to detect movements or anomalies. In video analysis, it is a natural idea to move the focus from the pixel-level foreground to the feature-level foreground. In this paper, the feature foreground is generated by the middle layer of a convolutional neural network (CNN) to guide the temporal modeling process for smoke objects. A novel temporal module called the Feature Foreground Module (FFM) is proposed to boost learning of a smoke temporal representation. Consider the problem of smoke analysis in video, we present a novel unifying approach, named an enhanced feature foreground network (EFFNet), that performs both smoke source prediction and detection. Efficient branch networks are designed in EFFNet, to predict the source mask and bounding boxes of smoke plumes in video. To the best of our knowledge, this is the first paper to study the source of smoke using deep learning methods. Finally, experiments on a realistic smoke dataset and a public dataset show that EFFNet method performs much better than do previous state-of-the-art methods. [ABSTRACT FROM AUTHOR]