Search

The development of social media enables fake news to be expressed in a multi-modal form, which is disseminated on various social platforms and brings harmful social impacts. To handle this challenge, the fake news detection task was proposed to examine whether false information is contained in multi-modal news. Existing methods exploit various approaches with cross-modal interaction and fusion, which have proven to be effective in detecting common fake news. However, although the description of multi-modal news is narrated around entities, the previously developed methods pay less attention to this characteristic. They do not explore its benefits to the detection task and underperform with respect to the detection of fake news that requires entity-centric comparisons. To make up for this omission, we explore a novel paradigm to detect fake news by aligning and fusing multi-modal entities and propose the Entity-oriented Multi-modal Alignment and Fusion network (EMAF). Our work adopts entity-centric cross-modal interaction, which can reserve semantic integrity and capture the details of multi-modal entities. Specifically, we design an Alignment module with the improved dynamic routing algorithm and introduce a Fusion module based on the comparison, the former aligns and captures the important entities and the latter compares and aggregates entity-centric features. Comparative experiments conducted on multiple public datasets, including Weibo, Twitter, and Reddit, reveal the superiority of the proposed EMAF method, and extensive analytical experiments demonstrate the effectiveness of our proposed modules.

Facing the challenge of small open labeled data sets in remote sensing change detection, this letter proposes a novel supervised change detection method by taking advantages of capsule network which can reach the same performance as traditional convolutional neural networks (CNNs) but with less training data. To achieve this aim, we propose a pseudo-Siamese capsule network which takes both rotational invariance and spatial hierarchies between features into account for aerial images change detection. First, the features of image pairs are extracted by two identical nonshared weights convolutional capsule networks. Second, the extracted features are directly concatenated and sent to another convolutional capsule layer. The change probability map is obtained by calculating the length of the capsule vectors in the final layer. Additionally, to reduce the influence of imbalance samples when we optimize our network, we design a margin-focal loss function to pay more attention to the misclassified samples. Finally, binary change map can be produced by a simple threshold. Experimental results carried out on the SZTAKI AirChange Benchmark Set show that the proposed method achieves comparable and even better results with existing state-of-the-art methods in terms of F-measure.

Due to the differences in the feature distribution between classes, when the model learns in a continuous data stream, it will encounter catastrophic forgetting. The incremental learning methods have shown great potential to solve this problem. However, most existing methods based on task-incremental learning are difficult to adapt to characteristics of remote sensing scenes with few differences in appearance but large differences in features, which is not conducive to artificially distinguish task-ID. Thus, we propose a class-incremental learning network for small objects enhancing of semantic segmentation in aerial imagery. Specifically, considering the superior accurate of the binary classifier, we propose a Twin-Auxiliary model that adds an auxiliary binary classification task. Then, for expansion and contraction at the edge and small object confusion problems, we introduce a diversity distillation loss, using the results of binary-classifier to constrain the multi-class segmentation results, and strengthen the attention to the locations of the segmentation results that have changed. Finally, we design a conflict reduction mechanism for multi-head classifier to achieve single-head prediction for class-incremental learning. Experiments demonstrate that our method has good performance on the Vaihingen and Potsdam data sets by ISPRS, outperforming state-of-the-art incremental learning methods. The code will be available soon.

Multi-modal abstractive summarization for videos is an emerging task, aiming to integrate multi-modal and multi-source inputs (video, audio transcript) into a compressed textual summary. Although recent multi-encoder-decoder models on this task have shown promising performance, they did not explicitly model interactions of multi-source inputs. While some strategies like co-attention are utilized for modeling this interaction, considering ultra-long sequences and additional decoder in this task, the coupling of multi-modal data from multi-encoders and decoder needs complicated structure and additional parameters. In this paper, we propose a concise Decoder-only Multi-modal Transformer (D-MmT) based on the above observations. Specifically, we cut the encoder structure, and introduce an in-out shared multi-modal decoder to make the multi-source and target fully interact and couple in the shared feature space, reducing the model parameter redundancy. Also, we design a concise cascaded cross-modal interaction (CXMI) module in the multi-modal decoder that generates joint fusion representations and spontaneously establishes a fine-grained intra- and inter- association between multi-modalities. In addition, to make full use of the ultra-long sequence information, we introduce a joint in-out loss to make the input transcript also participate in backpropagation to enhance the contextual feature representation. The experimental results on the How2 dataset show that the proposed model outperforms the current state-of-the-art approach with fewer model parameters. Further analysis and visualization show the effectiveness of our proposed framework.

Existing methods that use document-level information for event detection ignore the dependencies between sentences and also have shortcomings in modeling the dependencies among words. In this paper, we propose a novel Hierarchical Graph Enhanced Event Detection (HGEED) framework to make full use of syntax and document information for the task of event detection. First, a sentence graph is used to model word-to-word dependencies, enriching the local information of words by incorporating syntactic features. Then, a document graph is built to model sentence-to-sentence dependencies, obtaining global semantic representations for word-level prediction. The experiment results on the widely used ACE 2005 and TAC KBP 2015 corpora show that our model can capture local and global information with dependencies and achieve significant improvements as compared to all baselines.

End-to-end aspect-based sentiment analysis (E2E-ABSA) is a sequence labeling task which detects aspect terms and the corresponding sentiment simultaneously. Previous works ignore the useful task-specific knowledge and embed the vital aspect and sentiment attributes implicitly in the intermediate layers. In this paper, we propose a hierarchical multi-task learning framework, which explicitly leverages task-related knowledge via the supervision of intermediate layers. Specifically, aspect term extraction, sentiment lexicon detection, and aspect sentiment detection are designed to encode the aspect boundary and sentiment information. The tasks are in charge of different perspectives and levels of knowledge, which provide multi-fold regulation effects to optimize the main task. Unlike vanilla multi-task learning, all the tasks are integrated into a hierarchical structure to help the higher-level tasks make full use of the lower-level tasks’ information. Experimental results on three datasets demonstrate that the proposed method achieves state-of-the-art results. Further analysis shows that the proposed method achieves better performance than single-task and vanilla multi-task learning methods and yields a more discriminative feature representation.

Aspect based sentiment analysis (ABSA) can be divided into aspect category sentiment analysis (ACSA) and aspect term sentiment analysis (ATSA). To detect the sentiment polarity in a context given an aspect, both ACSA and ATSA highly rely on aspect-centroid sentiment information. However, aspect terms in ATSA appear in the context, but aspect categories in ACSA may not. This make it difficult to use important aspect position information and further design effective aspect injection strategy when modeling ACSA and ATSA in a unified framework. To address this problem, we propose a novel Unified Position-aware Convolutional Neural Network (UP-CNN). There are two major modifications in UP-CNN. Firstly, to handle the absence of aspect position in ACSA, we propose an aspect detection network with prior knowledge. Thus ABSA can be solved in a unified view with the important aspect position. Secondly, to fit CNN in ABSA, an aspect mask is proposed to construct aspect-aware context representation. Experimental results on seven datasets demonstrate that our model performs effectively and efficiently on both ACSA and ATSA tasks.

Claim verification aims to verify the truthfulness of claims according to the evidence. In the real-world scenarios, the claim verification methods are required 1) to identify and filter out the noise; 2) to enable the interaction among evidence; and 3) to predict the labels in mixed label sets. However, the prior studies adopt either the implicit or cumbersome interaction processes, which disable them from capturing the commonalities among evidence nor reducing the impact of noise, and few of them consider the label prediction in mixed label sets. To alleviate these issues, we propose an Enhanced Dynamic Interaction Network (EDIN). The EDIN contains two main modules: A Gate-based Dynamic Interaction Module to promote the interaction of evidence and summarize the commonality to verify the claim, which employs hub node and gates to control the information transmission and denoise. An Enhancement Module to assist the EDIN with predicting labels in a mixed label sets environment, where we introduce the auxiliary domain detection task and label transition component to enhance the expressiveness of label representation. Experimental results on multiple public datasets reveal the superiority of our method, and further analysis validates the effectiveness of our proposed modules.

Target-oriented Opinion Words Extraction (TOWE) aims to identify opinion words toward a specific target given the sentence. Syntax structure, which contains dependency relationships among words, is a vital clue for this task. With the help of syntax structure as a constraint, the model could remove irrelevant words and focus on tokens that are relevant to the given target. Directly adapting existing syntactic-based methods faces the problem that these models do not explicitly learn target-centric representations. Another challenge is that prior works only learn fixed order dependency relations, while context words require syntactic information in different scales. To handle these issues, we propose Target-Specific Graph Convolutional Network (TS-GCN) to explicitly integrate dependency structure. The proposed method could build high-quality syntax-aware representations by propagating target information to syntactically related words via graph convolution. Furthermore, we design a memory-based module to dynamically learn multi-granularity syntactic knowledge and infuse local features. Experimental results demonstrate the effectiveness of our method, and we achieve state-of-the-art performances on four SemEval datasets.

Natural Language Inference (NLI) aims to determine the relationship of a pair of sentences. As a critical component of NLI models, attention mechanism has been proven to be effective in the representation and interaction of sentences. However, the attention methods adopted in the existing NLI models are non-parametric or trained inside the model without explicit supervision, and the attention results are poorly explained. In this paper, we propose a Supervised Attention based Natural Language Inference (SA-NLI) framework to solve this problem. In our framework, the intra attention module is trained to focus on syntactically related tokens, while the inter attention module is constrained to capture alignment between sentences. Moreover, the supervised training of intra attention module and inter attention module are unified with the training of the NLI model by multi-task learning and transfer learning, respectively. We conduct extensive experiments on multiple NLI datasets, and the results demonstrate the effectiveness of our supervised attention based method. Further visual analysis validates the interpretability of attention results, and the extended experimental results indicate the generalization of our SA-NLI framework.

Convolutional neural networks have recently demonstrated high-quality reconstruction for single image super-resolution (SR). These CNN networks effectively recover a high-resolution (HR) image from a low-resolution (LR) image, at the cost of enormous parameters and heavy computational burden. In this work, we propose a recursive efficient deep convolutional network for fast and accurate single-image SR with only 0.28M parameters. A Split-Concatenate-Residual (SCR) block is proposed to reduce computation and parameters. With downsampling block and upsampling block, we significantly reduce computational complexity and enlarge the size of the receptive field. Specifically, two-level recursive learning is proposed which can improve accuracy by increasing depth without adding any weight parameters. We also employ local, semi-global and global residual techniques to train our very deep network steadily and improve its performance. Extensive experiments indicate that our proposed method Split-Concatenate-Residual Super Resolution (SCRSR) yields promising SR performance while maintaining shorter running time and fewer parameters.

Object detection plays an important role in the field of remote sensing imagery analysis. The most challenging issues in advancing this task are the large variation in object scales and the arbitrary orientation of objects. In this paper, we build a unified framework upon the region-based convolutional neural network for arbitrary-oriented and multi-scale object detection in remote sensing images. To handle the problem of multi-scale object detection, a feature-fusion architecture is proposed to generate a multi-scale feature hierarchy, which augments the features of shallow layers with semantic representations via a top-down pathway and combines the feature maps of top layers with low-level information by a bottom-up pathway. By combining features of different levels, we can form a powerful feature representation for multi-scale objects. Most previous methods locate objects with arbitrary orientations and dense spatial distributions via axis-aligned boxes, which may cover adjacent instances and background areas. We build a rotation-aware object detector that uses oriented boxes to localize objects in remote sensing images. The region proposal network augments the anchors with multiple default angles to cover oriented objects. It utilizes oriented proposal boxes to enclose objects rather than horizontal proposals that coarsely locate oriented objects. The orientation RoI pooling operation is introduced to extract the feature maps of oriented proposals for the following R-CNN subnetwork. We conduct comprehensive experiments on a public dataset for oriented object detection in remote sensing images. Our method achieves state-of-the-art performance, which demonstrates the effectiveness of the proposed methods.

Aspect-based sentiment analysis, which aims to predict the sentiment polarities for the given aspects or targets, is a broad-spectrum and challenging research area. Recently, pre-trained models, such as BERT, have been used in aspect-based sentiment analysis. This fine-grained task needs auxiliary information to distinguish each aspect. But the input form of BERT is only a words sequence which can not provide extra contextual information. To address this problem, we introduce a new method named GBCN which uses a gating mechanism with context-aware aspect embeddings to enhance and control the BERT representation for aspect-based sentiment analysis. Firstly, the input texts are fed into BERT and context-aware embedding layer to generate BERT representation and refined context-aware embeddings separately. These refined embeddings contain the most correlated information selected in the context. Then, we employ a gating mechanism to control the propagation of sentiment features from BERT output with context-aware embeddings. The experiments of our model obtain new state-of-the-art results on the SentiHood and SemEval-2014 datasets, achieving a test F1 of 88.0 and 92.9 respectively.

Considering the lack of labeled training data sets for the supervised change detection task, in this letter, we try to relieve this problem by proposing a convolutional neural network (CNN)-based change detection method with a newly designed loss function to achieve transfer learning among different data sets. To reach this goal, we first pretrain a U-Net model on an open source data set by taking advantages of the relatively sufficient training data used for the supervised semantic segmentation task. Then, we minimize a skillfully designed loss function to combine the high-level features extracted from the pretrained model and the semantic information contained in the change detection data set, by which a transfer learning is achieved. Third, we compute the distance between the feature vectors obtained from the above step and produce a difference map. Finally, a simple clustering method used on the difference map can even obtain satisfied change map. Experiments carried out on typical optical aerial image data sets validate that the proposed approach compares favorably to the state-of-the-art unsupervised methods.

Network embedding (NE) aims to represent network information appropriately by learning low-dimensional and dense vectors for the nodes and edges of information network. Actually, the real world is almost full of heterogeneous information networks, which stimulates the emergence of heterogeneous information networks (HINs) embedding models. However, parts of existing HIN embedding models like meta-path-based methods only capture limited and aggregated information of relations, whereas some models based on metric or distance learning are usually of high computational complexity and slow training speed. In this paper, we present a novel heterogeneous information network embedding model, which applies dynamic projection metrics and translation mechanisms to the complicated heterogeneous information networks including multiple nodes and different relations. In order to overcome the imbalance of the distribution of relations in HIN and optimize the training process, we introduce an adaptive loss function for model optimization. Further more, we propose a hybrid model with baseline method as the initialization of the model. Experiments have been implemented on some real-world HIN datasets. And empirical results show that our model significantly outperforms the state-of-the-art representation learning models.

Event argument extraction, which aims to identify arguments of specific events and label their roles, is a challenging subtask of event extraction. Previous approaches solve this problem in a two-stage manner that first extracts named entities as argument candidates and then determines their roles. However, many nested entities may be missed or wrongly predicted during the argument candidate extraction procedure, which substantially affects the performance of the downstream classifier. In this paper, we propose a novel one-step question answering based framework, which performs argument candidate extraction and argument role classification simultaneously to mitigate the error propagation problem in conventional two-stage methods. Since the conventional question answering based framework cannot be applied directly to this task, we design a Q uestion A nswering based S equence L abeling (QA-SL) model to tackle inexistent argument roles and multiple argument token spans. Moreover, considering the overwhelming number of parameters in question answering based neural network models and the relatively small size of event extraction corpus, we fine-tune the pre-trained model from BERT to mitigate the data scarcity problem. Extensive experiments demonstrate the benefits of the proposed method, leading to a competitive performance compared with state-of-the-art methods. To the best of our knowledge, this is the first work to cast event argument extraction as a question answering task.

Knowledge Base Question Answering (KBQA) is a promising approach for users to access substantial knowledge and has become a research focus in recent years. Our paper focuses on relation detection, a subtask of KBQA and proposes an adversarial training improved multi-path multi-scale relation detector (AdvT-MMRD) to improve the performance of a common KBQA system. To solve the problem of matching the casual form of a question with the logical form of a predicate, we use question pattern-relation matching, in which an attention-based bidirectional recurrent neural network with gated recurrent units (Bi-GRUs) is used to match semantic similarity and a convolutional neural network (CNN) is used to learn literal similarity between question and relation. We also explore two ways to measure the relevance of entity type-relation pairs through several level representations. Additionally, an adversarial training strategy is conducted to enhance our model. The experimental results demonstrate that our approach not only achieves a state-of-the-art accuracy of 93.8% on relation detection task, but contributes our KBQA system to reaching an outstanding accuracy of 79.0% on the SimpleQuestions benchmark.

Simplex volume is the most commonly used parameter for nonnegative matrix factorization (NMF)-based endmember estimation methods, and one of the most popular methods is the NMF method with minimum volume constraint (MVC-NMF). However, when outliers exist in the image, MVC-NMF tends to extract them as endmembers. In most cases, those outlier endmembers could be either physically meaningless or not representative enough for prevalent land covers. So how to extract prevalent land covers instead of outliers as endmembers is a very challenging question. In this letter, we propose a new NMF method with the dual constraints of simplex volume and information content, named the “minimum volume and information constraint NMF” (MIVC-NMF). The method is based on the following facts: when a real endmember is replaced by an outlier, it will cause some pixels containing the replaced endmember not to locate within the endmember hyperplane, and the overall information content contained in the endmember hyperplane will be reduced. The experimental results based on the simulated and real data show that the proposed method outperforms several other commonly used endmember extraction approaches.

Event detection is an essential and challenging task in Information Extraction (IE). Recent advances in neural networks make it possible to build reliable models without complicated feature engineering. However, data scarcity hinders their further performance. Moreover, training data has been underused since majority of labels in datasets are not event triggers and contribute very little to the training process. In this paper, we propose a novel multi-task learning framework to extract more general patterns from raw data and make better use of the training data. Specifically, we present two paradigms to incorporate neural language model into event detection model on both word and character levels: (1) we use the features extracted by language model as an additional input to event detection model. (2) We use a hard parameter sharing approach between language model and event detection model. The extensive experiments demonstrate the benefits of the proposed multi-task learning framework for event detection. Compared to the previous methods, our method does not rely on any additional supervision but still beats the majority of them and achieves a competitive performance on the ACE 2005 benchmark.

This letter presents a novel supervised change detection method based on a deep siamese semantic network framework, which is trained by using improved triplet loss function for optical aerial images. The proposed framework can not only extract features directly from image pairs which include multiscale information and are more abstract as well as robust, but also enhance the interclass separability and the intraclass inseparability by learning semantic relation. The feature vectors of the pixels pair with the same label are closer, and at the same time, the feature vectors of the pixels with different labels are farther from each other. Moreover, we use the distance of the feature map to detect the changes on the difference map between the image pair. Binarized change map can be obtained by a simple threshold. Experiments on optical aerial image data set validate that the proposed approach produces comparable, even better results, favorably to the state-of-the-art methods in terms of F-measure.

Graph layout investigates the structure of the graph in order to better obtain the information implied in the graph. To solve the shortcomings of dimension reduction layouts on local adjustment and the insufficiency of energy models to maintain the overall structure of the graphs, this paper proposes a new graph layout framework called “tNEM” that layouts graphs by combining t-distributed neighbor retrieval visualizer (t-NeRV) and energy models. In the process of layout, our algorithm considers global and local structures at the same time. The layout results are more conform to aesthetic standards, meanwhile, maintain the structural information of the graph. We evaluate our algorithm on a wide variety of datasets and compare it with many other methods. We produce better visualization results than tsNET and tsNET* methods by reducing the tendency to crowd points together, and can better capture the global structure of the graph.

Named entity recognition (NER) is one of the basic techniques in natural language processing tasks. Chinese NER is complicated and difficult which remains a major challenge. One of the main reasons is that the boundaries of entities in Chinese are blurred and closely related to word segmentation results. Previous studies for this task have broadly divided into two categories, word-based, and character-based methods. However, the former class suffers from the word segmentation errors, and the latter cannot make full use of the information on multiple granularities. To address these problems, we investigate a new dynamic meta-embeddings method and apply it to Chinese NER task, which utilizes attention mechanism to combine features of both character and word granularity in embedding layer. The meta-embeddings created by our method are dynamic, data-specific, and task-specific, since the meta-embeddings for same characters in different sentence sequences are distinct. The experiments on MSRA and LiteratureNER datasets validate the effectiveness of our model, and this method achieves the state-of-the-art results on LiteratureNER.

Chinese Named Entity Recognition (Chinese NER) is an important task in Chinese natural language processing field. It is difficult to identify the boundary of entities because Chinese texts lack natural delimiters to separate words. For this task, two major methods can be distinguished by the model inputs, i.e., word-based model and character-based model. However, the word-based model relies on the result of the Chinese Word Segmentation (CWS), and the character-based model cannot utilize enough word-level information. In this paper, we propose a multi-granularity information fusion model (MIFM) for the Chinese NER task. We introduce a novel multi-granularity embedding layer that utilizes the attention mechanism and an information gate to fuse the character and word level features. The results of this embedding method are dynamic and data-specific because they are calculated based on different contexts. Moreover, we apply the reverse stacked LSTM layer to gain deep semantic information for a sequence. Experiments on two benchmark datasets, MSRA and ResumeNER, show that our approach can effectively improve the performance of Chinese NER.

Aspect-level sentiment analysis is a fine-grained task in sentiment analysis that aims at detecting sentiment polarity towards a specific target in a sentence. Previous studies focus on using global attention mechanism that attends to all words in the context to model the interaction between target and sentence. However, global attention suffers from assigning high-attention score to irrelevant sentiment words in the cases where sentence contains noisy words or multiple targets. To address this problem, we propose a novel syntax-directed hybrid attention network (SHAN). In SHAN, a global attention is employed to capture coarse information about the target, and a syntax-directed local attention is used to take a look at words syntactically close to the target. An information gate is then utilized to synthesize the information from local and global attention results and adaptively generate a less-noisy and more sentiment-oriented representation. The experimental results on SemEval 2014 Datasets demonstrate the effectiveness of the proposed method.

In the electromagnetic space, a single channel radar receiver will often intercept several periodic pulse trains radiating from the surrounding emitters simultaneously. The aim of radar pulse deinterleaving is to sort out the pulses coming from different emitters. Most traditional pulse repetition interval (PRI) deinterleaving methods are easy to sort out the pulses with small PRI fluctuations but difficult in dealing with relatively bigger fluctuation or staggered PRIs. In addition, this searching procedure can easily cause pulse omission phenomenon and even generate false pulses. In this paper, we propose an improved histogram method for PRI deinterleaving based on pulse correlation to overcome the above-mentioned shortcomings. After calculating the multi-level time difference histogram, we introduce the mean filter and interquartile range algorithm to optimize the estimated PRI values. Our method extracts the pulse pairs based on pulse correlation directly instead of searching for the pulses and then determines whether the PRI is staggered or not. The experiments on simulation data show that our method can achieve better performance on both the pulse trains of jittered PRIs and the staggered PRIs.

Scattering characteristics of targets are of great importance for synthetic aperture radar (SAR) image analysis. In this paper, a novel template matching based aircraft recognition method with scattering structure feature (SSF) is proposed to improve recognition accuracy and efficiency in SAR images, mainly including the feature model construction stage and the recognition stage. In the former stage, the SSF, being composed of strong scattering point and its corresponding scattering intensity distribution, is extracted by a scattering structure feature model newly defined with Gaussian mixture model. In the recognition stage, template matching is implemented via the proposed sample-decision optimization algorithm. Specifically, in the sample step, a geometric prior based Monte Carlo method with Hausdorff distance is introduced to improve the efficiency of candidate template selection. In the decision step, coordinate translation Kullback–Leibler divergence is proposed by defining a new entropy function of translation coordinates to achieve the goal of translation invariance. Experimental results are given to demonstrate the accuracy and efficiency of the proposed method.

With the development of information technology for modern military confrontations, radar emitter fingerprint identification has become a hot and difficult topic in the field of electronic warfare, especially in the field of electronic reconnaissance. Owing to the confidentiality of military systems, most of the existing studies use simulation data for radar emitter fingerprint identification experiments and analysis. However, most of the existing modeling methods focus on the mechanism analysis of the nonlinear fingerprint characteristics of a single independent component. Its main disadvantage is that it can only represent the nonlinear fingerprint characteristics of some components in the radar emitter system but cannot fully reflect the nonlinear fingerprint characteristics of the whole radar emitter system. In this paper, a nonlinear fingerprint-level radar simulation modeling method is proposed. In contrast to the previous single component modeling method, the systematic nonlinear characteristic modeling method of this model can provide individual radar signal data under different modulation modes and working parameters, and provide experimental conditions for data support and theoretical analysis of radar emitter fingerprint identification.

High-quality digital road maps are essential prerequisites of location-based services and smart city applications. The massive and accessible GPS trajectory data generated by mobile GPS devices provide a new means through which to generate maps. However, due to the low sampling rate and multi-level disparity problems, automatically generating road maps is challenging and the generated maps cannot yet meet commercial requirements. In this paper, we present a GPS trajectory data-based road tracking algorithm, including an active contour-based road centerline refinement algorithm as the necessary post-processing. First, the low-frequency trajectory data were transferred into a density estimation map representing the roads through a kernel density estimator, for a seeding algorithm to automatically generate the initial points of the road-tracking algorithm. Then, we present a template-matching-based road-direction extraction algorithm for the road trackers to conduct simple correction, based on local density information. Last, we present an active contour-based road centerline refinement algorithm, considering both the geometric information of roads and density information. The generated road map was quantitatively evaluated using maps offered by the OpenStreetMap. Compared to other methods, our approach could produce a higher quality map with fewer zig-zag roads, and therefore more accurately represents reality.

Image classification models have achieved satisfactory performance on many datasets, sometimes even better than humans. However, the model attention is unclear since the lack of interpretability. This paper investigates the fidelity and interpretability of model attention. We propose an Explainable Attribute-based Multi-task (EAT) framework to concentrate the model attention on the discriminative image area and make the attention interpretable. We introduce attributes prediction to the multi-task learning network, helping the network to concentrate attention on the foreground objects. We generate attribute-based textual explanations for the network and ground the attributes on the image to show visual explanations. The multi-modal explanation can not only improve user trust but also help to find the weakness of the network and dataset. Our framework can be generalized to any basic model. We perform experiments on three datasets and five basic models. Results indicate that the EAT framework can give multi-modal explanations that interpret the network decision. The performance of several recognition approaches is improved by guiding network attention.

The requirements of location-based services have generated an increasing need for up-to-date digital road maps. However, traditional methods are expensive and time-consuming, requiring many skilled operators. The feasibility of using massive GPS trajectory data provides a cheap and quick means for generating and updating road maps. The detection of road intersections, being the critical component of a road map, is a key problem in map generation. Unfortunately, low sampling rates and high disparities are ubiquitous among floating car data (FCD), making road intersection detection from such GPS trajectories very challenging. In this paper, we extend a point clustering-based road intersection detection framework to include a post-classification course, which utilizes the geometric features of road intersections. First, we propose a novel turn-point position compensation algorithm, in order to improve the concentration of selected turn-points under low sampling rates. The initial detection results given by the clustering algorithm are recall-focused. Then, we rule out false detections in an extended classification course based on an image thinning algorithm. The detection results of the proposed method are quantitatively evaluated by matching with intersections from OpenStreetMap using a variety of distance thresholds. Compared with other methods, our approach can achieve a much higher recall rate and better overall performance, thereby better supporting map generation and other similar applications.

The question answering system can answer questions from various fields and forms with deep neural networks, but it still lacks effective ways when facing multiple evidences. We introduce a new model called SRQA, which means Synthetic Reader for Factoid Question Answering. This model enhances the question answering system in the multi-document scenario from three aspects: model structure, optimization goal, and training method, corresponding to Multilayer Attention (MA), Cross Evidence (CE), and Adversarial Training (AT) respectively. First, we propose a multilayer attention network to obtain a better representation of the evidences. The multilayer attention mechanism conducts interaction between the question and the passage within each layer, making the token representation of evidences in each layer takes the requirement of the question into account. Second, we design a cross evidence strategy to choose the answer span within more evidences. We improve the optimization goal, considering all the answers' locations in multiple evidences as training targets, which leads the model to reason among multiple evidences. Third, adversarial training is employed to high-level variables besides the word embedding in our model. A new normalization method is also proposed for adversarial perturbations so that we can jointly add perturbations to several target variables. As an effective regularization method, adversarial training enhances the model's ability to process noisy data. Combining these three strategies, we enhance the contextual representation and locating ability of our model, which could synthetically extract the answer span from several evidences. We perform SRQA on the WebQA dataset, and experiments show that our model outperforms the state-of-the-art models (the best fuzzy score of our model is up to 78.56%, with an improvement of about 2%)., Comment: arXiv admin note: text overlap with arXiv:1809.00676

Ship detection is an important issue in both civil and military fields. There are many researches about ship detection in wide sea areas and inshore areas. Most of these researches use the image dataset. In this research, a novel framework for specific ship detection using the electronic reconnaissance dataset is proposed. In the new framework, the clustering methods based on time and trajectory are used to process the dataset. Then, several machine-learning methods are used to build detection models. The long short-term memory models are used to extract time-series features and the carefully designed one-dimensional (1D) convolution neural networks (CNNs) are introduced to extract local features. Experimental results on a large electronic reconnaissance dataset collected from real scenario show the model based on 1D CNN gets better performance than classic detection models and the authors' system achieves a good detection accuracy of 92.5%. Above all, this research is a very valuable exploration for the detection of specific ship based on electronic reconnaissance dataset.

Radar emitter classification (REC) is an essential part of electronic warfare (EW) systems. In REC tasks, after deinterleaving, the intercepted radar signals are classified into specific radar types. With new radar types arising and the electromagnetism environment getting complicated, REC has become a big data problem. Meanwhile, there exist inconsistent features among samples. These two problems can affect the performance of classification. In this work, first, the authors designed a novel encoding method to deal with the inconsistent features. High-dimension sequences of equal length are generated as new features. Then a deep learning model named unidimensional convolutional neural network (U-CNN) is proposed to classify the encoded high-dimension sequences with big data. A large and complex radar emitter's dataset is used to evaluate the performance of the U-CNN model with the encoding method. Experiments show that the authors' proposal gains an improvement of 2-3% in accuracy compared with the state-of-the-art methods, while the time consumed for identifying 45,509 emitters is only 1.95 s using a GPU. Specifically, for 12 indistinguishable radars, the classification accuracy is improved about 15%.

Aircraft type recognition in remote sensing images is a meaningful task. It remains challenging due to the difficulty of obtaining appropriate representation of aircrafts for recognition. To solve this problem, we propose a novel aircraft type recognition framework based on deep convolutional neural networks. First, an aircraft segmentation network is designed to obtain refined aircraft segmentation results which provide significant details to distinguish different aircrafts. Then, a keypoints’ detection network is proposed to acquire aircrafts’ directions and bounding boxes, which are used to align the segmentation results. A new multirotation refinement method is carefully designed to further improve the keypoints’ precision. At last, we apply a template matching method to identify aircrafts, and the intersection over union is adopted to evaluate the similarity between segmentation results and templates. The proposed framework takes advantage of both shape and scale information of aircrafts for recognition. Experiments show that the proposed method outperforms the state-of-the-art methods and can achieve 95.6% accuracy on the challenging data set.

Multiple-object tracking (MOT) has received an increasing attention due to the rapid development of autonomous driving. However, the MOT problem is still challenging mainly due to the occlusion and scale variation. Motivated by the fact that the discriminative correlation filters-based (DCFB) tracking algorithms can tackle these problems and significantly improve the accuracy of single object tracking, how to exploit the DCFB tracking algorithms for MOT is worthy studying. Moreover, the corrupted training samples due to the occlusion make DCFB tracking methods to update the appearance model of target uncorrected and result in tracking drift. In this paper, we exploit Markov decision process to integrate the DCFB tracking method into our MOT framework and address the update problem of the appearance model in DCFB tracking method. Moreover, in order to overcome the challenges of occlusion and scale variation, to prevent target drift during tracking, we use two DCFB trackers with different update frequencies and a novel update strategy to predict the location of targets. The part-based method is used to extract robust features to tackling the challenges of occlusion and scale change. In order to verify the efficiency of our algorithm, experiments are performed based on KITTI tracking benchmark. The results demonstrate that our method achieves state-of-the-art performance and outperforms the state-of-the-art algorithms in road scenarios.

Unlike single geospatial objects extraction from high-resolution remote sensing images, the task of road extraction faces more challenges, including its narrowness, sparsity, diversity, multiscale characteristics, and class imbalance. Focusing on these challenges, this paper proposes an end-to-end framework called the multiple feature pyramid network (MFPN). In MFPN, we design an effective feature pyramid and a tailored pyramid pooling module, taking advantage of multilevel semantic features of high-resolution remote sensing images. In the optimization stage, a weighted balance loss function is presented to solve the class imbalance problem caused by the sparseness of roads. The proposed novel loss function is more sensitive to the misclassified and the sparse real labeled pixels and helps to focus on the spare set of hard pixels in the training stage. Compared with the cross-entropy loss function, the weighted balance loss can reduce training time dramatically for the same precision. Experiments on two challenging datasets of high-resolution remote sensing images which illustrate the performance of the proposed algorithm have achieved significant improvements, especially for narrow rural roads.

Relation detection plays a crucial role in knowledge base question answering, and it is challenging because of the high variance of relation expression in real-world questions. Traditional relation detection models based on deep learning follow an encoding-comparing paradigm, where the question and the candidate relation are represented as vectors to compare their semantic similarity. Max- or average-pooling operation, which is used to compress the sequence of words into fixed-dimensional vectors, becomes the bottleneck of information flow. In this paper, we propose an attention-based word-level interaction model (ABWIM) to alleviate the information loss issue caused by aggregating the sequence into a fixed-dimensional vector before the comparison. First, attention mechanism is adopted to learn the soft alignments between words from the question and the relation. Then, fine-grained comparisons are performed on the aligned words. Finally, the comparison results are merged with a simple recurrent layer to estimate the semantic similarity. Besides, a dynamic sample selection strategy is proposed to accelerate the training procedure without decreasing the performance. Experimental results of relation detection on both SimpleQuestions and WebQuestions datasets show that ABWIM achieves the state-of-the-art accuracy, demonstrating its effectiveness.

Community detection plays a vital role in network analysis, simplification, and compression, which reveals the network structure by dividing a network into several internally dense modules. Among plenty of methods, those based on statistical inference are widely used because they are theoretically sound and consistent. However, in many of them, the number of communities needs to be provided in advance or computed in a time-consuming way and parameters are usually initialized randomly, resulting in unstable accuracy and low convergence rate. In this paper, we present a community detection method based on modified PageRank and stochastic block model, which is able to compute the number of communities by finding community centers and initialize community assignments according to the centers and distance. Experiments on both synthetic and real-world networks prove that our method can intuitively give the number of communities, steadily get results of high NMI and modularity and efficiently speed up the convergence of optimizing likelihood probability.

With the development of deep learning, supervised learning has frequently been adopted to classify remotely sensed images using convolutional networks (CNNs). However, due to the limited amount of labeled data available, supervised learning is often difficult to carry out. Therefore, we proposed an unsupervised model called multiple-layer feature-matching generative adversarial networks (MARTA GANs) to learn a representation using only unlabeled data. MARTA GANs consists of both a generative model $G$ and a discriminative model $D$. We treat $D$ as a feature extractor. To fit the complex properties of remote sensing data, we use a fusion layer to merge the mid-level and global features. $G$ can produce numerous images that are similar to the training data; therefore, $D$ can learn better representations of remotely sensed images using the training data provided by $G$. The classification results on two widely used remote sensing image databases show that the proposed method significantly improves the classification performance compared with other state-of-the-art methods., IEEE GRSL

Scene classification has long been a challenging task in the remote sensing field. Conventional approaches based on hand-crafted features are not suitable in large scale remote sensing images. Conv...

Cloud removal is an essential step to enhance the quality of cloud-covered remote sensing image. In recent years, conditional Generative Adversarial Network (cGAN) yields promising improvement in plentiful image-to-image translation tasks. In this paper, we propose a novel objective function to upgrade the structural similarity index based on cGAN. We discover that ImageGAN is effective to focus on global information for thick cloud-covered images and Patch-GAN has fewer parameters while maintaining outstanding performance for thin cloud-covered remote sensing images in the experiments. Experimental results demonstrate that our method achieves remarkable performance in both PSNR, SSIM and visual effect on cloud-covered remote sensing images especially thin cloud-covered images.

In this paper, we present a semi-supervised remote sensing change detection method based on graph model with Generative Adversarial Networks (GANs). Firstly, the multi-temporal remote sensing change detection problem is converted as a problem of semi-supervised learning on graph where a majority of unlabeled nodes and a few labeled nodes are contained. Then, GANs are adopted to generate samples in a competitive manner and help improve the classification accuracy. Finally, a binary change map is produced by classifying the unlabeled nodes to a certain class with the help of both the labeled nodes and the unlabeled nodes on graph. Experimental results carried on several very high resolution remote sensing image data sets demonstrate the effectiveness of our method.

In recent years, entity-based ranking models have led to exciting breakthroughs in the research of information retrieval. Compared with traditional retrieval models, entity-based representation enables a better understanding of queries and documents. However, the existing entity-based models neglect the importance of entities in a document. This paper attempts to explore the effects of the importance of entities in a document. Specifically, the dataset analysis is conducted which verifies the correlation between the importance of entities in a document and document ranking. Then, this paper enhances two entity-based models&mdash, toy model and Explicit Semantic Ranking model (ESR)&mdash, by considering the importance of entities. In contrast to the existing models, the enhanced models assign the weights of entities according to their importance. Experimental results show that the enhanced toy model and ESR can outperform the two baselines by as much as 4.57% and 2.74% on NDCG@20 respectively, and further experiments reveal that the strength of the enhanced models is more evident on long queries and the queries where ESR fails, confirming the effectiveness of taking the importance of entities into account.

Aspect Term Extraction (ATE) plays an important role in aspect-based sentiment analysis. Syntax-based neural models that learn rich linguistic knowledge have proven their effectiveness on ATE. However, previous approaches mainly focus on modeling syntactic structure, neglecting rich interactions along dependency arcs. Besides, these methods highly rely on results of dependency parsing and are sensitive to parsing noise. In this work, we introduce a syntax-directed attention network and a contextual gating mechanism to tackle these issues. Specifically, a graphical neural network is utilized to model interactions along dependency arcs. With the help of syntax-directed self-attention, it could directly operate on syntactic graph and obtain structural information. We further introduce a gating mechanism to synthesize syntactic information with structure-free features. This gate is utilized to reduce the effects of parsing noise. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on three widely used benchmark datasets.

Link prediction is a task predicting whether there is a link between two nodes in a network. Traditional link prediction methods that assume handcrafted features (such as common neighbors) as the link&rsquo, s formation mechanism are not universal. Other popular methods tend to learn the link&rsquo, s representation, but they cannot represent the link fully. In this paper, we propose Edge-Nodes Representation Neural Machine (ENRNM), a novel method which can learn abundant topological features from the network as the link&rsquo, s representation to promote the formation of the link. The ENRNM learns the link&rsquo, s formation mechanism by combining the representation of edge and the representations of nodes on the two sides of the edge as link&rsquo, s full representation. To predict the link&rsquo, s existence, we train a fully connected neural network which can learn meaningful and abundant patterns. We prove that the features of edge and two nodes have the same importance in link&rsquo, s formation. Comprehensive experiments are conducted on eight networks, experiment results demonstrate that the method ENRNM not only exceeds plenty of state-of-the-art link prediction methods but also performs very well on diverse networks with different structures and characteristics.

Ship detection plays an important role in automatic remote sensing image interpretation. The scale difference, large aspect ratio of ship, complex remote sensing image background and ship dense parking scene make the detection task difficult. To handle the challenging problems above, we propose a ship rotation detection model based on a Feature Fusion Pyramid Network and deep reinforcement learning (FFPN-RL) in this paper. The detection network can efficiently generate the inclined rectangular box for ship. First, we propose the Feature Fusion Pyramid Network (FFPN) that strengthens the reuse of different scales features, and FFPN can extract the low level location and high level semantic information that has an important impact on multi-scale ship detection and precise location of dense parking ships. Second, in order to get accurate ship angle information, we apply deep reinforcement learning to the inclined ship detection task for the first time. In addition, we put forward prior policy guidance and a long-term training method to train an angle prediction agent constructed through a dueling structure Q network, which is able to iteratively and accurately obtain the ship angle. In addition, we design soft rotation non-maximum suppression to reduce the missed ship detection while suppressing the redundant detection boxes. We carry out detailed experiments on the remote sensing ship image dataset, and the experiments validate that our FFPN-RL ship detection model has efficient detection performance.

The growing use of deep neural networks in critical applications is making interpretability urgently to be solved. Local interpretation methods are the most prevalent and accepted approach for understanding and interpreting deep neural networks. How to effectively evaluate the local interpretation methods is challenging. To address this question, a unified evaluation framework is proposed, which assesses local interpretation methods from three dimensions: accuracy, persuasibility and class discriminativeness. Specifically, in order to assess correctness, we designed an interactive user feature annotation tool to provide ground truth for local interpretation methods. To verify the usefulness of the interpretation method, we iteratively display part of the interpretation results, and then ask users whether they agree with the category information. At the same time, we designed and built a set of evaluation data sets with a rich hierarchical structure. Surprisingly, one finding is that the existing visual interpretation methods cannot satisfy all evaluation dimensions at the same time, and each has its own shortcomings.

As most networks come with some content in each node, attributed network embedding has aroused much research interest. Most existing attributed network embedding methods aim at learning a fixed representation for each node encoding its local proximity. However, those methods usually neglect the global information between nodes distant from each other and distribution of the latent codes. We propose Structural Adversarial Variational Graph Auto-Encoder (SAVGAE), a novel framework which encodes the network structure and node content into low-dimensional embeddings. On one hand, our model captures the local proximity and proximities at any distance of a network by exploiting a high-order proximity indicator named Rooted Pagerank. On the other hand, our method learns the data distribution of each node representation while circumvents the side effect its sampling process causes on learning a robust embedding through adversarial training. On benchmark datasets, we demonstrate that our method performs competitively compared with state-of-the-art models.

Named Entity Recognition (NER) is a challenging task that extracts named entities from unstructured text data, including news, articles, social comments, etc. The NER system has been studied for decades. Recently, the development of Deep Neural Networks and the progress of pre-trained word embedding have become a driving force for NER. Under such circumstances, how to make full use of the information extracted by word embedding requires more in-depth research. In this paper, we propose an Adversarial Trained LSTM-CNN (ASTRAL) system to improve the current NER method from both the model structure and the training process. In order to make use of the spatial information between adjacent words, Gated-CNN is introduced to fuse the information of adjacent words. Besides, a specific Adversarial training method is proposed to deal with the overfitting problem in NER. We add perturbation to variables in the network during the training process, making the variables more diverse, improving the generalization and robustness of the model. Our model is evaluated on three benchmarks, CoNLL-03, OntoNotes 5.0, and WNUT-17, achieving state-of-the-art results. Ablation study and case study also show that our system can converge faster and is less prone to overfitting.