Search

In recent years, the bionic polarized light compass has been widely studied for the unmanned aerial vehicle navigation. However, it is found from the obtained investigation results that a polarized light compass with a sensitive and high dynamic range polarimeter still provides inferior output precision of the heading angle due to the presence of the noise generating from the compass. The noise is existed not only in the angle of the polarization image acquired by polarimeters but also in the output heading data, which leads to a sharp reduction in the accuracy of a polarized light compass. Herein, we present noise analysis and a novel multiscale transform denoising method of a polarized light compass used for the unmanned aerial vehicle navigation. Specifically, a multiscale principle component analysis utilizing one-dimensional image entropy as classification criterion is directly implemented to suppress the noise in the acquired polarization image. Subsequently, a multiscale time–frequency peak filtering method using the sample entropy as classification criterion is applied for the output heading data so as to further increase the heading measurement accuracy from the denoised image above. These two approaches are combined to significantly reduce the heading error affected by different types of noises. Our experimental results indicate the proposed multiscale transform denoising method exhibits high performance in suppressing the noise of a polarized light compass used for the unmanned aerial vehicle navigation compared to existing prior arts.

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm., arXiv admin note: text overlap with arXiv:1808.02215 by other authors

Bike-sharing systems are becoming popular and generate a large volume of trajectory data. In a bike-sharing system, users can borrow and return bikes at different stations. In particular, a bike-sharing system will be affected by weather, the time period, and other dynamic factors, which challenges the scheduling of shared bikes. In this article, a new shared-bike demand forecasting model based on dynamic convolutional neural networks, called SDF , is proposed to predict the demand of shared bikes. SDF chooses the most relevant weather features from real weather data by using the Pearson correlation coefficient and transforms them into a two-dimensional dynamic feature matrix, taking into account the states of stations from historical data. The feature information in the matrix is extracted, learned, and trained with a newly proposed dynamic convolutional neural network to predict the demand of shared bikes in a dynamical and intelligent fashion. The phase of parameter update is optimized from three aspects: the loss function, optimization algorithm, and learning rate. Then, an accurate shared-bike demand forecasting model is designed based on the basic idea of minimizing the loss value. By comparing with classical machine learning models, the weight sharing strategy employed by SDF reduces the complexity of the network. It allows a high prediction accuracy to be achieved within a relatively short period of time. Extensive experiments are conducted on real-world bike-sharing datasets to evaluate SDF. The results show that SDF significantly outperforms classical machine learning models in prediction accuracy and efficiency.

For the multi-label text classification problems with many classes, many existing multi-label classification algorithms become infeasible or suffer an unaffordable cost. Some researches hence perform the Label Space Dimension Reduction(LSDR) to solve this problem, but a number of methods ignore the sequence information of texts and the label correlation in the original label space, and treat the label as a meaningless multi-hot vector. In this paper, we put forward a multi-label text classification algorithm LELC(joint learning from Label Embedding and Label Correlation) based on the multi-layer attention and label correlation to solve the issue of multi-label text classification with a large number of class labels. Specifically, we firstly extract features through Bidirectional Gated Recurrent Unit Network(Bi-GRU), multi-layer attention and linear layers. Bi-GRU will capture the content information and sequence information of the text at the same time, and the attention mechanism can help us select the valid features related to labels. Then, we use matrix factorization to perform LSDR, and consider label correlation of the original label space in this process, which allows us to implicitly encode the latent space and simplify the model learning. Finally, Deep Canonical Correlation Analysis(CCA) technology is exploited to couple features and the latent space in an end-to-end pattern, so that these two can influence each other to learn the mapping of feature space to latent space. Experiments on 11 real-world datasets show the comparability between our proposed model and the state-of-the-art methods.

Knowledge Graph (KG), which commonly consists of fruitful connected facts about items, presents an unprecedented opportunity to alleviate the sparsity problem in recommender system. However, existing KG based recommendation methods mainly rely on handcrafted meta-path features or simple triple-level entity embedding, which cannot automatically capture entities’ long-term relational dependencies for the recommendation. Specially, entity embedding learning is not properly designed to combine user-item interaction information with KG context information. In this paper, a two-channel neural interaction method named Knowledge Graph enhanced Neural Collaborative Filtering with Residual Recurrent Network (KGNCF-RRN) is proposed, which leverages both long-term relational dependencies KG context and user-item interaction for recommendation. (1) For the KG context interaction channel, we propose Residual Recurrent Network (RRN) to construct context-based path embedding, which incorporates residual learning into traditional recurrent neural networks (RNNs) to efficiently encode the long-term relational dependencies of KG. The self-attention network is then applied to the path embedding to capture the polysemy of various user interaction behaviours. (2) For the user-item interaction channel, the user and item embeddings are fed into a newly designed two-dimensional interaction map. (3) Finally, above the two-channel neural interaction matrix, we employ a convolutional neural network to learn complex correlations between user and item. Extensive experimental results on three benchmark datasets show that our proposed approach outperforms existing state-of-the-art approaches for knowledge graph based recommendation.

Local causal structure learning aims to discover and distinguish the direct causes and direct effects of a target variable. However, the state-of-the-art local causal structure learning algorithms need to perform an exhaustive subset search within the currently selected variables for PC (i.e., parents and children) discovery. In this article, we propose an efficient local causal structure learning algorithm around a target variable, called LCS-FS ( L ocal C ausal S tructure learning by F eature S election). First, to construct the local causal skeleton of the target, we employ feature selection for finding PC without searching for conditioning sets to speed up PC discovery, leading to improve the skeleton construction efficiency. Second, to orient edges in this local causal skeleton, we propose an efficient method to find separating sets from the subsets of PC for identifying V-structures. With the integration of feature selection and the new way of finding separating sets, LCS-FS recursively finds the spouses of Markov blankets in local causal skeleton for edge orientations, until the direct causes and direct effects of the target are distinguished. The experiments on five benchmark Bayesian networks with the number of variables from 35 to 801 validate that our algorithm achieves higher efficiency and better accuracy than the state-of-the-art local causal structure learning algorithms.

In multi-label learning, label correlations commonly exist in the data. Such correlation not only provides useful information, but also imposes significant challenges for multi-label learning. Recently, label-specific feature embedding has been proposed to explore label-specific features from the training data, and uses feature highly customized to the multi-label set for learning. While such feature embedding methods have demonstrated good performance, the creation of the feature embedding space is only based on a single label, without considering label correlations in the data. In this article, we propose to combine multiple label-specific feature spaces, using label correlation, for multi-label learning. The proposed algorithm, mu lti- l abel-specific f eature space e nsemble (MULFE), takes consideration label-specific features, label correlation, and weighted ensemble principle to form a learning framework. By conducting clustering analysis on each label’s negative and positive instances, MULFE first creates features customized to each label. After that, MULFE utilizes the label correlation to optimize the margin distribution of the base classifiers which are induced by the related label-specific feature spaces. By combining multiple label-specific features, label correlation based weighting, and ensemble learning, MULFE achieves maximum margin multi-label classification goal through the underlying optimization framework. Empirical studies on 10 public data sets manifest the effectiveness of MULFE.

Cross-domain recommendation can alleviate the data sparsity problem in the target domain and has become a promising research area. Recently, various models have been proposed to provide recommendation across domains. Some models successfully embedded adversarial learning to reduce domain discrepancies between the source and target domains; however, these models only focuse on extracting the domain-shared features among multiple domains. In this paper, we devise a novel framework that considers both domain-shared and domain-specific knowledge across domains, namely, DAAN . Specifically, we tightly couple matrix factorization-based collaborative filtering with deep adversarial domain adaptation via an attention network . In this framework, we first learn the domain-specific representations for each user and each item from the source and target user-item interaction matrices . Then, we capture the domain-shared features between two domains with common user (or item) embeddings in a domain-adversarial paradigm. Additionally, an attention network is used to adjust the degree of importance between domain-shared and domain-specific knowledge. Extensive experiments on six tasks demonstrate that our method outperforms various baselines in terms of two ranking metrics. To the best of our knowledge, our proposed approach is the first deep model that considers both domain-shared and domain-specific knowledge across domains within a domain-adversarial training paradigm for cross-domain recommendation.

Most recent approaches for Text Simplification (TS) have drawn on insights from machine translation to learn simplification rewrites from the monolingual parallel corpus of complex and simple sentences, yet their effectiveness strongly relies on large amounts of parallel sentences. However, there has been a serious problem haunting TS for decades, that is, the availability of parallel TS corpora is scarce or not fit for the learning task. In this paper, we will focus on one especially useful and challenging problem of unsupervised TS without a single parallel sentence. To the best of our knowledge, we present the first unsupervised text simplification system based on phrase-based machine translation system, which leverages a careful initialization of phrase tables and language models. On the widely used WikiLarge and WikiSmall benchmarks, our system respectively obtains 39.08 and 25.12 SARI points, even outperforms some supervised baselines.

Deep learning seeks to achieve excellent performance for representation learning in image datasets. However, supervised deep learning models such as convolutional neural networks require a large number of labeled image data, which is intractable in applications, while unsupervised deep learning models like stacked denoising auto-encoder cannot employ label information. Meanwhile, the redundancy of image data incurs performance degradation on representation learning for aforementioned models. To address these problems, we propose a semi-supervised deep learning framework called stacked convolutional sparse auto-encoder, which can learn robust and sparse representations from image data with fewer labeled data records. More specifically, the framework is constructed by stacking layers. In each layer, higher layer feature representations are generated by features of lower layers in a convolutional way with kernels learned by a sparse auto-encoder. Meanwhile, to solve the data redundance problem, the algorithm of Reconstruction Independent Component Analysis is designed to train on patches for sphering the input data. The label information is encoded using a Softmax Regression model for semi-supervised learning. With this framework, higher level representations are learned by layers mapping from image data. It can boost the performance of the base subsequent classifiers such as support vector machines. Extensive experiments demonstrate the superior classification performance of our framework compared to several state-of-the-art representation learning methods.

Learning with streaming data has received extensive attention during the past few years. Existing approaches assume that the feature space is fixed or changes by following explicit regularities, limiting their applicability in real-time applications. For example, in a smart healthcare platform, the feature space of the patient data varies when different medical service providers use nonidentical feature sets to describe the patients' symptoms. To fill the gap, we in this article propose a novel learning paradigm, namely, Generative Learning With Streaming Capricious (GLSC) data, which does not make any assumption on the feature space dynamics. In other words, GLSC handles the data streams with a varying feature space, where each arriving data instance can arbitrarily carry new features and/or stop carrying partial old features. Specifically, GLSC trains a learner on a universal feature space that establishes relationships between old and new features, so that the patterns learned in the old feature space can be used in the new feature space. The universal feature space is constructed by leveraging the relatednesses among features. We propose a generative graphical model to model the construction process, and show that learning from the universal feature space can effectively improve the performance with theoretical guarantees. The experimental results demonstrate that GLSC achieves conspicuous performance on both synthetic and real data sets.

With the prevalence of online review websites, large-scale data promote the necessity of focused analysis. This task aims to capture the information that is highly relevant to a specific aspect. However, the broad scope of the aspects of the various products makes this task overarching but challenging. A commonly used solution is to modify the topic models with additional information to capture the features for a specific aspect (referred to as a targeted aspect). However, the existing topic models, either perform the full analysis to capture features as many as possible or estimate the similarity to capture features as coherent as possible, overlook the fine-grained semantic relations between the features, resulting in the captured features coarse and confusing. In this paper, we propose a novel Hierarchical Features-based Topic Model (HFTM) to extract targeted aspects from online reviews, then to capture the aspect-specific features. Specifically, our model can not only capture the direct features posing target-to-feature semantics but also capture the latent features posing feature-to-feature semantics. The experiments conducted on real-world datasets demonstrate that HFTMl outperforms the state-of-the-art baselines in terms of both aspect extraction and document classification.

Lexical simplification has attracted much attention in many languages, which is the process of replacing complex words in a given sentence with simpler alternatives of equivalent meaning. Although the richness of vocabulary in Chinese makes the text very difficult to read for children and non-native speakers, there is no research work for the Chinese lexical simplification (CLS) task. To circumvent difficulties in acquiring annotations, we manually create the first benchmark dataset for CLS, which can be used for evaluating the lexical simplification systems automatically. To acquire a more thorough comparison, we present five different types of methods as baselines to generate substitute candidates for the complex word that includes synonym-based approach, word embedding-based approach, BERT-based approach, sememe-based approach, and a hybrid approach. Finally, we design the experimental evaluation of these baselines and discuss their advantages and disadvantages. To our best knowledge, this is the first study for CLS task.

Chinese word embedding models capture Chinese semantics based on the character feature of Chinese words and the internal features of Chinese characters such as radical, component, stroke, structure and pinyin. However, some features are overlapping and most methods do not consider their relevance. Meanwhile, they express words as point vectors that cannot better capture different aspect semantics of Chinese words. In this paper, we propose a Feature Subsequence based Probability Representation Model (FSPRM) for learning Chinese word embeddings, in which we first integrate the morphological and phonetic features (stroke, structure and pinyin) of Chinese characters and learn their relevance by designing a feature subsequence to capture relatively comprehensive semantics of Chinese words, then feature probability distribution is proposed for capturing different aspect meanings of Chinese words based on the three internal features and probability representation by estimating its mean as the sum of feature subsequences. Chinese words with similar features may have similar semantics, then we map Chinese words to feature probability distributions and design a similarity-based objective for predicting the contextual words of the target word to learn their semantics. Extensive experiments on word analogy, word similarity, text classification and named entity recognition tasks demonstrate that the proposed method outperforms most state-of-the-art approaches.

Lexical simplification (LS) aims at replacing complex words with simpler alternatives. LS commonly consists of three main steps: complex word identification, substitute generation, and substitute ranking. Existing LS methods focus on the contextual information of the complex word in the last step (substitute ranking). However, they miss out the following two facts: (1) The word complexity of a polysemous word is very closely related to its context; (2) The step of substitute generation regardless of the context will inevitably produce a large number of spurious candidates. Therefore, we propose a novel LS system LSBert based on pretrained language model BERT to address the aforementioned issues, which is capable of making use of the wider context when both identifying the words in need of simplification and generating substitute candidates for the complex words.Specifically, LSBert consists of a network for complex word identification by fine-tuning BERT and a network for substitute generation based on BERT. Experimental results show that LSBert performs well in both complex word identification and substitute generation, achieving state-of-the-art results in three benchmarks. To facilitate reproducibility, the code of the LSBert system is available at https://github.com/qiang2100/BERT-LS .

Topic modeling has demonstrated its value in short text topic discovery. For this task, a common way adopted by many topic models is to perform a full analysis to find all the possible topics. However, these topic models overlook the importance of deeper topics, leading to confusing topics discovered. In practice, people always tend to find more focused topics on some special aspects (or events), rather than a set of coarse topics. Therefore, in this paper, we propose a novel method, Targeted Aspects Oriented Topic Modeling (TATM), to discover more focused topics on specific aspects in short texts. Specifically, each short text is assigned to only one targeted aspect derived from an enhanced Dirichlet Multinomial Mixture process (E-DMM). This process helps group similar words as many as possible, which achieves topic homogeneity. In addition, TATM discovers the topics for each targeted aspect from as many angles as possible by performing target-level modeling, which achieves topic completeness. Thus, TATM can make a balance between the two conflicting properties without employing any additional information or pre-trained knowledge. The extensive experiments conducted on five real-world datasets demonstrate that our proposed model can effectively discover more focused and complete topics, and it outperforms the state-of-the-art baselines.

With the rapid increase of internet information, personalized recommendation systems are an effective way to alleviate the information overload problem, which has attracted extensive attention in recent years. The traditional collaborative filtering utilizes matrix factorization methods to learn hidden feature representations of users and/or items. With deep learning achieved good performance in representation learning, the autoencoder model is widely applied in recommendation systems for the advantages of fast convergence and no label requirement. However, the previous recommendation systems may take the reconstruction output of an autoencoder as the prediction of missing values directly, which may deteriorate their performance and cause unsatisfactory results of recommendation. In addition, the parameters of an autoencoder need to be pre-trained ahead, which greatly increases the time complexity. To address these problems, in this paper, we propose a Hybrid Collaborative Recommendation method via Dual-Autoencoder (HCRDa). More specifically, firstly, a novel dual-autoencoder is utilized to simultaneously learn the feature representations of users and items in our HCRDa, which obviously reduces time complexity. Secondly, embedding matrix factorization into the training process of the autoencoder further improves the quality of hidden features for users and items. Finally, additional attributes of users and items are utilized to alleviate the cold start problem and to make hybrid recommendations. Comprehensive experiments on several real-world data sets demonstrate the effectiveness of our proposed method in comparison with several state-of-the-art methods.

The Markov blanket (MB), a key concept in a Bayesian network (BN), is essential for large-scale BN structure learning and optimal feature selection. Many MB discovery algorithms that are either efficient or effective have been proposed for addressing high-dimensional data. In this paper, we propose a new algorithm for E fficient and E ffective MB discovery, called EEMB. Specifically, given a target feature, the EEMB algorithm discovers the PC (i.e., parents and children) and spouses of the target simultaneously and can distinguish PC from spouses during MB discovery. We compare EEMB with the state-of-the-art MB discovery algorithms using a series of benchmark BNs and real-world datasets. The experiments demonstrate that EEMB is competitive with the fastest MB discovery algorithm in terms of computational efficiency and achieves almost the same MB discovery accuracy as the most accurate of the compared algorithms.

The excellent performance of transfer learning has emerged in the past few years. How to find feature representations which minimize the distance between source and target domains is a crucial problem in transfer learning. Recently, deep learning methods have been proposed to learn higher level and robust representations. However, in traditional methods, label information in source domain is not designed to optimize both feature representations and parameters of the learning model. Additionally, the redundancy of data may incur performance degradation on transfer learning. To address these problems, we propose a novel semi-supervised representation deep learning framework for transfer learning. To obtain this framework, manifold regularization is integrated for the parameter optimization, and the label information is encoded using a softmax regression model in auto-encoders. Meanwhile, whitening layer is introduced to reduce the redundancy of data before auto-encoders. Extensive experiments demonstrate the effectiveness of our proposed framework compared to other competing state-of-the-art baseline methods.

The spread of the Coronavirus pandemic has been accompanied by an infodemic. The false information that is embedded in the infodemic affects people’s ability to have access to safety information and follow proper procedures to mitigate the risks. This research aims to target the falsehood part of the infodemic, which prominently proliferates in news articles and false medical publications. Here, we present NeoNet, a novel supervised machine learning text mining algorithm that analyzes the content of a document (news article, a medical publication) and assigns a label to it. The algorithm is trained by TFIDF bigram features which contribute a network training model. The algorithm is tested on two different real-world datasets from the CBC news network and Covid-19 publications. In five different fold comparisons, the algorithm predicted a label of an article with a precision of 97-99 %. When compared with prominent algorithms such as Neural Networks, SVM, and Random Forests NeoNet surpassed them. The analysis highlighted the promise of NeoNet in detecting disputed online contents which may contribute negatively to the COVID-19 pandemic.

For sequence classification, an important issue is to find discriminative features, where sequential pattern mining (SPM) is often used to find frequent patterns from sequences as features. To improve classification accuracy and pattern interpretability, contrast pattern mining emerges to discover patterns with high-contrast rates between different categories. To date, existing contrast SPM methods face many challenges, including excessive parameter selection and inefficient occurrences counting. To tackle these issues, this article proposes a top- k self-adaptive contrast SPM, which adaptively adjusts the gap constraints to find top- k self-adaptive contrast patterns (SCPs) from positive and negative sequences. One of the key tasks of the mining problem is to calculate the support (the number of occurrences) of a pattern in each sequence. To support efficient counting, we store all occurrences of a pattern in a special array in a Nettree, an extended tree structure with multiple roots and multiple parents. We employ the array to calculate the occurrences of all its superpatterns with one-way scanning to avoid redundant calculation. Meanwhile, because the contrast SPM problem does not satisfy the Apriori property, we propose Zero and Less strategies to prune candidate patterns and a Contrast-first mining strategy to select patterns with the highest contrast rate as the prefix subpattern and calculate the contrast rate of all its superpatterns. Experiments validate the efficiency of the proposed algorithm and show that contrast patterns significantly outperform frequent patterns for sequence classification. The algorithms and datasets can be downloaded from https://github.com/wuc567/Pattern-Mining/tree/master/SCP-Miner.

The spread of the Coronavirus pandemic has been accompanied by an infodemic. The false information that is embedded in the infodemic affects people’s ability to have access to safety and follow proper procedures to mitigate the risks. Here, we present a novel supervised machine learning text mining algorithm that analyzes the content of a given news article and assign a label to it. The NeoNet algorithm is trained by noun-phrases features which contributes a network model. The algorithm was tested on a real-world dataset and predicted the label of never-seem articles and flags ones that are suspicious or disputed. In five different fold comparisons, NeoNet surpassed prominent contemporary algorithm such as Neural Networks, SVM, and Random Forests. The analysis shows that the NeoNet algorithm predicts a label of an article with a 100% precision using a non-pruned model. This highlights the promise of detecting disputed online contents that may contribute negatively to the COVID-19 pandemic. Indeed, using machine learning combined with powerful text mining and network science provide the necessary tools to counter the spread of misinformation, disinformation, fake news, rumors, and conspiracy theories that is associated with the COVID19 Infodemic.

Lifelong machine learning (LML) has driven the development of extensive web applications, enabling the learning systems deployed on web servers to deal with a sequence of tasks in an incremental fashion. Such systems can retain knowledge from learned tasks in a knowledge base and seamlessly apply it to improve the future learning. Unfortunately, most existing LML methods require labels in every task, whereas providing persistent human labeling for all future tasks is costly, onerous, error-prone, and hence impractical. Motivated by this situation, we propose a new paradigm named unsupervised lifelong learning with curricula (ULLC), where only one task needs to be labeled for initialization and the system then performs lifelong learning for subsequent tasks in an unsupervised fashion. A main challenge of realizing this paradigm lies in the occurrence of negative knowledge transfer, where partial old knowledge becomes detrimental for learning a given task yet cannot be filtered out by the learner without the help of labels. To overcome this challenge, we draw insights from the learning behaviors of humans. Specifically, when faced with a difficult task that cannot be well tackled by our current knowledge, we usually postpone it and work on some easier tasks first, which allows us to grow our knowledge. Thereafter, once we go back to the postponed task, we are more likely to tackle it well as we are more knowledgeable now. The key idea of ULLC is similar – at any time, a pool of candidate tasks are organized in a curriculum by their distances to the knowledge base. The learner then starts from the closer tasks, accumulates knowledge from learning them, and moves to learn the faraway tasks with a gradually augmented knowledge base. The viability and effectiveness of our proposal are substantiated through extensive empirical studies on both synthetic and real datasets.

After entering the big data era, a new term of ‘big knowledge’ has been coined to deal with challenges in mining a mass of knowledge from big data. While researchers used to explore the basic characteristics of big data, we have not seen any studies on the general and essential properties of big knowledge. To fill this gap, this paper studies the concepts of big knowledge, big-knowledge system, and big-knowledge engineering. Ten massiveness characteristics for big knowledge and big-knowledge systems, including massive concepts, connectedness, clean data resources, cases, confidence, capabilities, cumulativeness, concerns, consistency, and completeness, are defined and explored. Based on these characteristics, a comprehensive investigation is conducted on some large-scale knowledge engineering projects, including the Fifth Comprehensive Traffic Survey in Shanghai, the China's Xia-Shang-Zhou Chronology Project, the Troy and Trojan War Project, and the International Human Genome Project, as well as the online free encyclopedia Wikipedia. We also investigate the recent research efforts on knowledge graphs, where they are analyzed to determine which ones can be considered as big knowledge and big-knowledge systems. Further, a definition of big-knowledge engineering and its life cycle paradigm is presented. All of these projects are accordingly checked to determine whether they belong to big-knowledge engineering projects. Finally, the perspectives of big knowledge research are discussed.

We study the problem of online learning with varying feature spaces. The problem is challenging because, unlike traditional online learning problems, varying feature spaces can introduce new features or stop having some features without following a pattern. Other existing methods such as online streaming feature selection (Wu et al. 2013), online learning from trapezoidal data streams (Zhang et al. 2016), and learning with feature evolvable streams (Hou, Zhang, and Zhou 2017) are not capable to learn from arbitrarily varying feature spaces because they make assumptions about the feature space dynamics. In this paper, we propose a novel online learning algorithm OLVF to learn from data with arbitrarily varying feature spaces. The OLVF algorithm learns to classify the feature spaces and the instances from feature spaces simultaneously. To classify an instance, the algorithm dynamically projects the instance classifier and the training instance onto their shared feature subspace. The feature space classifier predicts the projection confidences for a given feature space. The instance classifier will be updated by following the empirical risk minimization principle and the strength of the constraints will be scaled by the projection confidences. Afterwards, a feature sparsity method is applied to reduce the model complexity. Experiments on 10 datasets with varying feature spaces have been conducted to demonstrate the performance of the proposed OLVF algorithm. Moreover, experiments with trapezoidal data streams on the same datasets have been conducted to show that OLVF performs better than the state-of-the-art learning algorithm (Zhang et al. 2016).

With the crowdsourcing of small tasks becoming easier, it is possible to obtain non-expert/imperfect labels at low cost. With low-cost imperfect labeling, it is straightforward to collect multiple labels for the same data items. This paper proposes strategies of utilizing these multiple labels for supervised learning, based on two basic ideas: majority voting and pairing. We show several interesting results based on our experiments. (i) The strategies based on the majority voting idea work well under the situation where the certainty level is high. (ii) On the contrary, the pairing strategies are more preferable under the situation where the certainty level is low. (iii) Among the majority voting strategies, soft majority voting can reduce the bias and roughness, and perform better than majority voting. (iv) Pairing can completely avoid the bias by having both sides (potentially correct and incorrect/noisy information) considered. Beta estimation is applied to reduce the impact of the noise in pairing. Our experimental results show that pairing with Beta estimation always performs well under different certainty levels. (v) All strategies investigated are labeling quality agnostic strategies for real-world applications, and some of them perform better than or at least very close to the gnostic strategies.

Along with the fast development of network applications, network research has attracted more and more attention, where one of the most important research directions is networked multi-label classification. Based on it, unknown labels of nodes can be inferred by known labels of nodes in the neighborhood. As both the scale and complexity of networks are increasing, the problems of previously neglected system overhead are turning more and more seriously. In this article, a novel multi-objective optimization-based networked multi-label seed node selection algorithm (named as MOSS) is proposed to improve both the prediction accuracy for unknown labels of nodes from labels of seed nodes during classification and the system overhead for mining the labels of seed nodes with third parties before classification. Compared with other algorithms on several real networked data sets, MOSS algorithm not only greatly reduces the system overhead before classification but also improves the prediction accuracy during classification.

Semantic embedding has demonstrated its value in latent representation learning of data, and can be effectively adopted for many applications. However, it is difficult to propose a joint learning framework for semantic embedding in Community Question Answer (CQA), because CQA data have multi-view and sparse properties. In this paper, we propose a generic Multi-modal Multi-view Semantic Embedding (MMSE) framework via a Bayesian model for question answering. Compared with existing semantic learning methods, the proposed model mainly has two advantages: (1) To deal with the multi-view property, we utilize the Gaussian topic model to learn semantic embedding from both local view and global view. (2) To deal with the sparse property of question answer pairs in CQA, social structure information is incorporated to enhance the quality of general text content semantic embedding from other answers by using the shared topic distribution to model the relationship between these two modalities (user relationship and text content). We evaluate our model for question answering and expert finding task, and the experimental results on two real-world datasets show the effectiveness of our MMSE model for semantic embedding learning.

Online streaming feature selection which deals with streaming features in an online manner plays a critical role in big data problems. Many approaches have been proposed to handle this problem. However, most existing methods need domain information before learning and specify some parameters in advance. In real-world applications, we cannot always require the domain information and it is a big challenge to specify uniform parameters for all different types of data sets. Motivated by this, we propose a new online streaming feature selection method based on adaptive density neighborhood relation, named OFS-Density. More specifically, with the neighborhood rough set theory, OFS-Density does not require the domain information before learning. Meanwhile, we propose a new adaptive neighborhood relation using the density information of the surrounding instances, which does not need to specify any parameters in advance. By the fuzzy equal constraint, OFS-Density can select features with a low redundancy. Finally, experimental studies on fourteen datasets show that OFS-Density is superior to traditional feature selection methods with the same numbers of features and state-of-the-art online streaming feature selection algorithms in an online manner.

Based on the performance of entire social networks, anomaly analysis for evolving social networks generally ignores the otherness of the evolutionary behaviors of different nodes, such that it is difficult to precisely identify the anomalous evolutionary behaviors of nodes ( AEBN ). Assuming that a node's evolutionary behavior that generates and removes edges normally follows stable evolutionary mechanisms, this study focuses on detecting and assessing AEBN, whose evolutionary mechanisms deviate from their past mechanisms, and proposes a link prediction detection ( LPD ) method and a matrix perturbation assessment ( MPA ) method. LPD describes a node's evolutionary behavior by fitting its evolutionary mechanism, and designs indexes for edge generation and removal to evaluate the extent to which the evolutionary mechanism of a node's evolutionary behavior can be fitted by a link prediction algorithm. Furthermore, it detects AEBN by quantifying the differences among behavior vectors that characterize the node's evolutionary behaviors in different periods. In addition, MPA considers AEBN as a perturbation of the social network structure, and quantifies the effect of AEBN on the social network structure based on matrix perturbation analysis. Extensive experiments on eight disparate real-world networks demonstrate that analyzing AEBN from the perspective of evolutionary mechanisms is important and beneficial.

Keyphrase extraction is the task of automatically extracting descriptive phrases or concepts that represent the main topics in a document. Finding good keyphrases in a document can quickly summarize knowledge for information retrieval and decision making. Existing keyphrase extraction methods cannot be customized to each specific document, and cannot capture flexible semantic relations. In this paper, a keyphrase extraction algorithm using maximum sequential pattern mining with one-off and general gaps condition, called Ke-MSMING, is presented. Ke_MSMING first searches all keyphrase candidates from a document using sequential patterns mining and the topic model, and then adopts supervised machine learning to classify each keyphrase candidate as a keyphrase or not. Finally, Ke_MSMING selects top-N keyphrases as the final keyphrases. Ke_MSMING not only uses baseline features and pattern features but also uses centrality features obtained from the cooccurrence semantic network, and the cooccurrence networks can yield powerful semantic relations for keyphrase extraction. Experimental results on two datasets demonstrate that Ke_MSMING has better performance than other state-of-the-art keyphrase extraction approaches.

This paper defines big wisdom with a HAO/BIBLE framework, which integrates human intelligence (HI), artificial intelligence (AI), and organizational/business intelligence (O/BL) with Bigdata analytics in Large Environments, for industrial intelligence in organizational activities. Big wisdom starts with Bigdata, discovers big knowledge, and facilitates human and machine synergism for complex problem solving. When the HAO/BIBLE framework is applied to a regular (non-Bigdata) environment, it becomes the well-known PEAS agent structure, and when the knowledge graph in HAO/BIBLE relies on domain expertise (rather than big knowledge), HAO/BIBLE serves as an expert system. This paper compares and contrasts Bigdata, big knowledge, and big wisdom and instantiates HAO/BIBLE with a case study for intelligent catering services to illustrate synergized HAO intelligence (HI?+?AI?+?OI) for big wisdom.

Local causal structure learning aims to discover and distinguish direct causes (parents) and direct effects (children) of a variable of interest from data. While emerging successes have been made, existing methods need to search a large space to distinguish direct causes from direct effects of a target variable T. To tackle this issue, we propose a novel Efficient Local Causal Structure learning algorithm, named ELCS. Specifically, we first propose the concept of N-structures, then design an efficient Markov Blanket (MB) discovery subroutine to integrate MB learning with N-structures to learn the MB of T and simultaneously distinguish direct causes from direct effects of T. With the proposed MB subroutine, ELCS starts from the target variable, sequentially finds MBs of variables connected to the target variable and simultaneously constructs local causal structures over MBs until the direct causes and direct effects of the target variable have been distinguished. Using eight Bayesian networks the extensive experiments have validated that ELCS achieves better accuracy and efficiency than the state-of-the-art algorithms., Comment: Published on IEEE Transactions on Big Data (TBD). arXiv admin note: text overlap with arXiv:1910.01288

Named entity recognition (NER) is a basic task of natural language processing (NLP), whose purpose is to identify named entities such as the names of persons, places, and organizations in the corpus. Utilizing neural networks for feature extraction, followed by conditional random field (CRF) layer decoding, is effective for the NER task. However, achieving reliable results using neural networks generally requires a large amount of labeled data and the acquisition of high-quality labeled data is costly. To obtain a better NER effect without labeled data, we propose a weak supervision approach with adversarial training (WSAT). WSAT obtains supervised information and domain knowledge through labeling functions, including external knowledge bases, heuristic functions, and generic entity recognition tools. The labeled results are aggregated through the linked hidden Markov model (linked HMM), and adversarial training strategies are added when using the aggregated results for training. We evaluate WSAT on two real-world datasets. When compared to rival algorithms, the F1 values are improved by approximately 2% and 1% on the MSRA and Resume NER datasets, respectively.

Predictive modeling of networked data finds many real-world applications, such as fraud detection in social networks, drug discovery in biomedical networks, paper topic classification in citation networks, and so forth. Although the advanced machine learning approaches can help build reasonably accurate predictive models, their applicability is immensely hindered by the data labeling tasks, which are onerous, time-consuming, and error-prone. In this paper, we propose a novel active learning paradigm for networked data, named topology-and-content-aware (TACA) active learning, aiming to minimize the number of labels while achieving a desirable level of model accuracy. Overall, TACA advances existing works from two aspects: (1) TACA makes no assumption on the network property, whereas most existing works only perform effectively on a locally consistent network in which linked nodes are expected to share the same labels and (2) TACA generates queries without relying on model performance, thereby enjoying robust predictive results even when noises exist in the queried labels. Both theoretical and empirical evidences are presented, substantiating the effectiveness of and optimism our approach.

Machine learning can provide deep insights into data, allowing machines to make high-quality predictions and having been widely used in real-world applications, such as text mining, visual classification, and recommender systems. However, most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. This issue calls for the need of {Large-scale Machine Learning} (LML), which aims to learn patterns from big data with comparable performance efficiently. In this paper, we offer a systematic survey on existing LML methods to provide a blueprint for the future developments of this area. We first divide these LML methods according to the ways of improving the scalability: 1) model simplification on computational complexities, 2) optimization approximation on computational efficiency, and 3) computation parallelism on computational capabilities. Then we categorize the methods in each perspective according to their targeted scenarios and introduce representative methods in line with intrinsic strategies. Lastly, we analyze their limitations and discuss potential directions as well as open issues that are promising to address in the future.

Multi-pattern matching with variable-length wildcards is an interesting and important problem in bioinformatics, information retrieval and other domains. Most of the previously developed multi-pattern matching methods, such as famous Aho–Corasick and Wu–Manber algorithms, aimed to solve some classical string matching problems. However, these algorithms are not efficient for patterns with flexible wildcards or do-not-care characters. In this paper, we propose two efficient algorithms for multi-pattern matching with variable-length wildcards based on suffix tree, called MMST-L and MMST-S, according to the length of exact characters in a pattern. Experimental results show that the two MMST algorithms, in most cases, outperform other various versions of comparing algorithms.

As a media for sharing knowledge, forming communities with similar hobbies and interacting with friends, social networks are booming dramatically recently. The study on collective behaviors in social networks is a key to analyze community dynamics and network functionalities. Hence, it is significant and necessary to accurately extract and analyze these collective behaviors. At present, the semantic behaviors of any individual in semantic social networks can be extracted conveniently. However, how to automatically extract collective semantic behaviors, to a large scale, in social networks is still an open question. Our proposed collective semantic behavior extraction process works as follows: Firstly, as for the popular semantic social networks, such as Facebook, Twitter and QQ, it is convenient to extract semantic behaviors from any semantic information. Secondly, as similar behaviors will form a community spontaneously, the communities with similar extracted semantic behaviors can be determined with DeepWalk. Hence, as for a determined community, our proposed collective semantic behavior extraction approach can properly extract the collective semantic behaviors in social networks. The experimental results of our proposed approach executed on real semantic information show that our proposed approach can automatically extract collective semantic behaviors accurately.

Online streaming feature selection has received extensive attention in the past few years. Existing approaches have a common assumption that the feature space of the fixed data instances increases dynamically without any missing entry. This assumption, however, does not always hold in many real-world applications. For example, in a credit evaluation system, we cannot collect the complete dynamic features for each person and/or enterprise. Motivated by this observation, this paper aims at conducting online feature selection from capricious streaming features, where features flow in one by one with some random missing entries while the number of data instances remains fixed. To do so, we propose a general framework named GF-CSF. The main idea of GF-CSF is to adopt latent factor analysis to preprocess capricious streaming features for completing their missing entries before conducting feature selection. Both theoretical and experimental analyses indicate that GF-CSF can efficiently improve any existing model of online streaming features selection to achieve online capricious streaming features selection.

Automatic knowledge graph construction seeks to build a knowledge graph from unstructured text in a specific domain or cross multiple domains, without human intervention. IEEE ICDM 2019 and ICBK 2019 invited teams from both degree-granting institutions and industrial labs to compete in the 2019 Knowledge Graph Contest by automatically constructing knowledge graphs in at least two different domains. This article reports the outcomes of the Contest. The participants were expected to build a model to extract knowledge represented as triplets from text data and develop a web application to visualize the triplets. Awards were given to five teams. Their models and key techniques used to construct knowledge graphs are summarized.

A large amount of data with rich semantic and structural information has been accumulated in many real-world applications. In order to effectively describe the concepts and connections in these data sets, knowledge graph was proposed as a tool to handle it. The genealogy is a typical tree structure data and can be stored in the knowledge graph. However, due to the complexity and increasing volume of the data, how to efficiently extract and visualize the customized information from the big knowledge graph is hence a challenge and worthy of in-depth study. Motivated by this, we propose a novel user-specified information extraction and visualization tool, named CEPV (the Customized information Extracting, Processing and Visualization tool), for converting the big graph structure data into a specified tree structure display. The main steps of CEPV are as follows: firstly, according to the requirements of users, extracting the specified data from the massive, complex, heterogeneous data as fewer times as possible, which can reduce the frequency of database access and improve the overall efficiency of the algorithm. Secondly, the fault tolerance mechanism and attribute judgment rules are executed to ensure the correctness during the data processing. Finally, the processed data with a complex relationship is presented to the user in multiple visualization models. The high availability and effectiveness of our proposed tool is verified on a big knowledge graph dataset.

A huge amount of texts available on the World Wide Web presents an unprecedented opportunity for information extraction (IE). One important assumption in IE is that frequent extractions are more likely to be correct. Sparse IE is hence a challenging task because no matter how big a corpus is, there are extractions supported by only a small amount of evidence in the corpus. However, there is limited research on sparse IE, especially in the assessment of the validity of sparse IEs. Motivated by this, we introduce a lightweight, explicit semantic approach for assessing sparse IE. 1 We first use a large semantic network consisting of millions of concepts, entities, and attributes to explicitly model the context of any semantic relationship. Second, we learn from three semantic contexts using different base classifiers to select an optimal classification model for assessing sparse extractions. Finally, experiments show that as compared with several state-of-the-art approaches, our approach can significantly improve the F -score in the assessment of sparse extractions while maintaining the efficiency.

© 1989-2012 IEEE. Multi-instance learning (MIL) is a useful tool for tackling labeling ambiguity in learning because it allows a bag of instances to share one label. Bag mapping transforms a bag into a single instance in a new space via instance selection and has drawn significant attention recently. To date, most existing work is based on the original space, using all instances inside each bag for bag mapping, and the selected instances are not directly tied to an MIL objective. As a result, it is difficult to guarantee the distinguishing capacity of the selected instances in the new bag mapping space. In this paper, we propose a discriminative mapping approach for multi-instance learning (MILDM) that aims to identify the best instances to directly distinguish bags in the new mapping space. Accordingly, each instance bag can be mapped using the selected instances to a new feature space, and hence any generic learning algorithm, such as an instance-based learning algorithm, can be used to derive learning models for multi-instance classification. Experiments and comparisons on eight different types of real-world learning tasks (including 14 data sets) demonstrate that MILDM outperforms the state-of-The-Art bag mapping multi-instance learning approaches. Results also confirm that MILDM achieves balanced performance between runtime efficiency and classification effectiveness.

Traditional image set classification methods measure the similarities between different sets based on the extracted characteristics of each set. Most of these methods build their models on one kind of visual features or a simply concatenated feature vector of several kinds of features. However, due to redundant or irrelevant information, the concatenated features are usually not discriminant or suffer from the curse of dimensionality . Meanwhile, if the sizes of sets are small, or improper features are employed, the useful information will be limited and conflictive. So in this paper, we propose a set-level joint sparse representation classification (SJSRC) model to combine multiple features to accomplish image set classification task . In the SJSRC, the atom-level and concept-level regularization terms are both imposed to obtain robust representations and the images in the same concept are regarded as a whole to optimize the objective on them jointly to strengthen the intra similarities via a set-level regularization. In addition, we adopt two schemes, namely ‘Anchor Graph’ and ‘Regularized Nearest Points (RNP)’, to improve computational efficiency and identification rate. Experiments on several benchmark datasets show that our model obtains competitive recognition performance for image set classification.

In the era of big data, the dimensionality of data is increasing dramatically in many domains. To deal with high dimensionality, online feature selection becomes critical in big data mining. Recently, online selection of dynamic features has received much attention. In situations where features arrive sequentially over time, we need to perform online feature selection upon feature arrivals. Meanwhile, considering grouped features, it is necessary to deal with features arriving by groups. To handle these challenges, some state-of-the-art methods for online feature selection have been proposed. In this paper, we first give a brief review of traditional feature selection approaches. Then we discuss specific problems of online feature selection with feature streams in detail. A comprehensive review of existing online feature selection methods is presented by comparing with each other. Finally, we discuss several open issues in online feature selection.

This paper proposes an approach KeyRank to extract proper keyphrases from a document in English. It first searches all keyphrase candidates from the document, and then ranks them for selecting top-N ones as final keyphrases. Existing studies show that extracting a complete keyphrase candidate set that includes semantic relations in context, and evaluating the effectiveness of each candidate are crucial to extract high quality keyphrases from documents. Based on that words do not repeatedly appear in an effective keyphrase in English, a novel keyphrase candidate search algorithm using sequential pattern mining with gap constraints (called KCSP) is proposed to extract keyphrase candidates for KeyRank. And then an effectiveness evaluation measure pattern frequency with entropy (called PF-H) is proposed for KeyRank to rank these keyphrase candidates. Our experimental results show that KeyRank has better performance. Its first component KCSP is much more efficient than a closely related approach SPMW, and its second component PF-H is an effective evaluation mechanism for ranking keyphrase candidates. 1

Multilabel learning deals with examples having multiple class labels simultaneously. It has been applied to a variety of applications, such as text categorization and image annotation. A large number of algorithms have been proposed for multilabel learning, most of which concentrate on multilabel classification problems and only a few of them are feature selection algorithms. Current multilabel classification models are mainly built on a single data representation composed of all the features which are shared by all the class labels. Since each class label might be decided by some specific features of its own, and the problems of classification and feature selection are often addressed independently, in this paper, we propose a novel method which can perform joint feature selection and classification for multilabel learning, named JFSC. Different from many existing methods, JFSC learns both shared features and label-specific features by considering pairwise label correlations, and builds the multilabel classifier on the learned low-dimensional data representations simultaneously. A comparative study with state-of-the-art approaches manifests a competitive performance of our proposed method both in classification and feature selection for multilabel learning.

In the past decades, recommendation systems have provided lots of valuable personalized suggestions for the users to address the problem of information over-loaded. Collaborative Filtering (CF) is one of the most commonly applied and successful recommendation approaches, which refers to using the preferences of groups with similar interests to recommend information to other users. Recently, in addition to the traditional matrix factorization techniques, deep learning methods have been proposed to learn more abstract and higher-level representations for recommendation. However, most previous deep recommendation methods learn the higher-level feature representations of users and items through an identical model structure, which ignores the different characteristics of the user-based and item-based data. In addition, the rating matrix is usually sparse which may result in a significant degradation of recommendation performance. To address these problems, we propose a representation learning method with Collaborative Autoencoder for Personalized Recommendation (CAPR for short). In this method, user-based and item-based feature representations are learned by two different autoencoders for capturing different features of the data. Meanwhile, items’ attributions are combined into the feature representations with semi-autoencoder for alleviating the sparsity problem. Extensive experimental results confirm the effectiveness of our proposed method compared to other state-of-the-art matrix factorization methods and deep recommendation methods.

Nonoverlapping sequential pattern mining (SPM) is a data analysis task, which aims at identifying repetitive sequential patterns with gap constraint in a set of discrete sequences. Nonoverlapping means that any character in the sequence can be rematched by characters at different positions in the pattern, while overlapping means that any character can be reused at the same position, which can be regarded as no condition. Compared with overlapping SPM, nonoverlapping SPM has more strict constraint on the occurrences of pattern, and meets the Apriori property. However, current algorithms mine frequent or closed patterns, resulting in some low-frequency but extremely important patterns being ignored. To tackle this issue, this paper proposes to mine high average utility nonoverlapping sequential patterns (HANP). An efficient algorithm called HANP-Miner is proposed, which involves two key steps: support calculation and candidate pattern reduction. To calculate the support (the occurrence frequency of a pattern), depth-first search and backtracking strategies based on the simplified Nettree structure are adopted, an approach that effectively reduces the time and space complexities of the algorithm. To effectively reduce the number of candidate patterns, a pattern join strategy based on an upper bound on the average utility is proposed. Experiments on biological sequences and real sales sequences are carried out to verify the efficiency of HANP-Miner and the superiority of HANPs. The results demonstrate that HANP-Miner is not only more efficient, but that the HANPs mined in this way are also more valuable than existing frequent patterns. The algorithms and datasets can be downloaded from https://github.com/wuc567/Pattern-Mining/tree/master/HANP-Miner .