Search

When acquiring labels from crowdsourcing platforms, a task may be designed to include multiple labels and the values of each label may belong to a set of various distinct options, which is the so-called multi-class multi-label annotation. To improve the quality of labels, requesters usually let one task be independently completed by a group of heterogeneous crowdsourced workers. Then, the true values of the multiple labels of each task are inferred from these repeated noisy labels. In this paper, we propose two novel probabilistic models MCMLI and MCMLD to address the multi-class multi-label inference problem in crowdsourcing. MCMLI assumes that the labels of each task are mutually independent and MCMLD utilizes a mixture of multiple independently multinoulli distributions to capture the correlation among the labels. Both models can jointly infer multiple true labels of each instance as well as estimate the reliability of crowdsourced workers modeled by a set of confusion matrices with an expectation–maximization algorithm. Experiments with three typical crowdsourcing scenarios and a real-world dataset show that our proposed models significantly outperform existing competitive alternatives. When the labels are strongly correlated, MCMLD substantially outperforms MCMLI. Furthermore, our models can be easily simplified to the one-coin models, which show more advantageous when errors are uniformly distributed, or labels are sparse. [ABSTRACT FROM AUTHOR]

Most recent approaches for Text Simplification (TS) have drawn on insights from machine translation to learn simplification rewrites from the monolingual parallel corpus of complex and simple sentences, yet their effectiveness strongly relies on large amounts of parallel sentences. However, there has been a serious problem haunting TS for decades, that is, the availability of parallel TS corpora is scarce or not fit for the learning task. In this paper, we will focus on one especially useful and challenging problem of unsupervised TS without a single parallel sentence. To the best of our knowledge, we present the first unsupervised text simplification system based on phrase-based machine translation system, which leverages a careful initialization of phrase tables and language models. On the widely used WikiLarge and WikiSmall benchmarks, our system respectively obtains 39.08 and 25.12 SARI points, even outperforms some supervised baselines. [ABSTRACT FROM AUTHOR]

The excellent performance of transfer learning has emerged in the past few years. How to find feature representations which minimize the distance between source and target domains is a crucial problem in transfer learning. Recently, deep learning methods have been proposed to learn higher level and robust representations. However, in traditional methods, label information in source domain is not designed to optimize both feature representations and parameters of the learning model. Additionally, the redundancy of data may incur performance degradation on transfer learning. To address these problems, we propose a novel semi-supervised representation deep learning framework for transfer learning. To obtain this framework, manifold regularization is integrated for the parameter optimization, and the label information is encoded using a softmax regression model in auto-encoders. Meanwhile, whitening layer is introduced to reduce the redundancy of data before auto-encoders. Extensive experiments demonstrate the effectiveness of our proposed framework compared to other competing state-of-the-art baseline methods. [ABSTRACT FROM AUTHOR]

One of the prevalent studies on Topic Detection and Tracking (TDT) is topic evolution. With the emergence of Internet data, there is a clear need for intuitive and adaptive methods to analyze a series of evolutions. Most approaches completely depend on topic models and only focus on whether topics are changed while ignoring the degree of changes, resulting in poor quality topics and insensitivity of changes. In this paper, we propose a framework of topic evolution based on semantic connections which not only indicates the content similarity between documents but also shows the time decay for an adaptive number of topics and rapid responses to the changes of contents. Additionally, semantic connection features can be used to visualize topic evolution, which makes the analyses much easier. For empirical studies, three data sets in real applications are chosen to prove the effectiveness of our method, and the results show that our method has a better performance in reducing redundant topics, avoiding topic suppression, and discerning the vanishment of old topics and the appearance of new topics. [ABSTRACT FROM AUTHOR]

A keyphrase (a multi-word unit) in a document denotes one or multiple keywords capturing a main topic of the underlying document. Finding good keyphrases of a document can quickly summarize knowledge for efficient decision making and benefit domains involving intensive text information. To date, existing keyphrase extraction methods cannot be customized to each specific document, mainly because their patterns used to form paraphrases are too restrictive and may not capture flexible keyword relationships inside the text. In this paper, we propose a sequential pattern mining based document-specific keyphrase extraction method. Our key innovation is to use wildcards (or gap constraints) to help extract sequential patterns, so the flexible wildcard constraints within a pattern can capture semantic relationships between words, and the system will have full flexibility to discover different types of sequential patterns as candidates for keyphrase extraction. To achieve the goal, we regard each single document as a sequential dataset, and propose an efficient algorithm to mine sequential patterns with wildcard and one-off conditions that allows important keyphrases to be captured during the mining process. For each extracted keyphrase candidate, we use some statistical pattern features to characterize it, and further collect all keyphrases from the document to form a training set. A supervised learning classifier is trained to identify keyphrases from a test document. Because our pattern mining and pattern characterization processes are customized to each single document, keyphases extracted from our method are highly specific for each document. Experimental results demonstrate that the proposed sequential pattern mining method outperforms existing pattern mining methods in both runtime performance and completeness. Comparisons on keyphrase benchmark datasets also confirm that the proposed document-specific keyphrase extraction method is effective in improving the quality of extracted keyphrases. [ABSTRACT FROM AUTHOR]

• We propose a novel representation l earning method DSML to achieve a better performance based on deep sparse auto encoder for Multi task Learning. • We propose a new stacked sparse auto encoder for feature reconstruction which can learn higher level and better representations by using the deep learning method and overcome the overfitting problem effectively. • The training of parameters in DSML has lower computational cost than other common deep learning methods. • Experimental studies of compared with seven representation l earning methods show that DSML is superior to traditional and state of the art representation l earning for Multi task Learning. We demonstrate an effective framework to achieve a better performance based on Deep Sparse auto-encoder for Multi-task Learning, called DSML for short. To learn the reconstructed and higher-level features on cross-domain instances for multiple tasks, we combine the labeled and unlabeled data from all tasks to reconstruct the feature representations. Furthermore, we propose the model of Stacked Reconstruction Independence Component Analysis (SRICA for short) for the optimization of feature representations with a large amount of unlabeled data, which can effectively address the redundancy of image data. Our proposed SRICA model is developed from RICA and is based on deep sparse auto-encoder. In addition, we adopt a Semi-Supervised Learning method (SSL for short) based on model parameter regularization to build a unified model for multi-task learning. There are several advantages in our proposed framework as follows: 1) The proposed SRICA makes full use of a large amount of unlabeled data from all tasks. It is used to pursue an optimal sparsity feature representation, which can overcome the over-fitting problem effectively. 2) The deep architecture used in our SRICA model is applied for higher-level and better representation learning, which is designed to train on patches for sphering the input data. 3) Training parameters in our proposed framework has lower computational cost compared to other common deep learning methods such as stacked denoising auto-encoders. Extensive experiments on several real image datasets demonstrate our proposed framework outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

k NN is extensively studied for multilabel classification in the literature. Several k NN-based multilabel learning algorithms have been witnessed during the past years. They usually take k NN as their base classifiers to construct classification models, and then predict the class labels by virtue of Bayesian or majority rules. In this paper, a nearest neighbor selection for multilabel classification is proposed. Specifically, the target labels of new data are predicted with the help of those relevant and reliable data, which explored by the concept of shelly nearest neighbor. For effectiveness, the certainty factor is further adopted to well address the problem of unbalanced and uncertain data. The comparison experiments with eleven popular multilabel classifiers are conducted on ten benchmark data sets. The experimental results show that the performance of the proposed method is competitive and outperforms the popular multilabel classifiers in most cases. [ABSTRACT FROM AUTHOR]

Mining with multilabel data is a popular topic in data mining. When performing classification on multilabel data, existing methods using traditional classifiers, such as support vector machines (SVMs), k-nearest neighbor (k-NN), and decision trees, have relatively poor accuracy and efficiency. Motivated by this, we present a new algorithm adaptation method, namely, a decision tree–based method for multilabel classification in domains with large-scale data sets called decision tree for multi-label classification (DTML). We build an incremental decision tree to reduce the learning time and divide the training data and adopt the k-NN classifier at leaves to improve the classification accuracy. Extensive studies show that our algorithm can efficiently learn from multilabel data while maintaining good performance on example-based evaluation metrics compared to nine state-of-the-art multilabel classification methods. Thus, we draw a conclusion that we provide an efficient and effective incremental algorithm adaptation method for multilabel classification especially in domains with large-scale multilabel data. [ABSTRACT FROM AUTHOR]

Few online classification algorithms based on traditional inductive ensembling, such as online bagging or boosting, focus on handling concept drifting data streams while performing well on noisy data. Motivated by this, an incremental algorithm based on Ensemble Decision Trees for Concept-drifting data streams (EDTC) is proposed in this paper. Three variants of random feature selection are introduced to implement split-tests and two thresholds specified in Hoeffding Bounds inequality are utilized to distinguish concept drifts from noisy data. Extensive studies on synthetic and real streaming databases demonstrate that our algorithm of EDTC performs very well compared to several known online algorithms based on single models and ensemble models. A conclusion is hence drawn that multiple solutions are provided for learning from concept drifting data streams under noise. [ABSTRACT FROM AUTHOR]

It can be easy to collect multiple noisy labels for the same object via Internet-based crowdsourcing systems. Labelers may have bias when labeling, due to lacking expertise, dedication, and personal preference. These cause Imbalanced Multiple Noisy Labeling. In most cases, we have no information about the labeling qualities of labelers and the underlying class distributions. It is important to design agnostic solutions to utilize these noisy labels for supervised learning. We first investigate how imbalanced multiple noisy labeling affects the class distributions of training sets and the performance of classification. Then, an agnostic algorithm Positive LAbel frequency Threshold (PLAT) is proposed to deal with the imbalanced labeling issue. Simulations on eight UCI data sets with different underlying class distributions show that PLAT not only effectively deals with the imbalanced multiple noisy labeling problems that off-the-shelf agnostic methods cannot cope with, but also performs nearly the same as majority voting under the circumstances without imbalance. We also apply PLAT to eight real-world data sets with imbalanced labels collected from Amazon Mechanical Turk, and the experimental results show that PLAT is efficient and better than other ground truth inference algorithms. [ABSTRACT FROM PUBLISHER]

An open question in ensemble-based active learning is how to choose one classifier type, or appropriate combinations of multiple classifier types, to construct ensembles for a given task. While existing approaches typically choose one classifier type, this paper presents a method that trains and adapts multiple instances of multiple classifier types toward an appropriate ensemble during active learning. The method is termed adaptive heterogeneous ensembles (henceforth referred to as AHE). Experimental evaluations show that AHE constructs heterogeneous ensembles that outperform homogeneous ensembles composed of any one of the classifier types, as well as bagging, boosting and the random subspace method with random sampling. We also show in this paper that the advantage of AHE over other methods is increased if (1) the overall size of the ensemble also adapts during learning; and (2) the target data set is composed of more than two class labels. Through analysis we show that the AHE outperforms other methods because it automatically discovers complementary classifiers: for each data instance in the data set, instances of the classifier type best suited for that data point vote together, while instances of the other, inappropriate classifier types disagree, thereby producing a correct overall majority vote. [ABSTRACT FROM AUTHOR]

Abstract: Authorship identification is a task of identifying authors of anonymous texts given examples of the writing of authors. The increasingly large volumes of anonymous texts on the Internet enhance the great yet urgent necessity for authorship identification. It has been applied to more and more practical applications including literary works, intelligence, criminal law, civil law, and computer forensics. In this paper, we propose a semantic association model about voice, word dependency relations, and non-subject stylistic words to represent the writing style of unstructured texts of various authors, design an unsupervised approach to extract stylistic features, and employ principal components analysis and linear discriminant analysis to identify authorship of texts. This paper provides a uniform quantified method to capture syntactic and semantic stylistic characteristics of and between words and phrases, and this approach can solve the problem of the independence of different dimensions to some extent. Experimental results on two English text corpora show that our approach significantly improves the overall performance over authorship identification. [Copyright &y& Elsevier]

With the rapid development of information techniques, the dimensionality of data in many application domains, such as text categorization and bioinformatics, is getting higher and higher. The high-dimensionality data may bring many adverse situations, such as overfitting, poor performance, and low efficiency, to traditional learning algorithms in pattern classification. Feature selection aims at reducing the dimensionality of data and providing discriminative features for pattern learning algorithms. Due to its effectiveness, feature selection is now gaining increasing attentions from a variety of disciplines and currently many efforts have been attempted in this field. In this paper, we propose a new supervised feature selection method to pick important features by using information criteria. Unlike other selection methods, the main characteristic of our method is that it not only takes both maximal relevance to the class labels and minimal redundancy to the selected features into account, but also works like feature clustering in an agglomerative way. To measure the relevance and redundancy of feature exactly, two different information criteria, i.e., mutual information and coefficient of relevance, have been adopted in our method. The performance evaluations on 12 benchmark data sets show that the proposed method can achieve better performance than other popular feature selection methods in most cases. [ABSTRACT FROM AUTHOR]

Abstract: Chloroplasts are crucial organelles of green plants and eukaryotic algae since they conduct photosynthesis. Predicting the subchloroplast location of a protein can provide important insights for understanding its biological functions. The performance of subchloroplast location prediction algorithms often depends on deriving predictive and succinct features from genomic and proteomic data. In this work, a novel weighted Gene Ontology (GO) transfer model is proposed to generate discriminating features from sequence data and GO Categories. This model contains two components. First, we transfer the GO terms of the homologous protein, and then assign the bit-score as weights to GO features. Second, we employ term-selection methods to determine weights for GO terms. This model is capable of improving prediction accuracy due to the tolerance of the noise derived from homolog knowledge transfer. The proposed weighted GO transfer method based on bit-score and a logarithmic transformation of CHI-square (WS-LCHI) performs better than the baseline models, and also outperforms the four off-the-shelf subchloroplast prediction methods. [Copyright &y& Elsevier]

Pattern matching with wildcards and length constraints (PMWL) is a complex problem which has important applications in bioinformatics, network security and information retrieval. Existing algorithms use the traditional left-most strategy when selecting among multiple candidate matching positions, which leads to incomplete final matching results. This paper presents a new data structure CluTree and a new matching algorithm RBCT* based on CluTree. After establishing a cluster of trees with red and black nodes according to a pattern P and a text T, which is called CluTree, our RBCT algorithm uses the sharing degree, correlation degree and mixed information entropy of each node in the CluTree for path selection and dynamic pruning. Our RBCT algorithm traverses the CluTree and finds more occurrences compared to the existing algorithms under the one-off condition in a linear time cost. Theoretical analysis and experimental results show that the RBCT algorithm outperforms other peers in retrieval precision and matching efficiency. [ABSTRACT FROM AUTHOR]

Highlights: [•] We present an attribute-generalization reduct for decision tables with AVTs. [•] We analyze relationships between the attribute reduct and the generalization reduct. [•] We develop a heuristic algorithm AGR-SCE to find the generalization reduct. [•] The generalization reduct can objectively control the generalization process. [•] The generalization reduct can avoid over-generalization or under-generalization. [Copyright &y& Elsevier]

Big Data concern large-volume, complex, growing data sets with multiple, autonomous sources. With the fast development of networking, data storage, and the data collection capacity, Big Data are now rapidly expanding in all science and engineering domains, including physical, biological and biomedical sciences. This paper presents a HACE theorem that characterizes the features of the Big Data revolution, and proposes a Big Data processing model, from the data mining perspective. This data-driven model involves demand-driven aggregation of information sources, mining and analysis, user interest modeling, and security and privacy considerations. We analyze the challenging issues in the data-driven model and also in the Big Data revolution. [ABSTRACT FROM AUTHOR]

Information on the World Wide Web is congested with large amounts of news contents. Recommending, filtering, and summarization of Web news have become hot topics of research in Web intelligence, aiming to find interesting news for users and give concise content for reading. This paper presents our research on developing the Personalized News Filtering and Summarization system (PNFS). An embedded learning component of PNFS induces a user interest model and recommends personalized news. Two Web news recommendation methods are proposed to keep tracking news and find topic interesting news for users. A keyword knowledge base is maintained and provides real-time updates to reflect the news topic information and the user's interest preferences. The non-news content irrelevant to the news Web page is filtered out. A keyword extraction method based on lexical chains is proposed that uses the semantic similarity and the relatedness degree to represent the semantic relations between words. Word sense disambiguation is also performed in the built lexical chains. Experiments on Web news pages and journal articles show that the proposed keyword extraction method is effective. An example run of our PNFS system demonstrates the superiority of this Web intelligence system. [ABSTRACT FROM AUTHOR]

In the past decades, recommendation systems have provided lots of valuable personalized suggestions for the users to address the problem of information over-loaded. Collaborative Filtering (CF) is one of the most commonly applied and successful recommendation approaches, which refers to using the preferences of groups with similar interests to recommend information to other users. Recently, in addition to the traditional matrix factorization techniques, deep learning methods have been proposed to learn more abstract and higher-level representations for recommendation. However, most previous deep recommendation methods learn the higher-level feature representations of users and items through an identical model structure, which ignores the different characteristics of the user-based and item-based data. In addition, the rating matrix is usually sparse which may result in a significant degradation of recommendation performance. To address these problems, we propose a representation learning method with Collaborative Autoencoder for Personalized Recommendation (CAPR for short). In this method, user-based and item-based feature representations are learned by two different autoencoders for capturing different features of the data. Meanwhile, items' attributions are combined into the feature representations with semi-autoencoder for alleviating the sparsity problem. Extensive experimental results confirm the effectiveness of our proposed method compared to other state-of-the-art matrix factorization methods and deep recommendation methods. • Two different autoencoders are used to capture characteristics for users and items. • Manifold regularization is integrated into autoencoder for user's features learning. • The comprehensive experiments evaluate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

We propose a new online feature selection framework for applications with streaming features where the knowledge of the full feature space is unknown in advance. We define streaming features as features that flow in one by one over time whereas the number of training examples remains fixed. This is in contrast with traditional online learning methods that only deal with sequentially added observations, with little attention being paid to streaming features. The critical challenges for Online Streaming Feature Selection (OSFS) include 1) the continuous growth of feature volumes over time, 2) a large feature space, possibly of unknown or infinite size, and 3) the unavailability of the entire feature set before learning starts. In the paper, we present a novel Online Streaming Feature Selection method to select strongly relevant and nonredundant features on the fly. An efficient Fast-OSFS algorithm is proposed to improve feature selection performance. The proposed algorithms are evaluated extensively on high-dimensional datasets and also with a real-world case study on impact crater detection. Experimental results demonstrate that the algorithms achieve better compactness and higher prediction accuracy than existing streaming feature selection algorithms. [ABSTRACT FROM PUBLISHER]

Abstract: Identity disclosure is one of the most serious privacy concerns in many data mining applications. A well-known privacy model for protecting identity disclosure is k-anonymity. The main goal of anonymizing classification data is to protect individual privacy while maintaining the utility of the data in building classification models. In this paper, we present an approach based on rough sets for measuring the data quality and guiding the process of anonymization operations. First, we make use of the attribute reduction theory of rough sets and introduce the conditional entropy to measure the classification data quality of anonymized datasets. Then, we extend conditional entropy under single-level granulation to hierarchical conditional entropy under multi-level granulation, and study its properties by dynamically coarsening and refining attribute values. Guided by these properties, we develop an efficient search metric and present a novel algorithm for achieving k-anonymity, Hierarchical Conditional Entropy-based Top-Down Refinement (HCE-TDR), which combines rough set theory and attribute value taxonomies. Theoretical analysis and experiments on real world datasets show that our algorithm is efficient and improves data utility. [Copyright &y& Elsevier]

Causal discovery is highly desirable in science and technology. In this paper, we study a new research problem of discovery of causal relationships in the context of streaming features, where the features steam in one by one. With a Bayesian network to represent causal relationships, we propose a novel algorithm called causal discovery from streaming features (CDFSF) which consists of a two-phase scheme. In the first phase, CDFSF dynamically discovers causal relationships between each feature seen so far with an arriving feature, while in the second phase CDFSF removes the false positives of each arrived feature from its current set of direct causes and effects. To improve the efficiency of CDFSF, using the symmetry properties between parents (causes) and children (effects) in a faithful Bayesian network, we present a variant of CDFSF, S-CDFSF. Experimental results validate our algorithms in comparison with the existing algorithms of causal relationship discovery. [ABSTRACT FROM PUBLISHER]

Abstract: Most existing work on classification of data streams assumes that all streaming data are labeled and the class labels are immediately available. However, in real-world applications, such as credit fraud and intrusion detection, this assumption is not always valid. Thus, it is a challenge to learn from concept drifting data streams with unlabeled data. With this motivation, we propose a Semi-supervised classification algorithm for data streams with concept drifts and UNlabeled data (SUN) in this paper. In SUN, a clustering algorithm is developed from k-Modes and implemented to produce concept clusters at leaves in an incremental decision tree. In terms of deviations between history concept clusters and new ones, potential concept drifts are distinguished from noise. Extensive studies on both synthetic and real-world data demonstrate that SUN performs well compared to several state-of-the-art online supervised and semi-supervised algorithms, even when there are more than 90% unlabeled data. A conclusion is hence drawn that SUN provides a promising framework for tackling concept drifting data streams with unlabeled data. [Copyright &y& Elsevier]

Abstract: In face of the clustering problem, many clustering methods usually require the designer to provide the number of clusters as input. Unfortunately, the designer has no idea, in general, about this information beforehand. In this article, we develop a genetic algorithm based clustering method called automatic genetic clustering for unknown K (AGCUK). In the AGCUK algorithm, noising selection and division–absorption mutation are designed to keep a balance between selection pressure and population diversity. In addition, the Davies–Bouldin index is employed to measure the validity of clusters. Experimental results on artificial and real-life data sets are given to illustrate the effectiveness of the AGCUK algorithm in automatically evolving the number of clusters and providing the clustering partition. [Copyright &y& Elsevier]

Abstract: Influence maximization in a social network is to target a given number of nodes in the network such that the expected number of activated nodes from these nodes is maximized. A social network usually exhibits some degree of modularity. Previous research efforts that made use of this topological property are restricted to random networks with two communities. In this paper, we firstly transform the influence maximization problem in a modular social network to an optimal resource allocation problem. We assume that the communities of the social network are disconnected. We then propose a recursive relation for finding such an optimal allocation. We prove that finding an optimal allocation in a modular social network is NP-hard and propose a new dynamic programming algorithm to solve the problem. We name our new algorithm OASNET (Optimal Allocation in a Social NETwork). We compare OASNET with the high degree heuristics, the single degree discount heuristics, and the degree discount heuristics on three real world datasets. Experimental results show that OASNET outperforms comparison heuristics significantly on the independent cascade model when the diffusion probability is greater than a certain threshold. [Copyright &y& Elsevier]

A key goal of the Semantic Web is to shift social interaction patterns from a producer-centric paradigm to a consumer-centric one. Treating customers as the most valuable assets and making the business models work better for them are at the core of building successful consumer-centric business models. It follows that customizing business processes constitutes a major concern in the realm of a knowledge-pull-based human semantic Web. This paper conceptualizes the customization of service-based business processes leveraging the existing knowledge of Web services and business processes. We represent this conceptualization as a new Extensible Markup Language (XML) markup language Web Ontology Language-Business Process Customization (OWL-BPC), based on the de facto semantic markup language for Web-based information [Web Ontology Language (OWL)]. Furthermore, we report a framework, built on OWL-BPC, for customizing service-based business processes, which supports customization detection and enactment. Customization detection is enabled by a business-goal analysis, and customization enactment is enabled via event–condition–action rule inference. Our solution and framework have the following capabilities in dealing with inconsistencies and misalignments in business process interactions: 1) resolve semantic mismatch of process parameters; 2) handle behavioral mismatches which may or may not be compatible; and 3) process misaligned rendezvous requirements. Such capabilities are applicable to business processes with heterogeneous domain ontology. We present an architectural description of the implementation and a walk-through of an example of solving a customization problem as a validation of the proposed approach. [ABSTRACT FROM AUTHOR]

Detecting concept drifts and reducing the impact from the noise in real applications of data streams are challenging but valuable for inductive learning. It is especially a challenge in a light demand on the overheads of time and space. However, though a great number of inductive learning algorithms based on ensemble classification models have been proposed for handling concept drifting data streams, little attention has been focused on the detection of the diversity of concept drifts and the influence from noise in data streams simultaneously. Motivated by this, we present a new light-weighted inductive algorithm for concept drifting detection in virtue of an ensemble model of random decision trees (named CDRDT) to distinguish various types of concept drifts from noisy data streams in this article. We use variably small data chunks to generate random decision trees incrementally. Meanwhile, we introduce the inequality of Hoeffding bounds and the principle of statistical quality control to detect the different types of concept drifts and noise. Extensive studies on synthetic and real streaming data demonstrate that CDRDT could effectively and efficiently detect concept drifts from the noisy streaming data. Therefore, our algorithm provides a feasible reference framework of classification for concept drifting data streams with noise. [ABSTRACT FROM AUTHOR]

Abstract: Nonnegative Matrix Factorization (NMF), a parts-based representation using two small factor matrices to approximate an input data matrix, has been widely used in data mining, pattern recognition and signal processing. Orthogonal NMF which imposes orthogonality constraints on the factor matrices can improve clustering performance. However, the existing orthogonal NMF algorithms are either computationally expensive or have to incorporate prior information to achieve orthogonality. In our research, we propose an algorithm called Nonnegative Matrix Factorization on Orthogonal Subspace (NMFOS), in which the generation of orthogonal factor matrices is part of objective function minimization. Thus, orthogonality is achieved without resorting to additional constraints, and the computational complexity is decreased. We develop two algorithms based on the Euclidean distance metric and the generalized Kullback–Leibler divergence, respectively. Experiments on 10 document datasets show that NMFOS improves clustering accuracy. On a facial image database, NMFOS achieves a better parts-based representation with a significant reduction in computational complexity. [Copyright &y& Elsevier]

Abstract: Learning often occurs through comparing. In classification learning, in order to compare data groups, most existing methods compare either raw instances or learned classification rules against each other. This paper takes a different approach, namely conceptual equivalence, that is, groups are equivalent if their underlying concepts are equivalent while their instance spaces do not necessarily overlap and their rule sets do not necessarily present the same appearance. A new methodology of comparing is proposed that learns a representation of each group’s underlying concept and respectively cross-exams one group’s instances by the other group’s concept representation. The innovation is fivefold. First, it is able to quantify the degree of conceptual equivalence between two groups. Second, it is able to retrace the source of discrepancy at two levels: an abstract level of underlying concepts and a specific level of instances. Third, it applies to numeric data as well as categorical data. Fourth, it circumvents direct comparisons between (possibly a large number of) rules that demand substantial effort. Fifth, it reduces dependency on the accuracy of employed classification algorithms. Empirical evidence suggests that this new methodology is effective and yet simple to use in scenarios such as noise cleansing and concept-change learning. [Copyright &y& Elsevier]

Many large organizations have multiple databases distributed in different branches, and therefore multi-database mining is an important task for data mining. To reduce the search cost in the data from all databases, we need to identify which databases are most likely relevant to a data mining application. This is referred to as database selection. For real-world applications, database selection has to be carried out multiple times to identify relevant databases that meet different applications. In particular, a mining task may be without reference to any specific application. In this paper, we present an efficient approach for classifying multiple databases based on their similarity between each other. Our approach is application-independent. [Copyright &y& Elsevier]

Dealing with very large databases is one of the defining challenges in data mining research and development. Some databases are simply too large (e.g., with terabytes of data) to be processed at one time. For efficiency and space reasons, partitioning them into subsets for processing is necessary. However, since the number of itemsets in each partitioned data subset can be a combinatorial amount and each of them may be a large itemset in the original database, data mining results from these subsets can be very large in size. Therefore, the key to data partitioning is how to aggregate the results from these subsets. It is not realistic to keep all results from each subset, because the rules from one subset need to be verified for usefulness in other subsets. This article presents a model of aggregating association rules from different data subsets by weighting. In particular, the aggregation efficiency is enhanced by rule selection. [ABSTRACT FROM AUTHOR]

Presents a heuristic, attribute-based, noise-tolerant data mining program, HCV (Version 2.0), based on a developed extension matrix approach. Methods used; Issues to be addressed when integrating machine learning techniques into database systems to build practical data mining systems; Conclusions of the study; Statistical information supporting this research.

Presents an introduction to the articles that have been published in the June 2005 issue of the journal "Transactions of Knowledge and Data Engineering."

Approximate multi-pattern matching is an important issue that is widely and frequently utilized, when the pattern contains variable-length wildcards. In this paper, two suffix array-based algorithms have been proposed to solve this problem. Suffix array is an efficient data structure for exact string matching in existing studies, as well as for approximate pattern matching and multi-pattern matching. An algorithm called MMSA-S is for the short exact characters in a pattern by dynamic programming, while another algorithm called MMSA-L deals with the long exact characters by the edit distance method. Experimental results of Pizza & Chili corpus demonstrate that these two newly proposed algorithms, in most cases, are more time-efficient than the state-of-the-art comparison algorithms. [ABSTRACT FROM AUTHOR]

Graph pattern matching has been widespread used for protein structure analysis, expert finding and social group selection, ect. Recently, the study of graph pattern matching using the abundant attribute information of vertices and edges as constraint conditions has attracted the attention of scholars, and multi-constrained simulation has been proposed to address the problem in contextual social networks. Actually, multi-constrained graph pattern matching is an NP-complete problem and the fuzziness of constraint variables may exist in many applications. In this paper, we introduce a multi-fuzzy-constrained graph pattern matching problem in big graph data, and propose an efficient first-k algorithm Fuzzy-ETOF-K for solving it. Specifically, exploration-based method based on edge topology is adopted to improve the efficiency of edge connection, and breadth-first bounded search is used for edge matching instead of shortest path query between two nodes to improve the efficiency of edge matching. The results of our experiments conducted on three datasets of real social networks illustrate that our proposed algorithm Fuzzy-ETOF-K significantly outperforms existing approaches in efficiency and the introduction of fuzzy constraints makes our proposed algorithm more efficient and effective. [ABSTRACT FROM AUTHOR]

Multi-pattern matching with variable-length wildcards is an interesting and important problem in bioinformatics, information retrieval and other domains. Most of the previously developed multi-pattern matching methods, such as famous Aho-Corasick and Wu-Manber algorithms, aimed to solve some classical string matching problems. However, these algorithms are not efficient for patterns with flexible wildcards or do-not-care characters. In this paper, we propose two efficient algorithms for multi-pattern matching with variable-length wildcards based on suffix tree, called MMST-L and MMST-S, according to the length of exact characters in a pattern. Experimental results show that the two MMST algorithms, in most cases, outperform other various versions of comparing algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes an approach KeyRank to extract proper keyphrases from a document in English. It first searches all keyphrase candidates from the document, and then ranks them for selecting top- N ones as final keyphrases. Existing studies show that extracting a complete keyphrase candidate set that includes semantic relations in context, and evaluating the effectiveness of each candidate are crucial to extract high quality keyphrases from documents. Based on that words do not repeatedly appear in an effective keyphrase in English, a novel keyphrase candidate search algorithm using sequential pattern mining with gap constraints (called KCSP) is proposed to extract keyphrase candidates for KeyRank. And then an effectiveness evaluation measure pattern frequency with entropy (called PF-H) is proposed for KeyRank to rank these keyphrase candidates. Our experimental results show that KeyRank has better performance. Its first component KCSP is much more efficient than a closely related approach SPMW, and its second component PF-H is an effective evaluation mechanism for ranking keyphrase candidates. 1 1 Our two-page extended abstract is published in AAAI 2017 (Wang, Sheng, & Wu, 2017). [ABSTRACT FROM AUTHOR]

Adverse drug-drug interaction (ADDI) is a significant life-threatening issue, posing a leading cause of hospitalizations and deaths in healthcare systems. This paper proposes a unified Multi-Attribute Discriminative Representation Learning (MADRL) model for ADDI prediction. Unlike the existing works that equally treat features of each attribute without discrimination and do not consider the underlying relationship among drugs, we first develop a regularized optimization problem based on CUR matrix decomposition for joint representative drug and discriminative feature selection such that the selected drugs and features can well approximate the original feature spaces and the critical factors discriminative to ADDIs can be properly explored. Different from the existing models that ignore the consistent and unique properties among attributes, a Generative Adversarial Network (GAN) framework is then designed to capture the inter-attribute shared and intra-attribute specific representations of adverse drug pairs for exploiting their consensus and complementary information in ADDI prediction. Meanwhile, MADRL is compatible with any kind of attributes and capable of exploring their respective effects on ADDI prediction. An iterative algorithm based on the alternating direction method of multipliers is developed for optimization. Experiments on publicly available dataset demonstrate the effectiveness of MADRL when compared with eleven baselines and its six variants. [ABSTRACT FROM AUTHOR]

Adverse drug-drug interaction (ADDI) is a significant life-threatening issue, posing a leading cause of hospitalizations and deaths in healthcare systems. This paper proposes a unified Multi-Attribute Discriminative Representation Learning (MADRL) model for ADDI prediction. Unlike the existing works that equally treat features of each attribute without discrimination and do not consider the underlying relationship among drugs, we first develop a regularized optimization problem based on CUR matrix decomposition for joint representative drug and discriminative feature selection such that the selected drugs and features can well approximate the original feature spaces and the critical factors discriminative to ADDIs can be properly explored. Different from the existing models that ignore the consistent and unique properties among attributes, a Generative Adversarial Network (GAN) framework is then designed to capture the inter-attribute shared and intra-attribute specific representations of adverse drug pairs for exploiting their consensus and complementary information in ADDI prediction. Meanwhile, MADRL is compatible with any kind of attributes and capable of exploring their respective effects on ADDI prediction. An iterative algorithm based on the alternating direction method of multipliers is developed for optimization. Experiments on publicly available dataset demonstrate the effectiveness of MADRL when compared with eleven baselines and its six variants. [ABSTRACT FROM AUTHOR]

The article presents a message by the article author who is the new editor-in-chief of the journal "IEEE Transactions on Knowledge and Data Engineering" (TKDE) as the journal has completed its 16 years of publication. He has regarded the hard work and outstanding efforts of the four previous editors-in-chief, associate editors, reviewers, authors, and support staff from the IEEE Computer Society Publications Office. He said that TKDE is commonly recognized as the premier journal in the field of knowledge and data engineering, including database data mining, and knowledge engineering. The activities planned by the new EIC include revising TKDE topic areas to cover new emerging areas and technological advancements, and organizing special issues on promising topics when the need arises; inviting well-recognized researchers in TKDE areas to serve on the Editorial Board and minimizing the delay in publication by reducing the backlog of accepted papers. He has invited all authors and readers to share their comments and advice on how to further enhance the journal's value to the wider research community in knowledge and data engineering.

Visual classification has attracted considerable research interests in the past decades. In this paper, a novel $\ell _1$ -hypergraph model for visual classification is proposed. Hypergraph learning, as a natural extension of graph model, has been widely used in many machine learning tasks. In previous work, hypergraph is usually constructed by attribute-based or neighborhood-based methods. That is, a hyperedge is generated by connecting a set of samples sharing a same feature attribute or in a neighborhood. However, these methods are unable to explore feature space globally or sensitive to noises. To address these problems, we propose a novel hypergraph construction approach that leverages sparse representation to generate hyperedges and learns the relationship among hyperedges and their vertices. First, for each sample, a hyperedge is generated by regarding it as the centroid and linking it as well as its nearest neighbors. Then, the sparse representation method is applied to represent the centroid vertex by other vertices within the same hyperedge. The vertices with zero coefficients are removed from the hyperedge. Finally, the representation coefficients are used to define the incidence relation between the hyperedge and the vertices. In our approach, we also optimize the hyperedge weights to modulate the effects of different hyperedges. We leverage the prior knowledge on the hyperedges so that the hyperedges sharing more vertices can have closer weights, where a graph Laplacian is used to regularize the optimization of the weights. Our approach is named $\ell _1$ -hypergraph since the $\ell _1$ sparse representation is employed in the hypergraph construction process. The method is evaluated on various visual classification tasks, and it demonstrates promising performance. [ABSTRACT FROM AUTHOR]

Over the past few decades, short text classification has emerged as a critical downstream task in natural language processing (NLP). One crucial classification research issue is how to advance semantic understanding considering the short length, feature sparsity, and high ambiguity in short texts. Recently, prompt-tuning has been proposed to insert a template into the input and convert text classification tasks into equivalent cloze-style tasks. However, among most of the previous approaches, either the crafted template methods are time-consuming and labor-intensive, or automatic prompt generation methods cannot achieve satisfied performance. In this paper, we introduce a novel approach called Soft Knowledgeable Prompt-tuning for short text classification. Our method considers both the template generation and classification performance to construct prompts for label prediction. We employ five different strategies to expand the label words space for modifying soft prompts, and the integration of these strategies is used as the final verbalizer. Despite being automatic, experimental results show that our method achieved more desirable performance even than the crafted template methods, outperforming the state-of-the-art by more than 14 Accuracy points on four well-known benchmarks. • Both the automatic prompt generation and text classification performance are achieved. • 5 different strategies are used to construct verbalizer effectively and efficiently. • The comprehensive experiments evaluate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

Machine learning can provide deep insights into data, allowing machines to make high-quality predictions and having been widely used in real-world applications, such as text mining, visual classification, and recommender systems. However, most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. This issue calls for the need of Large-scale Machine Learning (LML), which aims to learn patterns from big data with comparable performance efficiently. In this paper, we offer a systematic survey on existing LML methods to provide a blueprint for the future developments of this area. We first divide these LML methods according to the ways of improving the scalability: 1) model simplification on computational complexities, 2) optimization approximation on computational efficiency, and 3) computation parallelism on computational capabilities. Then we categorize the methods in each perspective according to their targeted scenarios and introduce representative methods in line with intrinsic strategies. Lastly, we analyze their limitations and discuss potential directions as well as open issues that are promising to address in the future. [ABSTRACT FROM AUTHOR]

How to accurately predict unknown quality-of-service (QoS) data based on observed ones is a hot yet thorny issue in Web service-related applications. Recently, a latent factor (LF) model has shown its efficiency in addressing this issue owing to its high accuracy and scalability. An LF model can be improved by identifying user and service neighborhoods based on user and service geographical information. However, such information can be difficult to acquire in most applications with the considerations of information security, identity privacy, and commercial interests in a real system. Besides, the existing LF model-based QoS predictors mostly ignore the reliability of given QoS data where noises commonly exist to cause accuracy loss. To address the above issues, this paper proposes a data-characteristic-aware latent factor (DCALF) model to implement highly accurate QoS predictions, where ‘data-characteristic-aware’ indicates that it can appropriately implement QoS prediction according to the characteristics of given QoS data. Its main idea is two-fold: a) it detects the neighborhoods and noises of users and services based on the dense LFs extracted from the original sparse QoS data, b) it incorporates a density peaks-based clustering method into its modeling process for achieving the simultaneous detections of both neighborhoods and noises of QoS data. With such designs, it precisely represents the given QoS data in spite of their sparsity, thereby achieving highly accurate predictions for unknown ones. Experimental results on two QoS datasets generated by real-world Web services demonstrate that the proposed DCALF model outperforms state-of-the-art QoS predictors, making it highly competitive in addressing the issue of Web service selection and recommendation. [ABSTRACT FROM AUTHOR]

• A heading error analysis about variable attitude angles of a compass is presented. • A heading error modeling method for the attitude-change of a compass is developed. • GRU is used to model the heading error resulted in the attitude-change of a compass. The polarization compass used for unmanned aerial vehicle (UAV) navigation is usually hypothesized to be arranged horizontally in conventional heading measurement, which leads to a noticeable heading error due to the inevitable tilts called the pitch angle and the roll angle during UAV flight process. In addition, we found that the coupling of the angle between the solar meridian and the body axis of a carrier (A-SMBA) and tilted-angles will produce more remarkable heading errors. Consequently, we first introduce a comprehensive analysis of heading error in terms of variable attitude angles of the compass including the A-SMBA, the pitch angle and the roll angle. A novel heading error modeling and compensation for attitude-changed of the polarization compass by gated recurrent unit (GRU) neural network is developed subsequently. The experimental results demonstrate the proposed heading error modeling and compensation method performs best compared to state-of-the-art algorithms in predicting the UAV orientation. [ABSTRACT FROM AUTHOR]

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm. [ABSTRACT FROM AUTHOR]