Search

This paper presents an unsupervised feature selection method for multi-dimensional histogram-valued data. We define a multi-role measure, called the compactness, based on the concept size of given objects and/or clusters described using a fixed number of equal probability bin-rectangles. In each step of clustering, we agglomerate objects and/or clusters so as to minimize the compactness for the generated cluster. This means that the compactness plays the role of a similarity measure between objects and/or clusters to be merged. Minimizing the compactness is equivalent to maximizing the dis-similarity of the generated cluster, i.e., concept, against the whole concept in each step. In this sense, the compactness plays the role of cluster quality. We also show that the average compactness of each feature with respect to objects and/or clusters in several clustering steps is useful as a feature effectiveness criterion. Features having small average compactness are mutually covariate and are able to detect a geometrically thin structure embedded in the given multi-dimensional histogram-valued data. We obtain thorough understandings of the given data via visualization using dendrograms and scatter diagrams with respect to the selected informative features. We illustrate the effectiveness of the proposed method by using an artificial data set and real histogram-valued data sets.

Symbolic data is aggregated from bigger traditional datasets in order to hide entry specific details and to enable analysing large amounts of data, like big data, which would otherwise not be possible. Symbolic data may appear in many different but complex forms like intervals and histograms. Identifying patterns and finding similarities between objects is one of the most fundamental tasks of data mining. In order to accurately cluster these sophisticated data types, usual methods are not enough. Throughout the years different approaches have been proposed but they mainly concentrate on the “macroscopic” similarities between objects. Distributional data, for example symbolic data, has been aggregated from sets of large data and thus even the smallest microscopic differences and similarities become extremely important. In this paper a method is proposed for clustering distributional data based on these microscopic similarities by using quantile values. Having multiple points for comparison enables to identify similarities in small sections of distribution while producing more adequate hierarchical concepts. Proposed algorithm, called microscopic hierarchical conceptual clustering, has a monotone property and has been found to produce more adequate conceptual clusters during experimentation. Furthermore, thanks to the usage of quantiles, this algorithm allows us to compare different types of symbolic data easily without any additional complexity.

We propose AutoName, an unsupervised framework that extracts a name for a set of query entities from a large-scale text corpus. Entity-set naming is useful in many tasks related to natural language processing and information retrieval such as session-based and conversational information seeking. Previous studies mainly extract set names from knowledge bases which provide highly reliable entity relations, but suffer from limited coverage of entities and set names that represent broad semantic classes. To address these problems, AutoName generates hypernym-anchored candidate phrases via probing a pre-trained language model and the entities' context in documents. Phrases are then clustered to identify ones that describe common concepts among query entities. Finally, AutoName ranks refined phrases based on the co-occurrences of their words with query entities and the conceptual integrity of their respective clusters. We built a new benchmark dataset for this task, consisting of 130 entity sets with name labels. Experimental results show that AutoName generates coherent and meaningful set names and significantly outperforms all baselines.

One of the most used techniques among many others in the data mining field is the clustering. The aim of this technique is to synthetize and summarize huge amounts of data by splitting it into small and homogenous clusters such that the data (observations) inside the same cluster are more similar to each other than to the observations inside the other clusters. This definition assumes that there exists a well defined clustering quality measure that quantifies how much homogeneous are the obtained clusters. The aim of this chapter is to expose an original approach to merge different partitions, related to the same data set, which are obtained either by applying different clustering techniques either by the same clustering technique with different parameters. Fusing partitions has been broadly studied and has been given several names, depending on different scientific fields, like machine learning or bioinformatics (Dudoit & Fridlyand, 2003; Kim & Lee, 2007; Monti et al., 2003). Among these names we can quote: consensus clustering, clustering aggregation, clustering combination, fusion of clustering, ...etc. Several studies (Frossyniotis et al., 2002; Minaei-Bidgoli et al., 2004; Strehl & Ghosh, 2002; Topchy et al., 2004; 2005) have pioneered clustering data sets as a new branch of the conventional clustering methodology. In (Topchy et al., 2004) the authors propose a probabilistic formalism of clustering concensus using a finite mixture of multinomial distributions in a space of clustering. The approach proposed in (Frossyniotis et al., 2002) is designed for combining runs of clustering algorithms with the same number of clusters. In (Strehl & Ghosh, 2002) the authors proposed combiners based on a hyper-graph model to solve the cluster fusion problem. The authors discuss two manners of consensus clustering: (1) Feature Distributed Clustering (FDC): a set of clustering are obtained from partial view of variables using all observations (2) Object-Distributed Clustering (ODC): with this technique the ensemble clustering has limited to subset of observation with access to all variables. The 3

In the second decade of the twenty-first century, supply chain management (SCM) practices have gained an increasingly strategic character. The focus has shifted from an operational perspective to a strategic one, and SCM is repeatedly cited in the literature as a strategic management tool with a profound effect on the survival of the organizations. SCM domain is well-supported to be closely linked with a number of strategic management views and theories. This study investigates the main strategic management views and theories relevant to SCM and their interactions. It reveals how they complementarily explain supply chain management characteristics of being customer-driven, process-oriented, global, strategically coupled, complex, dynamic, adaptive, multi-agent, IT-intensive, value creating, collaborative and networked. Based on a comprehensive literature taxonomy, a mapping of the main supply chain characteristics to these management views and theories is provided. A conceptual clustering of different management perspectives is developed, and their interactions are analyzed.

Clustering is the method employed to group Web pages containing related information into clusters, which facilitates the allocation of relevant information. Clustering performance is mostly dependent on the text features' characteristics. The Arabic language has a complex morphology and is highly inflected. Thus, selecting appropriate features affects clustering performance positively. Many studies have addressed the clustering problem in Web pages with Arabic content. There are three main challenges in applying text clustering to Arabic Web page content. The first challenge concerns difficulty with identifying significant term features to represent original content by considering the hidden knowledge. The second challenge is related to reducing data dimensionality without losing essential information. The third challenge regards how to design a suitable model for clustering Arabic text that is capable of improving clustering performance. This paper presents an overview of existing Arabic Web page clustering methods, with the goals of clarifying existing problems and examining feature selection and reduction techniques for solving clustering difficulties. In line with the objectives and scope of this study, the present research is a joint effort to improve feature selection and vectorization frameworks in order to enhance current text analysis techniques that can be applied to Arabic Web pages. Keywords: Feature selection, Feature reduction, K-means, Review, Text clustering, ARABIC Web page

Research on the nature of episodic memory organization has suggested that the dominant organizing principle among episodic memories is either conceptual organization, where memories cluster around the same content (e.g., memories about the people), or a type of temporal organization known as general events, where memories cluster around extended events (e.g., a holiday in London), repeated events (e.g., Sunday walks in the park), or causally related events (i.e., where one event leads to another). Because these opposing conclusions have been derived separately from involuntary or voluntary measures of autobiographical memory, Mace, Clevinger, and Martin (2010) hypothesized that involuntary retrieval processes and voluntary retrieval processes may be accessing episodic memories differentially, such that one produces high frequencies of conceptually clustered memories, whereas the other produces high frequencies of general event clustered memories. This study failed to find support for this hypothesis. In 2 experiments, 3 different types of voluntary retrieval tasks found that conceptually clustered memories were substantially more frequent than general event clustered memories. We argue that these results support the view that episodic memories are organized mainly conceptually.

This paper presents an unsupervised feature selection method for multi-dimensional histogram-valued data. We define a multi-role measure, called the compactness, based on the concept size of given objects and/or clusters described by a fixed number of equal probability bin-rectangles. In each step of clustering, we agglomerate objects and/or clusters so as to minimize the compactness for the generated cluster. This means that the compactness plays the role of a similarity measure between objects and/or clusters to be merged. To minimize the compactness is equivalent to maximize the dis-similarity of the generated cluster, i.e., concept, against the whole concept in each step. In this sense, the compactness plays the role of cluster quality. We also show that the average compactness of each feature with respect to objects and/or clusters in several clustering steps is useful as feature effectiveness criterion. Features having small average compactness are mutually covariate, and are able to detect geometrically thin structure embedded in the given multi-dimensional histogram-valued data. We obtain thorough understandings of the given data by the visualization using dendrograms and scatter diagrams with respect to the selected informative features. We illustrate the effectiveness of the proposed method by using an artificial data set and real histogram-valued data sets.

Knowledge graph embeddings that generate vector space representations of knowledge graph triples, have gained considerable popularity in past years. Several embedding models have been proposed that achieve state-of-the-art performance for the task of triple completion in knowledge graphs. Relying on the presumed semantic capabilities of the learned embeddings, they have been leveraged for various other tasks such as entity typing, rule mining and conceptual clustering. However, a critical analysis of the utility as well as limitations of these embeddings for semantic representation of the underlying entities and relations has not been performed by previous work.

Concepts have been adopted in concept-cognitive learning (CCL) and conceptual clustering for concept classification and concept discovery. However, the standard CCL algorithms are incapable of tackling continuous data directly, and some standard conceptual clustering methods mainly focus on the attribute information, ignoring the object information that is also important to improve clustering analysis and concept classification ability. Therefore, in this article, we present a novel concept learning method, called the fuzzy-based concept learning model (FCLM), to address these two issues by exploiting concept hierarchical relations in concept lattices. More specifically, we first show some new related notions for FCLM based on a regular fuzzy formal decision context; among these notions, the object-oriented and attribute-oriented fuzzy concept similarities are used to achieve the concept similarity measure in concept lattices. Moreover, a novel fuzzy concept learning framework is designed, and its corresponding learning algorithms are developed. Finally, we conduct some experiments on various real-world datasets to demonstrate that the proposed method can achieve the state-of-the-art classification performance among similarity-based learning methods. In addition, we further verify the effectiveness of our method in concept discovery on the MNIST dataset.

We introduce the exactCover global constraint dedicated to the exact cover problem, the goal of which is to select subsets such that each element of a given set belongs to exactly one selected subset. This NP-complete problem occurs in many applications, and we more particularly focus on a conceptual clustering application. We introduce three propagation algorithms for exactCover, called Basic, DL, and DL+: Basic ensures the same level of consistency as arc consistency on a classical decomposition of exactCover into binary constraints, without using any specific data structure; DL ensures the same level of consistency as Basic but uses Dancing Links to efficiently maintain the relation between elements and subsets; and DL+ is a stronger propagator which exploits an extra property to filter more values than DL. We also consider the case where the number of selected subsets is constrained to be equal to a given integer variable k, and we show that this may be achieved either by combining exactCover with existing constraints, or by designing a specific propagator that integrates algorithms designed for the NValues constraint. These different propagators are experimentally evaluated on conceptual clustering problems, and they are compared with state-of-the-art declarative approaches. In particular, we show that our global constraint is competitive with recent ILP and CP models for mono-criterion problems, and it has better scale-up properties for multi-criteria problems.

Feature selection is an important topic in data mining, especially for high dimensional dataset. Feature selection is a process commonly used in machine learning, wherein subsets of the features available from the data are selected for application of learning algorithm. The best subset contains the least number of dimensions that most contribute to accuracy. Feature selection methods can be decomposed into three main classes, one is filter method, another one is wrapper method and third one is embedded method. This chapter presents an empirical comparison of feature selection methods and its algorithm. In view of the substantial number of existing feature selection algorithms, the need arises to count on criteria that enable to adequately decide which algorithm to use in certain situation. This chapter reviews several fundamental algorithms found in the literature and assess their performance in a controlled scenario.

Due to the weak academic advising systems in most schools, students need assistance to explore universities/colleges and study fields that match their interests and future careers. Unfortunately, the orientation programs in most schools are not well designed to cater to students' varied needs. While career orientation programs are much needed at schools, a large number of Lebanese schools have not integrated such programs in their educational systems yet. Thus, this research focuses on developing a hybrid recommender system based on machine learning techniques. Our research focuses on the implementation of the formal concept analysis (FCA) to study university graduates' trajectories. This data mining technique is applied in knowledge presentation and data processing. FCA enables the discovery of hidden knowledge by using association rule mining and machine learning. The objective of this study is to discover the university graduates' trajectories and demonstrate the effectiveness of the FCA technique in data clustering.

Hierarchical Clustering is an unsupervised learning task, whi-ch seeks to build a set of clusters ordered by the inclusion relation. It is usually assumed that the result is a tree-like structure with no overlapping clusters, i.e., where clusters are either disjoint or nested. In Hierarchical Conceptual Clustering (HCC), each cluster is provided with a conceptual description which belongs to a predefined set called the pattern language. Depending on the application domain, the elements in the pattern language can be of different nature: logical formulas, graphs, tests on the attributes, etc. In this paper, we tackle the issue of overlapping concepts in the agglomerative approach of HCC. We provide a formal characterization of pattern languages that ensures a result without overlaps. Unfortunately, this characterization excludes many pattern languages which may be relevant for agglomerative HCC. Then, we propose two variants of the basic agglomerative HCC approach. Both of them guarantee a result without overlaps; the second one refines the given pattern language so that any two disjoint clusters have mutually exclusive descriptions.

Critical computer systems strongly rely on event logs to record the occurrence of normative and anomalous events occurring at runtime. In spite of the advances in Security Information and Event Management for handling monitoring data in production, event logs remain quite underutilized with respect to more conventional security data sources. Eliciting actionable knowledge for situational awareness poses many challenges in the case of logs emitted by industrial systems due to the lack of standard practices, formats and threat models. This paper addresses log analysis in a critical industrial system. We conduct our study with a real-life system by a top leading company in the Air Traffic Control domain, which emits massive volumes of unstructured proprietary logs. We propose a filtering method that pinpoints interesting events from logs, i.e., events that should be followed up by analysts. Experiments are done with logs from normative and misuse scenarios; moreover, we compare the outcome of our method with a reference filtering technique based on the conceptual clustering. Results indicate that the proposed method is effective to retain interesting events at remarkable precision and recall and to pinpoint misuse indicators. We overcome several drawbacks of existing filtering techniques, such as the need for labeled logs and domain knowledge, which makes our method easier to use by practitioners.

Nowadays, data mining becomes a very important research filed for knowledge discovery process. Among various data mining techniques, we focus on studying how a small amount of prior knowledge can be effectively incorporated into some popular clustering techniques, not only to improve the existing models, but also to develop novel semi-supervised clustering methods, especially for the categorization of high dimensional text documents. To be more specific, our objective is to investigate into some of the key performance characteristics of a good semi-supervised clustering method, and make them achievable in our purposed methods, such as: how to effectively incorporate the knowledge to guide the cluster search, accurately capture the underlying structure of the data, and achieve high capability of handling overlaps etc. In other words, our final goal is to develop some simple, fast and highly applicable clustering methods which aim to achieve good results with high quality and effectiveness with the help of the prior knowledge. Fuzzy co-clustering (FCC) is a type of clustering approaches that has shown its capability for handling high dimensional textual data categorization by simultaneously grouping the documents and words into some co-clusters. Meanwhile, for a document which intuitively spans multiple topics, FCC is also able to capture the degree of memberships of that document to each topic. Under the FCC framework, we proposed three different semi-supervised approaches, namely Semi-Supervised Fuzzy Co-clustering with Labelling (SS-FCL), Semi-Supervised Fuzzy Co-clustering with Constraints (SS-FCC) and Dual Semi-Supervised Heuristic Fuzzy Co-clustering with Ruspini’s condition (DSS-HFCR), respectively. Two types of prior knowledge in the forms of class labels and pair-wise constraints from the document domain are incorporated into SS-FCL and SS-FCC through different additional supervised constraint terms. Other than the categorization results of the documents, these two approaches also generate a group of work ranking clusters, which are useful in other data mining techniques, such as text summarization. Meanwhile, a heuristic dual-partitioning based approach called DSS-HFCR is also proposed in order to make full use of the available prior knowledge in terms of pair-wise constraints from both document and word domain. Moreover, DSS-HFCR can be directly downgraded to a simplified version if the prior knowledge is available from only a single domain Through extensive experimental study on a number of benchmark textual datasets, we demonstrate how these approaches make good use of the knowledge to guide the cluster search during the clustering process for a better performance in terms of both accuracy and efficiency. Some useful guidelines for parameter selection are also discussed. A case study of sentiment data analysis by applying DSS-HFCR demonstrates the strength of our proposed methods in the specific application area. FCC model is suitable for handling large sparse textual datasets, as it avoids applying an explicit similarity measure between two documents. However, similarity measure is still one of the most essential factors in many discriminative clustering approaches, and most of the these approaches still make use of only a single reference point (viewpoint) i.e. the origin for the similarity assessment. It is interesting and challenging to explore more effective similarity measures for high dimensional textual data, especially when some prior knowledge is available to the user. For the second part of the thesis, inspired by a recently proposed multi-viewpoint based similarity measure (MVS) [1], we introduce another novel semi-supervised clustering framework, which is able to utilize multiple appropriate viewpoints for a more informative and effective similarity assessment by incorporating two types of knowledge. With the help of a small number of class labels or pair-wise constraints in the dataset, we formulate two MVS measures, and subsequently propose two new MVS-based clustering approaches: Label-based Clustering with Multi-Viewpoint based Similarity (namely LMVS) and Pair-wise Constraints-based Clustering with Multi-Viewpoint based Similarity (namely PMVS). Comparing with the existing semi-supervised clustering techniques, the key strength of LMVS and PMVS is a more effective similarity measure can be directly formulated in the MVS manner with the help of the knowledge, and immediately applied to clustering, rather than learned by an independent distance metric learning process before the real clustering process is carried out. Some validity tests are conducted to show the strength of the measures, and systematical theoretical analysis is also provided to explain how the prior knowledge is utilized for both similarity enhancement and search-guiding purpose during the clustering process. At the same time, some potential issues of MVSC reported in [1] can be successfully addressed. At last, extensive experimental study on a large number of benchmark textual datasets are presented to demonstrate the effectiveness and verify the merit of LMVS & PMVS Clustering, compared with other start-of-the-art semi-supervised clustering/learning approaches. DOCTOR OF PHILOSOPHY (EEE)