Search

In this paper, we explore data mining techniques for the task of identifying and describing risk groups for colorectal cancer (CRC) from population based administrative health data. Association rule discovery, association classification and scalable clustering analysis are applied to the colorectal cancer patients' profiles in contrast to background patients' profiles. These data mining methods enable us to identify the most common characteristics of the colorectal cancer patients. The knowledge discovered by data mining methods which are quite different from traditional survey approaches. Although it is heuristic, the data mining methods may identify risk groups for further epidemiological study, such as older patients living near health facilities yet seldom utilising those facilities, and with respiratory and circulatory diseases. [ABSTRACT FROM AUTHOR]

Open source data mining software represents a new trend in data mining research, education and industrial applications, especially in small and medium enterprises (SMEs). With open source software an enterprise can easily initiate a data mining project using the most current technology. Often the software is available at no cost, allowing the enterprise to instead focus on ensuring their staff can freely learn the data mining techniques and methods. Open source ensures that staff can understand exactly how the algorithms work by examining the source codes, if they so desire, and can also fine tune the algorithms to suit the specific purposes of the enterprise. However, diversity, instability, scalability and poor documentation can be major concerns in using open source data mining systems. In this paper, we survey open source data mining systems currently available on the Internet. We compare 12 open source systems against several aspects such as general characteristics, data source accessibility, data mining functionality, and usability. We discuss advantages and disadvantages of these open source data mining systems. [ABSTRACT FROM AUTHOR]

This paper presents a new method for effectively selecting initial cluster centers in k-means clustering. This method identifies the high density neighborhoods from the data first and then selects the central points of the neighborhoods as initial centers. The recently published Neighborhood-Based Clustering (NBC) algorithm is used to search for high density neighborhoods. The new clustering algorithm NK-means integrates NBC into the k-means clustering process to improve the performance of the k-means algorithm while preserving the k-means efficiency. NBC is enhanced with a new cell-based neighborhood search method to accelerate the search for initial cluster centers. A merging method is employed to filter out insignificant initial centers to avoid too many clusters being generated. Experimental results on synthetic data sets have shown significant improvements in clustering accuracy in comparison with the random k-means and the refinement k-means algorithms. Keywords: Clustering, k-means, Neighborhood-Based Clustering, Initial Cluster Center Selection. [ABSTRACT FROM AUTHOR]

This paper describes a time-changing feature selectionFeature is also called in other names such as attribute, property and characteristic. framework based on hierachical distribution method for extracting knowledge from health records. In the framework, we propose three steps for time-changing feature selection. The first step is a qualitative-based search, to find qualitative features (or, structural time-changing features). The second step performs a quantitative-based search, to find quantitative features (or, value time-changing features). In the third step, the results from the first two steps are combined to form hybrid search models to select a subset of global time-changing features according to a certain criterion of medical experts. The present application of the time-changing feature selection method involves time-changing episode history, an integral part of medical health records and it also provides some challenges in time-changing data mining techniques. The application task was to examine time related features of medical treatment services for diabetics. This was approached by clustering patients into groups receiving similar patterns of care and visualising the features devised to highlight interesting patterns of care. Keywords: time-changing data mining, event sequence, time-changing feature selection, hierarchical distribution, health records. [ABSTRACT FROM AUTHOR]

In order to cut down on soaring personal loan bankruptcies, the Hong Kong government had unveiled a plan in early of 2002 to allow banks to share more credit information about their customers. This paper analyses how effective the positive data sharing scheme will be and examines whether any other personal credit attributes can serve the same purpose. In our work, a survey was conducted to verify industry's perception on what attributes was essential for credit risk assessment. The result was compared with the implication from the neuro-fuzzy data mining on real transaction data. The comparison suggests that the perception on positive data is not absolutely correct and the positive data sharing cannot always achieve its purposes. [ABSTRACT FROM AUTHOR]

In the past, some sites selected for closure by a large international bank in Hong Kong were based on personal experience of a group of experts by formulating a set of evaluation guidelines. The current 300 existing sites are therefore considered to represent a set of rules and expert decisions which are manually recorded on paper files and de-centralized. In order to validate the guidelines/rules and discover any hidden knowledge, we employ a data mining approach to examine the data comprehensively. Several modeling techniques including neural network, C5.0 and General Rule Induction systems are used to determine the significance of those attributes in the data set. Various models based on the historical data set of sites in different forms are constructed to deduce a rule-based model for subsequent use. Promising result has been obtained which can be applied in future Branch and ATM Site Evaluation with a view of providing a better solution. The useful patterns and knowledge discovered will further add benefit to exploring customer intelligence and devising marketing planning strategies. Keywords: Datamining, rule induction, branch and site evaluation, model analysis. [ABSTRACT FROM AUTHOR]

The Scamseek project, as commissioned by ASIC has the principal objective of building an industrially viable system that retrieves potential scam candidate documents from the Internet and classifies them as to their potential risk of containing an illegal investment proposal or advice. The project produced multiple classifiers for different types of data, and achieved higher than expected performance statistics on classifications. The development of the system required the solution of two major problems in document classification, namely accurate identification of classes with very small footprints, <.1%, and classification using meaning intention rather than word strings. The approach taken used Systemic Functional Grammar to model the semantics of the scam classes and used unigrams with significant language pre-processing to assist in separating irrelevant documents. Litigations have been initiated by ASIC from classifications made by the systemSee ASIC Media Release 04-178: Grammax Investment Club operating unlicensed investment club is believed to have moved over $10M overseas in the prior year.. ASIC operates the system on a 24/7 basis. The estimate of savings in human effort in its monitoring role is the order of 100-fold. The estimate in savings to the community cannot be estimated readily but is likely to be of the order of tens of millions of dollars. [ABSTRACT FROM AUTHOR]

Severe Acute Respiratory Syndrome (SARS) has infected more than 8,000 persons [1] after it first broke out in Guangdong China. As there was no fast and effective detection method of suspected SARS cases,this paper proposes a computer aided SARS detection system (CADSARS) based on data mining techniques.‘Typical pneumonia' and SARS X-Ray chest radiographs were collected.Feature extraction of these images was performed after segmenting out pulmonary fields. Feature vectors were then constructed to build rules for the discrimination of SARS and ‘typical pneumonia'.Three methods were used to classify these images: C4.5, neural network and CART.Final results show that about 70.94% SARS cases can be detected. ROC charts and confusion matrix by these three methods are given and analyzed.Association rules mining was used to find whether there exists difference of lesions' location between SARS and pneumonia cases. [ABSTRACT FROM AUTHOR]

Lot of research has gone into understanding the composition and nature of proteins, still many things remain to be understood satisfactorily. It is now generally believed that amino acid sequences of proteins are not random, and thus the patterns of amino acids that we observe in the protein sequences are also non-random. In this study, we have attempted to decipher the nature of associations between different amino acids that are present in a protein. This very basic analysis provides insights into the co-occurrence of certain amino acids in a protein. Such association rules are desirable for enhancing our understanding of protein composition and hold the potential to give clues regarding the global interactions amongst some particular sets of amino acids occuring in proteins. Presence of strong non-trivial associations suggests further evidence for non-randomness of protein sequences. Knowledge of these rules or constraints is highly desirable for the in-vitro synthesis of artificial proteins. Keywords: Data mining, quantitative association rule mining, protein composition. [ABSTRACT FROM AUTHOR]

This paper provides a constructive algorithm in which a hierarchical tree of hidden Markov models may be obtained directly from data using an unsupervised learning regime. The method is applied to health insurance transaction data such that profiles with similar local temporal behaviours are grouped together. By judicious incorporation of limited additional prior information, it is found that profiles can be separated into various sub-behavioural groups thus providing a technique for large-scale automatic labelling of data. In the application to the health insurance transaction data set, by incorporating limited information concerning the medical functions used in a medical procedure, it is possible to label some individual medical transactions as to whether they are related to a particular medical condition or not. This automatic labelling process adds values to the collected transactional database for possible further applications, e.g. public health studies. [ABSTRACT FROM AUTHOR]

Traditionally text mining has had a strong link with information retrieval and classification and has largely aimed to classify documents according to embedded knowledge. Association rule mining and sequence mining, on the other hand, have had a different goal; one of eliciting relationships within or about the data being mined. Recently there has been research conducted using sequence mining techniques on digital document collections by treating the text as sequential data. In this paper we propose a multi-level framework that is applicable to text analysis and that improves the knowledge discovery process by finding additional or hitherto unknown relationships within the data being mined. We believe that this can lead to the detection or fine tuning of the context of documents under consideration and may lead to a more informed classification of those documents. Moreover, since we use a semantic map at varying stages in the framework, we are able to impose a greater degree of focus and therefore a greater transitivity of semantic relatedness that facilitates the improvement in the knowledge discovery process. [ABSTRACT FROM AUTHOR]

Data mining is finding hidden rules in given dataset using non-traditional methods. The objective is to discover useful or patterns from the given collection of data. This research investigates if the differences in accuracy of "time series forecasting" are related to the differences in one's cognitive style and subjective emotion. Two kinds of analyses were performed before applying data mining. Firstly, a statistical test was used to see if there was a positive correlation between a number of cognitive styles and subjective emotional states and the accuracy of time-series forecasting. This was not very revealing and the next step are to use a self-organizing neural network (SONN) to see if correlations between these variables could be discovered. The results showed that there were correlations but did not show whether the correlations were positive or negative. Finally data mining was applied to discover which cognitive styles and subjective emotions positively influence forecasting. It was found that subjects who have in analytic style and subjects who have a relaxed mode were more accurate in their judgments than those who do not these characteristics. Keywords: data mining, neural network, self-organizing neural network, self-supervised adaptive neural network, cognitive style, emotion, intuitive forecasting, decision making. [ABSTRACT FROM AUTHOR]

The paper presents the findings of an industry-based study in the utility of text mining. The purpose of the study was to evaluate the impact of textual information in claims cost prediction. The industrial research setting was a large Australian insurance company. The data mining methodologies used in this research included text mining, and the application of the results from the text mining in subsequent predictive data mining models. The researchers used software of the leading commercial vendors. The research found commercially interesting utility in textual information for claim cost prediction, and also identified new risk management factors. Keywords: text mining, predictive model, insurance claim prediction, risk management. [ABSTRACT FROM AUTHOR]

Information acquisition and validation play an important role in the decision making process during negotiation. In this chapter we briefly present the framework of a smart data mining system for providing contextual information extracted from the Internet to a negotiation agent. We then present one of its components in more details - an effective automated technique for extracting relevant articles from news web sites, so that they can be used further by the mining agents. Most current techniques experience difficulties in coping with changes in web site structure and formats. The proposed extraction process is completely automatic and independent of web site formats. Proposed technique identifies regularities in both format and content of news web sites. The algorithms are applicable to both single- and multi-document web sites. Since invalid URLs can cause errors in data extraction, we also present a method for the negotiation agent to estimate the validity of the extracted data based on the frequency of the relevant words in the news title. Once the news articles are extracted the next task is to construct sets of given articles. This chapter presents a new procedure for constructing news data sets on given topics. The extracted news data set is further utilised by the parties involved in negotiation. The information retrieved from the data set can support both human and automated negotiators. [ABSTRACT FROM AUTHOR]

Current major search engines on the web retrieve too many documents, of which only a small fraction are relevant to the user query. We propose a new intelligent document filtering algorithm to filter out documents irrelevant to the user query from the output of internet search engines. This algorithm uses output of ‘Google' search engine as the basic input and processes this input to filter documents most relevant to the query. The clustering algorithm used here is based on the fuzzy c-means with modifications to the membership function formulation and cluster prototype initialisation. It classifies input documents into 3 predefined clusters. Finally, clustered and context-based ranked URLs are presented to the user. The effectiveness of the algorithm has been tested using data provided by the eighth Text REtrieval Conference (TREC-8) [25] and also with on-line data. Experimental results were evaluated by using error matrix method, precision, recall and clustering validity measures. [ABSTRACT FROM AUTHOR]

The process of identifying record pairs that represent the same real-world entity in multiple databases, commonly known as record linkage, is one of the important steps in many data mining applications. In this paper, we address one of the sub-tasks in record linkage, i.e., the problem of assigning record pairs with an appropriate matching status. Techniques for solving this problem are referred to as decision models. Most existing decision models rely on good training data, which is, however, not commonly available in real-world applications. Decision models based on unsupervised machine learning techniques have recently been proposed. In this paper, we review several existing decision models and then propose an enhancement to cluster-based decision models. Experimental results show that our proposed decision model achieves the same accuracy of existing models while significantly reducing the number of record pairs required for manual review. The proposed model also provides a mechanism to trade off the accuracy with the number of record pairs required for clerical review. Keywords: data linking, record linkage, probabilistic linking, decision model, clustering, classification. [ABSTRACT FROM AUTHOR]

It is estimated that between 80% and 90% of governmental data collections contain address information. Geocoding - the process of assigning geographic coordinates to addresses - is becoming increasingly important in application areas that involve the analysis and mining of such data. In many cases, address records are captured and/or stored in a free-form or inconsistent manner. This fact complicates the task of accurately matching such addresses to spatially-annotated reference data. In this paper we describe a geocoding system that is based on a comprehensive high-quality geocoded national address database. It uses a learning address parser based on hidden Markov models to segment free-form addresses into components, and a rule-based matching engine to determine the best matches to the reference database. [ABSTRACT FROM AUTHOR]

This paper proposes a mining novel approach which consists of two new data mining algorithms for the classification over quantitative data, based on two new pattern called MOUCLAS (MOUntain function based CLASsification) Patterns and JumpingMOUCLAS Patterns. The motivation of the study is to develop two classifiers for quantitative attributes by the concepts of the association rule and the clustering. An illustration of using petroleum well logging data for oil/gas formation identification is presented in the paper. MPsandJMPs are ideally suitable to derive the implicit relationship between measured values (well logging data) and properties to be predicted (oil/gas formation or not). As a hybrid of classification and clustering and association rules mining, our approach have several advantages which are (1) it has a solid mathematical foundation and compact mathematical description of classifiers, (2) it does not require discretization, (3) it is robust when handling noisy or incomplete data in high dimensional data space. [ABSTRACT FROM AUTHOR]

During the life cycle of a software development project, many problems arise. Resolutions to these problems are time consuming and expensive. This paper discusses the use of data mining in solving some of these problems to improve the system development life cycle process. A case study of applying data mining to the software Problem Report management data is also presented. The empirical results demonstrate the capability and benefit of data mining analysis in systems development life cycle. [ABSTRACT FROM AUTHOR]

This paper proposes a decision theoretic fusion framework for actionability using data mining techniques in an embedded car navigation system. An embedded system having limited resources is not easy to manage the abundant information in the database. Thus, the proposed system stores and manages only multiple level-of-abstraction in the database to resolve the problem of resource limitations, and then represents the information received from the Web via the wireless network after connecting a communication channel with the data mining server. To do this, we propose a decision theoretic fusion framework that includes the multiple level-of-abstraction approach combining multiple-level association rules and the summary table, as well as an active interaction rule generation algorithm for actionability in an embedded car navigation system. In addition, it includes the sensory and data fusion level rule extraction algorithm to cope with simultaneous events occurring from multi-modal interface. The proposed framework can make interactive data mining flexible, effective, and instantaneous in extracting the proper action item. Keywords: Data mining, Embedded data mining, and Speech interactive approach. [ABSTRACT FROM AUTHOR]

Traditional methods for discovering frequent patterns from large databases assume equal weights for all items of the database. In the real world, managerial decisions are based on economic values attached to the item sets. In this paper, we first introduce the concept of the value based frequent item packages problems. Then we provide an integer linear programming (ILP) model for value based optimization problems in the context of transaction data. The specific problem discussed in this paper is to find an optimal set of item packages (or item sets making up the whole transaction) that returns maximum profit to the organization under some limited resources. The specification of this problem allows us to solve a number of practical decision problems, by applying the existing and new ILP solution techniques. The model has been implemented and tested with real life retail data. The test results are reported in the paper. [ABSTRACT FROM AUTHOR]

How to efficiently discard potentially uninteresting rules in exploratory rule discovery is one of the important research foci in data mining. Many researchers have presented algorithms to automatically remove potentially uninteresting rules utilizing background knowledge and user-specified constraints. Identifying the significance of exploratory rules using a significance test is desirable for removing rules that may appear interesting by chance, hence providing the users with a more compact set of resulting rules. However, applying statistical tests to identify significant rules requires considerable computation and data access in order to obtain the necessary statistics. The situation gets worse as the size of the database increases. In this paper, we propose two approaches for improving the efficiency of significant exploratory rule discovery. We also evaluate the experimental effect in impact rule discovery which is suitable for discovering exploratory rules in very large, dense databases. Keywords: Exploratory rule discovery, impact rule, rule significance, interestingness measure. [ABSTRACT FROM AUTHOR]

This paper presents a new hybrid classifier that combines the Nearest Neighbor distance based algorithm with the Classification Tree paradigm. The Nearest Neighbor algorithm is used as a preprocessing algorithm in order to obtain a modified training database for the posterior learning of the classification tree structure; experimental section shows the results obtained by the new algorithm; comparing these results with those obtained by the classification trees when induced from the original training data we obtain that the new approach performs better or equal according to the Wilcoxon signed rank statistical test. Keywords: Machine Learning, Supervised Classification, Classifier Combination, Classification Trees. [ABSTRACT FROM AUTHOR]

When different subsamples of the same data set are used to induce classification trees, the structure of the built classifiers is very different. The stability of the structure of the tree is of capital importance in many domains, such as illness diagnosis, fraud detection in different fields, customer's behaviour analysis (marketing), etc, where comprehensibility of the classifier is necessary. We have developed a methodology for building classification trees from multiple samples where the final classifier is a single decision tree (Consolidated Trees). The paper presents an analysis of the structural stability of our algorithm versus C4.5 algorithm. The classification trees generated with our algorithm, achieve smaller error rates and structurally more steady trees than C4.5 when using resampling techniques. The main focus on this paper is showing how Consolidated Trees built with different sets of subsamples tend to converge to the same tree when the number of used subsamples is increased. [ABSTRACT FROM AUTHOR]

Data mining practice in industry heavily depends on experienced data mining professionals to provide solutions. Normal business users cannot easily use data mining tools to solve their business problems, because of the complexity of data mining process and data mining tools. In this paper, we propose a case-based data mining platform, which reuses the knowledge captured in past data mining cases to semi-automatically solve new similar problems. We first extend generic data mining model for knowledge reuse. Then we define data mining case. And then we introduce this platform in detail from its storage bases, functional modules, user interface, and application scenario. Theoretically, this platform can simplify data mining process, reduce the dependency on data mining professional, and shorten business decision time. Keywords: Data Mining, Knowledge Reuse, Case-Based Reasoning, Case- Based Data Mining Platform. [ABSTRACT FROM AUTHOR]

We present a prototype application for graph-based exploration and mining of online databases, with particular emphasis on scientific data. The application builds structured graphs that allow the user to explore patterns in a data set, including clusters, trends, outliers, and relationships. A number of different graphs can be rapidly generated, giving complementary insights into a given data set. The application has a Flash-based graphical interface and uses semantic information from the data sources to keep this interface as intuitive as possible. Data can be accessed from local and remote databases and files. Graphs can be explored using an interactive visual browser, or graph-analytic algorithms. We demonstrate the approach using marine sediment data, and show that differences in benthic species compositions in two Antarctic bays are related to heavy metal contamination. [ABSTRACT FROM AUTHOR]

During knowledge acquisition it frequently occurs that multiple alternative potential rules all appear equally credible. This paper addresses the dearth of formal analysis about how to select between such alternatives. It presents two hypotheses about the expected impact of selecting between classification rules of differing levels of generality in the absence of other evidence about their likely relative performance on unseen data. We argue that the accuracy on unseen data of the more general rule will tend to be closer to that of a default rule for the class than will that of the more specific rule. We also argue that in comparison to the more general rule, the accuracy of the more specific rule on unseen cases will tend to be closer to the accuracy obtained on training data. Experimental evidence is provided in support of these hypotheses. These hypotheses can be useful for selecting between rules in order to achieve specific knowledge acquisition objectives. [ABSTRACT FROM AUTHOR]

An association classification algorithm has been developed to explore adverse drug reactions in a large medical transaction dataset with unbalanced classes. Rules discovered can be used to alert medical practitioners when prescribing drugs, to certain categories of patients, to potential adverse effects. We assess the rules using survival charts and propose two kinds of probability trees to present them. Both of them represent the risk of given adverse drug reaction for certain categories of patients in terms of risk ratios, which are familiar to medical practitioners. The first approach shows risk ratios when all rule conditions apply. The second presents the risk associated with a single risk factor with other parts of the rule identifying the cohort of the patient subpopulation. Thus, the probability trees can present clearly the risk of specific adverse drug reactions to prescribers. [ABSTRACT FROM AUTHOR]