Search

The duplicate detection is one of technical difficulties in data cleaning area. At present, the data volume of scientific database is increasing rapidly, bringing new challenges to the duplicate detection. In the scientific database, the duplicate detection model should be suitable for massive and numerical data, should independent from the domains, should well consider the relationships among tables, and should focus on common grounds of the scientific database. In the paper, a multilevel duplicate detection model for scientific database is proposed, which consider numerical data and general usage well. Firstly, the challenges are propose by analyzing duplicate-related characteristics of scientific data; Secondly, similarity measure of the proposed model are defined; Then the details of multilevel detecting algorithms are introduced; At last, some experiments and applications show that the proposed model is more domain-independent and effective, suitable for duplicate detection in scientific database. [ABSTRACT FROM AUTHOR]

The computation of a data cube is one of the most essential but challenging issues in data warehousing and OLAP. Partition based algorithm is one of the efficient methods to compute data cubes on high dimensionality, low cardinality, and moderate size datasets, which exist in real applications like bioinformatics, statistics, and text processing. To deal with such high dimensional data cubes, we propose an efficient indexing technique consisting of a compressed bitmap index and two algorithms for cube constructing and querying. Experimental results show that our method saves at least 25% on storage space and about 30% on computation time compared with the Frag-Cubing algorithm. [ABSTRACT FROM AUTHOR]

Due to the high-dimension and sparseness properties of documents, clustering the similar documents together is a tough task. The most popular document clustering method K-Means has the shortcoming of its cluster intra-dissimilarity, i.e. inclining to clustering unrelated documents together. One of the reasons is that all objects (documents) in a cluster produce the same influence to the mean of the cluster. SOM (Self Organizing Map) is a method to reduce the dimension of data and display the data in low dimension space, and it has been applied successfully to clustering of high-dimensional objects. The scalar factor is an important part of SOM. In this paper, an optimized K-Means algorithm is proposed. It introduces the scalar factor from SOM into means during K-Means assignment stage for controlling the influence to the means from new objects. Experiments show that the optimized K-Means algorithm has more F-Measure and less Entropy of clustering than standard K-Means algorithm, thereby reduces the intra-dissimilarity of clusters effectively. [ABSTRACT FROM AUTHOR]

Classification rule mining plays significant roles in practical applications. While how to mine classification rules from a database efficiently is an important issue, how to manage and apply the mined rules effectively is the same important. This paper presents a classification rule mining management system, named SM-Classifier, which integrates the functionality of generation, querying, maintenance and application of classification rules. The architecture of SM-Classifier, the classification algorithm, the rule store structure, and the rule query operations are described. [ABSTRACT FROM AUTHOR]