Search

Remote sensing has resulted in repositories of data that grow at a pace much faster than can be readily analyzed. One of the obstacles in dealing with remotely sensed data and others is the variable quality of the data. Instrument failures can result in entire missing observation cycles, while cloud cover frequently results in missing or distorted values. We investigated the use of several methods that automatically deal with corruptions in the data. These include robust measures which avoid overfitting, filtering which discards the corrupted instances, and polishing by which the corrupted elements are fitted with more appropriate values. We applied such methods to a data set of vegetation indices and land cover type assembled from NASA's Moderate Resolution Imaging Spectroradiometer (MODIS) data collection.

Polishing is a noise correction mechanism which makes use of the inter-relationship between attribute and class values in the data set to identify and selectively correct components that are noisy. We applied polishing to a data set of amino acid sequences and associated information of point mutations of the gene COLIA1 for the classification of the phenotypes of the genetic collagenous disease Osteogenesis Imperfecta (OI). OI is associated with mutations in one or both of the genes COLIA1 and COLIA2. There are at least four known phenotypes of OI, of which type II is the severest and often lethal. Preliminary results of polishing suggest that it can lead to a higher classification accuracy. We further investigated the use of polishing as a scoring mechanism for feature selection, and the effect of the features so derived on the resulting classifier. Our experiments on the OI data set suggest that combining polishing and feature selection is a viable mechanism for improving data quality.

Osteogenesis Imperfecta (OI) is a genetic collagenous disease caused by mutations in one or both of the genes COLIA1 and COLIA2. There are at least four known phenotypes of OI, of which type II is the severest and often lethal. We applied a noise correction mechanism called polishing to a data set of amino acid sequences and associated information of point mutations of COLIA1. Polishing makes use of the inter-relationship between attribute and class values in the data set to identify and selectively correct components that are noisy. Preliminary results suggest that polishing is a viable mechanism for improving data quality, resulting in a more accurate classification of the lethal OI phenotype.

The standard formulation of association rules is suitable for describing patterns found in a given data set. These rules may each be adequately supported by the evidence, yet provide conflicting recommendations regarding an unseen instance when considered together. We proposed an alternative formulation called interval association rules, and developed a set of principles to adjudicate between conflicting rules.

In many experiments randomization is an important part of the protocol, yet precisely the same data could be produced by an experiment in which randomization played no part. From a Bayesian point of view, randomization plays at most a small role. From a classical point of view, randomization is central to ensuring that the long run error rates are controlled as they are claimed to be.

Standard association rules encapsulate the positive relationship between two sets of items: the presence of X is a good predictor for the simultaneous presence of Y . We argue that the absence of an association rule conveys valuable information as well. Dissociation rules are rules that capture the negative relationship between two sets of items: the presence of X and z is not a good predictor for the presence of Y . We developed a representation for augmenting standard association rules with dissociation information, and presented some experimental results suggesting that such augmented rules can improve the quality of the associations obtained, both in terms of rule accuracy and in terms of using these rules as a guide to making decisions.

Data quality is a prime concern for many tasks in learning and induction. We proposed in a previous paper a noise correction mechanism called polishing, which exploits the interdependence between the different components of a data set, to identify the noisy values and their appropriate replacements. The design of a sound and informative metric for evaluating the effectiveness of a noise correction scheme turned out to be non-trivial. We motivate here a number of classifier dependent measures and proximity measures, each focusing on a different aspect of the corrected data and the associated classifier. We report on some extended experimentation with polishing, as measured by the proposed metrics. The results suggested that polishing is able to repair a corrupted data set to some extent, and the metrics we devised appear to be reasonable.

When we work with information from multiple sources, the formats of the knowledge bases may not be uniform. It is desirable to be able to combine a knowledge base of default rules with one containing autoepistemic formulas. “Cooperative combination” refers to the integration of knowledge bases in different formats, in a way that retains as much of the individual characteristics of the component formalisms as possible. Previous work on relating default logic and autoepistemic logic mostly imposes some constraints on autoepistemic logic, and thus is not suitable for combining the two logics. We first present a fixed point formulation of autoepistemic logic analogous to that of default logic. Then we introduce a possible world framework with a partition structure, which corresponds to our intuitive notion of accessibility as linking alternate “possible” worlds. We show that both default logic and autoepistemic logic can be characterized using this common framework, which is important for developing a semantics applicable to the two logics, both separately and combined. We show one way of cooperatively integrating the two logics, and provide both a syntactic fixed point and a semantic possible partition sequence characterization.