Search

Computing derivatives of tensor expressions, also known as tensor calculus, is a fundamental task in machine learning. A key concern is the efficiency of evaluating the expressions and their derivatives that hinges on the representation of these expressions. Recently, an algorithm for computing higher order derivatives of tensor expressions like Jacobians or Hessians has been introduced that is a few orders of magnitude faster than previous state-of-the-art approaches. Unfortunately, the approach is based on Ricci notation and hence cannot be incorporated into automatic differentiation frameworks from deep learning like TensorFlow, PyTorch, autograd, or JAX that use the simpler Einstein notation. This leaves two options, to either change the underlying tensor representation in these frameworks or to develop a new, provably correct algorithm based on Einstein notation. Obviously, the first option is impractical. Hence, we pursue the second option. Here, we show that using Ricci notation is not necessary for an efficient tensor calculus and develop an equally efficient method for the simpler Einstein notation. It turns out that turning to Einstein notation enables further improvements that lead to even better efficiency. The methods that are described in this paper for computing derivatives of matrix and tensor expressions have been implemented in the online tool . [ABSTRACT FROM AUTHOR]

The aim of this paper is to present the methods and algorithms of information systems decomposition. In the paper, decomposition with respect to reducts and the so-called global decomposition are considered. Moreover, coverings of information systems by components are discussed. An essential difference between two kinds of decomposition can be observed. In general, global decomposition can deliver more components of a given information system. This fact can be treated as some kind of additional knowledge about the system. The proposed approach is based on rough set theory. To demonstrate the usefulness of this approach, we present an illustrative example coming from the economy domain. The discussed decomposition methods can be applied e.g. for design and analysis of concurrent systems specified by information systems, for automatic feature extraction, as well as for control design of systems represented by experimental data tables. [ABSTRACT FROM AUTHOR]

In this paper we review Lattice Machine, a learning paradigm that 'learns' by generalising data in a consistent, conservative and parsimonious way, and has the advantage of being able to provide additional reliability information for any classification. More specifically, we review the related concepts such as hyper tuple and hyper relation, the three generalising criteria (equilabelledness, maximality, and supportedness) as well as the modelling and classifying algorithms. In an attempt to find a better method for classification in Lattice Machine, we consider the contextual probability which was originally proposed as a measure for approximate reasoning when there is insufficient data. It was later found to be a probability function that has the same classification ability as the data generating probability called primary probability. It was also found to be an alternative way of estimating the primary probability without much model assumption. Consequently, a contextual probability based Bayes classifier can be designed. In this paper we present a new classifier that utilises the Lattice Machine model and generalises the contextual probability based Bayes classifier. We interpret the model as a dense set of data points in the data space and then apply the contextual probability based Bayes classifier. A theorem is presented that allows efficient estimation of the contextual probability based on this interpretation. The proposed classifier is illustrated by examples. [ABSTRACT FROM AUTHOR]

This paper is devoted to the study of an extension of dynamic programming approach which allows optimization of partial decision rules relative to the length or coverage. We introduce an uncertainty measure J(T) which is the difference between number of rows in a decision table T and number of rows with the most common decision for T. For a nonnegative real number γ, we consider γ-decision rules (partial decision rules) that localize rows in subtables of T with uncertainty at most γ. Presented algorithm constructs a directed acyclic graph Δγ(T) which nodes are subtables of the decision table T given by systems of equations of the kind 'attribute = value'. This algorithm finishes the partitioning of a subtable when its uncertainty is at most γ. The graph Δγ(T) allows us to describe the whole set of so-called irredundant γ-decision rules. We can optimize such set of rules according to length or coverage. This paper contains also results of experiments with decision tables from UCI Machine Learning Repository. [ABSTRACT FROM AUTHOR]

Recent research has shown the integration of multiple classifiers to be one of the most important directions in machine learning and data mining. In this paper, we present an algorithm for the dynamic integration of classifiers in the space of extracted features (FEDIC). It is based on the technique of dynamic integration, in which local accuracy estimates are calculated for each base classifier of an ensemble, in the neighborhood of a new instance to be processed. Generally, the whole space of original features is used to find the neighborhood of a new instance for local accuracy estimates in dynamic integration. However, when dynamic integration takes place in high dimensions the search for the neighborhood of a new instance is problematic, since the majority of space is empty and neighbors can in fact be located far from each other. Furthermore, when noisy or irrelevant features are present it is likely that also irrelevant neighbors will be associated with a test instance. In this paper, we propose to use feature extraction in order to cope with the curse of dimensionality in the dynamic integration of classifiers. We consider classical principal component analysis and two eigenvector-based class-conditional feature extraction methods that take into account class information. Experimental results show that, on some data sets, the use of FEDIC leads to significantly higher ensemble accuracies than the use of plain dynamic integration in the space of original features. [ABSTRACT FROM AUTHOR]

Rough set theory is a kind of new tool to deal with knowledge, particularly when knowledge is imprecise, inconsistent and incomplete. In this paper, we discuss some inductive machine learning techniques in the framework of the knowledge reduction approach based on rough set theory. The Monk's problems introduced in the early of nineties are resolved again employing rough set methods and their results are compared and analyzed with those obtained at that time. As far as accuracy and conciseness are concerned, the learning algorithms based on rough sets have remarkable superiority. [ABSTRACT FROM AUTHOR]

This paper presents a new model of an artificial neural network solving classification problems, called Local Transfer Function Classifier (LTF-C). Its architecture is very similar to this of the Radial Basis Function neural network (RBF), however it utilizes an entirely different learning algorithm. This algorithm is composed of four main parts: changing positions of reception fields, changing their sizes, insertion of new hidden neurons and removal of unnecessary ones during the training. The paper presents also results of LTF-C application to three real-life tasks: handwritten digit recognition, credit approval and cancer diagnosis. LTF-C was able to solve each of these problems with better accuracy than most popular classification systems. Moreover, LTF-C was relatively small and fast. [ABSTRACT FROM AUTHOR]

This paper summarizes the some of the recent developments in the area of application of rough sets and granular computing in hierarchical learning. We present the general framework of rough set based hierarchical learning. In particular, we investigate several strategies of choosing the appropriate learning algorithms for first level concepts as well as the learning methods for the intermediate concepts. We also propose some techniques for embedding the domain knowledge into the granular, layered learning process in order to improve the quality of hierarchical classifiers. This idea, which has been envisioned and developed by professor Andrzej Skowron over the last 10 years, shows to be very efficient in many practical applications. Throughout the article, we illustrate the proposed methodology with three case studies in the area of pattern recognition. The studies demonstrate the viability of this approach for such problems as: sunspot classification, hand-written digit recognition, and car identification. [ABSTRACT FROM AUTHOR]

Machine learning methods are often used to classify objects described by hundreds of attributes; in many applications of this kind a great fraction of attributes may be totally irrelevant to the classification problem. Even more, usually one cannot decide a priori which attributes are relevant. In this paper we present an improved version of the algorithm for identification of the full set of truly important variables in an information system. It is an extension of the random forest method which utilises the importance measure generated by the original algorithm. It compares, in the iterative fashion, the importances of original attributes with importances of their randomised copies. We analyse performance of the algorithm on several examples of synthetic data, as well as on a biologically important problem, namely on identification of the sequence motifs that are important for aptameric activity of short RNA sequences. [ABSTRACT FROM AUTHOR]

The major problem of SVMs is the dependence of the nonlinear separating surface on the entire dataset which creates unwieldy storage problems. This paper proposes a novel design algorithm, called the extractive support vector algorithm, which provides improved learning speed and a vastly improved performance. Instead of learning and training with all input patterns, the proposed algorithm selects support vectors from the input patterns and uses these support vectors as the training patterns. Experimental results reveal that our proposed algorithmprovides near optimal solutions and outperforms the existing design algorithms. In addition, a significant framework which is based on extractive support vector algorithm is proposed for image restoration. In the framework, input patterns are classified by three filters: weighted order statistics filter, alpha-trimmed mean filter and identity filter. Our proposed filter can achieve three objectives: noise attenuation, chromaticity retention, and preservation of edges and details. Extensive simulation results illustrate that our proposed filter not only achieves these three objectives but also possesses robust and adaptive capabilities, and outperforms other proposed filtering techniques. [ABSTRACT FROM AUTHOR]

Feature selection is an important preprocessing step in pattern analysis and machine learning. The key issue in feature selection is to evaluate quality of candidate features. In this work, we introduce a weighted distance learning algorithm for feature selection via maximizing fuzzy dependency. We maximize fuzzy dependency between features and decision by distance learning and then evaluate the quality of features with the learned weight vector. The features deriving great weights are considered to be useful for classification learning. We test the proposed technique with some classical methods and the experimental results show the proposed algorithm is effective. [ABSTRACT FROM AUTHOR]

Presents a study that described a data mining technique where a classification algorithm is used as an evaluation function to differentiate between different feature subsets. Background on the use of an ensemble of classifiers; Strategy for dynamic integration of classifiers; Discussion on local feature selection based on the dynamic classifier integration; Application of the technique in experiments on data sets taken from a machine learning repository.

Presents a study that discussed ways of learning to predict ordinal classes using regression trees. Background on Structural Classification and Regression Trees (S-CART), an algorithm for first-order classification and regression trees; Modifications that turn S-CART into an algorithm for predicting ordinal classes; Performance of several S-CART variants in five application domains.