Search

Many methods exist for generating keyframe summaries of videos. However, relatively few methods consider on-line summarisation, where memory constraints mean it is not practical to wait for the full video to be available for processing. We propose a classification (taxonomy) for on-line video summarisation methods based upon their descriptive and distinguishing properties such as feature space for frame representation, strategies for grouping time-contiguous frames, and techniques for selecting representative frames. Nine existing on-line methods are presented within the terms of our taxonomy and subsequently compared by testing on two synthetic data sets and a collection of short videos. We find that success of the methods is largely independent of techniques for grouping time-contiguous frames and for measuring similarity between frames. On the other hand, decisions about the number of keyframes and the selection mechanism may substantially affect the quality of the summary. Finally, we remark on the difficulty in tuning the parameters of the methods “on-the-fly”, without knowledge of the video duration, dynamic or content.

Random Balance strategy (RandBal) has been recently proposed for constructing classifier ensembles for imbalanced, two-class data sets. In RandBal, each base classifier is trained with a sample of the data with a random class prevalence, independent of the a priori distribution. Hence, for each sample, one of the classes will be undersampled while the other will be oversampled. RandBal can be applied on its own or can be combined with any other ensemble method. One particularly successful variant is RandBalBoost which integrates Random Balance and boosting. Encouraged by the success of RandBal, this work proposes two approaches which extend RandBal to multiclass imbalance problems. Multiclass imbalance implies that at least two classes have substantially different proportion of instances. In the first approach proposed here, termed Multiple Random Balance (MultiRandBal), we deal with all classes simultaneously. The training data for each base classifier are sampled with random class proportions. The second approach we propose decomposes the multiclass problem into two-class problems using one-vs-one or one-vs-all, and builds an ensemble of RandBal ensembles. We call the two versions of the second approach OVO-RandBal and OVA-RandBal, respectively. These two approaches were chosen because they are the most straightforward extensions of RandBal for multiple classes. Our main objective is to evaluate both approaches for multiclass imbalanced problems. To this end, an experiment was carried out with 52 multiclass data sets. The results suggest that both MultiRandBal, and OVO/OVA-RandBal are viable extensions of the original two-class RandBal. Collectively, they consistently outperform acclaimed state-of-the art methods for multiclass imbalanced problems., Ministerio de Economía yCompetitividad[http://dx.doi.org/10.13039/501100003329] of theSpanishGovernmentthroughprojectTIN2015-67534-P(MINECO/FEDER, UE) and the Junta de Castilla y León through projectBU085P17 (JCyL/FEDER, UE)

Over the past few decades, the remarkable prediction capabilities of ensemble methods have been used within a wide range of applications. Maximization of base-model ensemble accuracy and diversity are the keys to the heightened performance of these methods. One way to achieve diversity for training the base models is to generate artificial/synthetic instances for their incorporation with the original instances. Recently, the mixup method was proposed for improving the classification power of deep neural networks (Zhang, Cisse, Dauphin, and Lopez-Paz, 2017). Mixup method generates artificial instances by combining pairs of instances and their labels, these new instances are used for training the neural networks promoting its regularization. In this paper, new regression tree ensembles trained with mixup, which we will refer to as Mixup Regression Forest, are presented and tested. The experimental study with 61 datasets showed that the mixup approach improved the results of both Random Forest and Rotation Forest.

Many real-life problems can be described as unbalanced, where the number of instances belonging to one of the classes is much larger than the numbers in other classes. Examples are spam detection, credit card fraud detection or medical diagnosis. Ensembles of classifiers have acquired popularity in this kind of problems for their ability to obtain better results than individual classifiers. The most commonly used techniques by those ensembles especially designed to deal with imbalanced problems are for example Re-weighting, Oversampling and Undersampling. Other techniques, originally intended to increase the ensemble diversity, have not been systematically studied for their effect on imbalanced problems. Among these are Random Oracles, Disturbing Neighbors, Random Feature Weights or Rotation Forest. This paper presents an overview and an experimental study of various ensemble-based methods for imbalanced problems, the methods have been tested in its original form and in conjunction with several diversity-increasing techniques, using 84 imbalanced data sets from two well known repositories. This paper shows that these diversity-increasing techniques significantly improve the performance of ensemble methods for imbalanced problems and provides some ideas about when it is more convenient to use these diversifying techniques.

Proportions of the classes for each ensemble member are chosen randomly.Member training data: sub-sample and over-sample through SMOTE.RB-Boost combines Random Balance with AdaBoost.M2.Experiments with 86 data sets demonstrate the advantage of Random Balance. In Machine Learning, a data set is imbalanced when the class proportions are highly skewed. Imbalanced data sets arise routinely in many application domains and pose a challenge to traditional classifiers. We propose a new approach to building ensembles of classifiers for two-class imbalanced data sets, called Random Balance. Each member of the Random Balance ensemble is trained with data sampled from the training set and augmented by artificial instances obtained using SMOTE. The novelty in the approach is that the proportions of the classes for each ensemble member are chosen randomly. The intuition behind the method is that the proposed diversity heuristic will ensure that the ensemble contains classifiers that are specialized for different operating points on the ROC space, thereby leading to larger AUC compared to other ensembles of classifiers. Experiments have been carried out to test the Random Balance approach by itself, and also in combination with standard ensemble methods. As a result, we propose a new ensemble creation method called RB-Boost which combines Random Balance with AdaBoost.M2. This combination involves enforcing random class proportions in addition to instance re-weighting. Experiments with 86 imbalanced data sets from two well known repositories demonstrate the advantage of the Random Balance approach.

Pattern recognition concerns assigning objects to classes. The objects are described by features (variables or measurements) organized as p-dimensional points in some feature space. A classifier is a formula, an algorithm or a technique that can assign a class label to any given point in the feature space. Pattern recognition comprises supervised learning (predefined class labels) and unsupervised learning (unknown class labels). Supervised learning includes choosing a classifier model, training and testing the classifier using a data set and selecting the relevant features. Unsupervised learning is usually approached by cluster analysis. Multiple classifier systems or classifier ensembles are a recent branch of pattern recognition whereby the outputs of several classifiers (regarded as experts) are combined for improved accuracy. Keywords: classification; discriminant analysis; cluster analysis; feature selection; classifier ensembles

When classifiers are deployed in real-world applications, it is assumed that the distribution of the incoming data matches the distribution of the data used to train the classifier. This assumption is often incorrect, which necessitates some form of change detection or adaptive classification. While there has been a lot of work on change detection based on the classification error monitored over the course of the operation of the classifier, finding changes in multidimensional unlabeled data is still a challenge. Here, we propose to apply principal component analysis (PCA) for feature extraction prior to the change detection. Supported by a theoretical example, we argue that the components with the lowest variance should be retained as the extracted features because they are more likely to be affected by a change. We chose a recently proposed semiparametric log-likelihood change detection criterion that is sensitive to changes in both mean and variance of the multidimensional distribution. An experiment with 35 datasets and an illustration with a simple video segmentation demonstrate the advantage of using extracted features compared to raw data. Further analysis shows that feature extraction through PCA is beneficial, specifically for data with multiple balanced classes.

This study brings together systematised views of two related areas: data editing for the nearest neighbour classifier and adaptive learning in the presence of concept drift. The growing number of studies in the intersection of these areas warrants a closer look. We revise and update the taxonomies of the two areas proposed in the literature and argue that they are not sufficiently discriminative with respect to methods for prototype selection and prototype generation in the presence of concept drift. We proceed to create a bespoke taxonomy of these methods and illustrate it with ten examples from the literature. The new taxonomy can serve as a road-map for researching the intersection area and inform the development of new methods.

Event-related potential data can be used to index perceptual and cognitive operations. However, they are typically high-dimensional and noisy. This study examines the original raw data and six feature-extraction methods as a pre-processing step before classification. Four traditionally used feature-extraction methods were considered: principal component analysis, independent component analysis, auto-regression, and wavelets. We add to these a less well-known method called interval feature extraction. It overproduces features from the ERP signal and then eliminates irrelevant and redundant features by the fast correlation-based filter. To make the comparisons fair, the other feature-extraction methods were also run with the filter. An experiment on two EEG datasets (four classification scenarios) was carried out to examine the classification accuracy of four classifiers on the extracted features: support vector machines with linear and perceptron kernel, the nearest neighbour classifier and the random forest ensemble method. The interval features led to the best classification accuracy in most of the configurations, specifically when used with the Random Forest classifier ensemble.

Functional Magnetic Resonance Imaging serves to identify networks and regions in the brain engaged in vari- ous mental activities, represented as a set of voxels in the 3D image. It is important to be able to measure how similar two selected voxel sets are. The major flaw of the currently used correlation-based and overlap-based measures is that they disregard the spatial proximity of the selected voxel sets. Here, we propose a measure for comparing two voxel sets, called Spatial Discrepancy, based upon the average Haus- dorff distance. We demonstrate that Spatial Discrepancy can detect genuine similarities and differences where other com- monly used measures fail to do so. A simulation experiment was carried out where distorted copies of the same voxel sets were compared, varying the level of distortion. The exper- iment revealed that the proposed measure correlates better with the level of distortion than any of the other measures. Data from a 10-subject experiment were used to demonstrate the advantages of the Spatial Discrepancy measure in multi- subject studies.

Functional magnetic resonance imaging (fMRI) is becoming a forefront brain–computer interface tool. To decipher brain patterns, fast, accurate and reliable classifier methods are needed. The support vector machine (SVM) classifier has been traditionally used. Here we argue that state-of-the-art methods from pattern recognition and machine learning, such as classifier ensembles, offer more accurate classification. This study compares 18 classification methods on a publicly available real data set due to Haxby et al. [ Science 293 (2001) 2425–2430]. The data comes from a single-subject experiment, organized in 10 runs where eight classes of stimuli were presented in each run. The comparisons were carried out on voxel subsets of different sizes, selected through seven popular voxel selection methods. We found that, while SVM was robust, accurate and scalable, some classifier ensemble methods demonstrated significantly better performance. The best classifiers were found to be the random subspace ensemble of SVM classifiers, rotation forest and ensembles with random linear and random spherical oracle.

The abundance of unlabelled data alongside limited labelled data has provoked significant interest in semi-supervised learning methods. ''Naive labelling'' refers to the following simple strategy for using unlabelled data in on-line classification. A new data point is first labelled by the current classifier and then added to the training set together with the assigned label. The classifier is updated before seeing the subsequent data point. Although the danger of a run-away classifier is obvious, versions of naive labelling pervade in on-line adaptive learning. We study the asymptotic behaviour of naive labelling in the case of two Gaussian classes and one variable. The analysis shows that if the classifier model assumes correctly the underlying distribution of the problem, naive labelling will drive the parameters of the classifier towards their optimal values. However, if the model is not guessed correctly, the benefits are outweighed by the instability of the labelling strategy (run-away behaviour of the classifier). The results are based on exact calculations of the point of convergence, simulations, and experiments with 25 real data sets. The findings in our study are consistent with concerns about general use of unlabelled data, flagged up in the recent literature.

Scrapie is a neuro-degenerative disease in small ruminants. A data set of 3113 records of sheep reported to the Scrapie Notifications Database in Great Britain has been studied. Clinical signs were recorded as present/absent in each animal by veterinary officials (VO) and a post-mortem diagnosis was made. In an attempt to detect healthy animals within the set of suspects using only the clinical signs, 18 classification methods were applied ranging from simple linear classifiers to classifier ensembles such as Bagging, AdaBoost and Random Forests. The results suggest that the clinical classification by the VO was adequate as no further differentiation within the set of suspects was feasible.

We propose a combined fusion-selection approach to classifier ensemble design. Each classifier in the ensemble is replaced by a miniensemble of a pair of subclassifiers with a random linear oracle to choose between the two. It is argued that this approach encourages extra diversity in the ensemble while allowing for high accuracy of the individual ensemble members. Experiments were carried out with 35 data sets from UCI and 11 ensemble models. Each ensemble model was examined with and without the oracle. The results showed that all ensemble methods benefited from the new approach, most markedly so random subspace and bagging. A further experiment with seven real medical data sets demonstrates the validity of these findings outside the UCI data collection

We aim to dispel the blind faith in theoretical criteria for optimisation of the edited nearest neighbour classifier and its version called the Voronoi classifier. Three criteria from past and recent literature are considered: two bounds using Vapnik-Chervonenkis (VC) dimension and a probabilistic criterion derived by a Bayesian approach. We demonstrate the shortcomings of these criteria for selecting the best reference set, and summarise alternative empirical criteria found in the literature.

We propose a method for generating classifier ensembles based on feature extraction. To create the training data for a base classifier, the feature set is randomly split into K subsets (K is a parameter of the algorithm) and Principal Component Analysis (PCA) is applied to each subset. All principal components are retained in order to preserve the variability information in the data. Thus, K axis rotations take place to form the new features for a base classifier. The idea of the rotation approach is to encourage simultaneously individual accuracy and diversity within the ensemble. Diversity is promoted through the feature extraction for each base classifier. Decision trees were chosen here because they are sensitive to rotation of the feature axes, hence the name "forest." Accuracy is sought by keeping all principal components and also using the whole data set to train each base classifier. Using WEKA, we examined the Rotation Forest ensemble on a random selection of 33 benchmark data sets from the UCI repository and compared it with Bagging, AdaBoost, and Random Forest. The results were favorable to Rotation Forest and prompted an investigation into diversity-accuracy landscape of the ensemble models. Diversity-error diagrams revealed that Rotation Forest ensembles construct individual classifiers which are more accurate than these in AdaBoost and Random Forest, and more diverse than these in Bagging, sometimes more accurate as well.

Wireless capsule endoscopy involves inspection of hours of video material by a highly qualified professional. Time episodes corresponding to intestinal contractions, which are of interest to the physician constitute about 1% of the video. The problem is to label automatically time episodes containing contractions so that only a fraction of the video needs inspection. As the classes of contraction and non-contraction images in the video are largely imbalanced, ROC curves are used to optimize the trade-off between false positive and false negative rates. Classifier ensemble methods and simple classifiers were examined. Our results reinforce the claims from recent literature that classifier ensemble methods specifically designed for imbalanced problems have substantial advantages over simple classifiers and standard classifier ensembles. By using ROC curves with the bagging ensemble method the inspection time can be drastically reduced at the expense of a small fraction of missed contractions.

Supervised classification into c mutually exclusive classes based on n binary features is considered. The only information available is an n×c table with probabilities. Knowing that the best d features are not the d best, simulations were run for 4 feature selection methods and an application to diagnosing BSE in cattle and Scrapie in sheep is presented.

Error-correcting output codes (ECOC) are used to design diverse classifier ensembles. Diversity within ECOC is traditionally measured by Hamming distance. Here we argue that this measure is insufficient for assessing the quality of code for the purposes of building accurate ensembles. We propose to use diversity measures from the literature on classifier ensembles and suggest an evolutionary algorithm to construct the code.

Boosting is recognized as one of the most successful techniques for generating classifier ensembles. Typically, the classifier outputs are combined by the weighted majority vote. The purpose of this study is to demonstrate the advantages of some fuzzy combination methods for ensembles of classifiers designed by Boosting. We ran two-fold cross-validation experiments on six benchmark data sets to compare the fuzzy and nonfuzzy combination methods. On the "fuzzy side" we used the fuzzy integral and the decision templates with different similarity measures. On the "nonfuzzy side" we tried the weighted majority vote as well as simple combiners such as the majority vote, minimum, maximum, average, product, and the Naive-Bayes combination. In our experiments, the fuzzy combination methods performed consistently better than the nonfuzzy methods. The weighted majority vote showed a stable performance, though slightly inferior to the performance of the fuzzy combiners.

In classifier combination, it is believed that diverse ensembles have a better potential for improvement on the accuracy than non-diverse ensembles. We put this hypothesis to a test for two methods for building the ensembles: Bagging and Boosting, with two linear classifier models: the nearest mean classifier and the pseudo-Fisher linear discriminant classifier. To estimate diversity, we apply nine measures proposed in the recent literature on combining classifiers. Eight combination methods were used: minimum, maximum, product, average, simple majority, weighted majority, Naive Bayes and decision templates. We carried out experiments on seven data sets for different sample sizes, different number of classifiers in the ensembles, and the two linear classifiers. Altogether, we created 1364 ensembles by the Bagging method and the same number by the Boosting method. On each of these, we calculated the nine measures of diversity and the accuracy of the eight different combination methods, averaged over 50 runs. The results confirmed in a quantitative way the intuitive explanation behind the success of Boosting for linear classifiers for increasing training sizes, and the poor performance of Bagging in this case. Diversity measures indicated that Boosting succeeds in inducing diversity even for stable classifiers whereas Bagging does not.

This study looks at the relationships between different methods of classifier combination and different measures of diversity. We considered 10 combination methods and 10 measures of diversity on two benchmark data sets. The relationship was sought on ensembles of three classifiers built on all possible partitions of the respective feature sets into subsets of pre-specified sizes. The only positive finding was that the Double-Fault measure of diversity and the measure of difficulty both showed reasonable correlation with Majority Vote and Naive Bayes combinations. Since both these measures have an indirect connection to the ensemble accuracy, this result was not unexpected. However, our experiments did not detect a consistent relationship between the other measures of diversity and the 10 combination methods.

We offer an algorithm for random generation of classifier outputs with specified individual accuracies and pairwise dependencies. The outputs are binary vectors (correct/incorrect classification) for a hypothetical data set. The generated team output can be used to study the majority vote over multiple dependent classifiers.

Affective gaming (AG) is a cross-disciplinary area drawing upon psychology, physiology, electronic engineering and computer science, among others. This paper presents a historical overview of affective gaming, bringing together psychophysiological system developments, a time-line of video game graphical advancements and industry trends, thereby offering an entry point into affective gaming research. It is proposed that video games may soon reach a peak in perceivable graphical improvements. This opens up the door for innovative game enhancement strategies, such as emotiondriven interactions between the player and the gaming environment.

Decision templates (DT) are a technique for classifier fusion for continuous-valued individual classifier outputs. The individual outputs considered here sum up to the same value (e.g., statistical classifiers, yielding some estimates of the posterior probabilities for the classes). First, the DT fusion algorithm is explained. Second, we show that two similarity measures ( S 1 and S 2 ) and two inclusion indices ( I 1 and I 2 ) between fuzzy sets (see Dubois and Prade, Fuzzy Sets and Systems: Theory and Applications, Academic Press, New York, 1980) produce the same DT classifier. The equivalence is proven by showing that for every object submitted for classification, all four measures induce the same ordering on the set of class labels (through DT fusion), thereby assigning the object to the same class.

Multiple classifier fusion may generate more accurate classification than each of the constituent classifiers. Fusion is often based on fixed combination rules like the product and average. Only under strict probabilistic conditions can these rules be justified. We present here a simple rule for adapting the class combiner to the application. c decision templates (one per class) are estimated with the same training set that is used for the set of classifiers. These templates are then matched to the decision profile of new incoming objects by some similarity measure. We compare 11 versions of our model with 14 other techniques for classifier fusion on the Satimage and Phoneme datasets from the database ELENA. Our results show that decision templates based on integral type measures of similarity are superior to the other schemes on both data sets.

We compare eleven methods for finding prototypes upon which to base the nearest prototype classifier. Four methods for prototype selection are discussed: Wilson+Hart (a condensation+error-editing method), and three types of combinatorial search—random search, genetic algorithm, and tabu search. Seven methods for prototype extraction are discussed: unsupervised vector quantization, supervised learning vector quantization (with and without training counters), decision surface mapping, a fuzzy version of vector quantization, c-means clustering, and bootstrap editing. These eleven methods can be usefully divided two other ways: by whether they employ pre- or postsupervision; and by whether the number of prototypes found is user-defined or “automatic.” Generalization error rates of the 11 methods are estimated on two synthetic and two real data sets. Offering the usual disclaimer that these are just a limited set of experiments, we feel confident in asserting that presupervised, extraction methods offer a better chance for success to the casual user than postsupervised, selection schemes. Finally, our calculations do not suggest that methods which find the “best” number of prototypes “automatically” are superior to methods for which the user simply specifies the number of prototypes. © 2001 John Wiley & Sons, Inc.

Nearest neighbor classifiers demand significant computational resources (time and memory). Editing of the reference set and feature selection are two diAerent approaches to this problem. Here we encode the two approaches within the same genetic algorithm (GA) and simultaneously select features and reference cases. Two data sets were used: the SATIMAGE data and a generated data set. The GA was found to be an expedient solution compared to editing followed by feature selection, feature selection followed by editing, and the individual results from feature selection and editing. ” 1999 Elsevier Science B.V. All rights reserved.

We propose a Generalized Nearest Prototype Classifier (GNPC) as a common framework for a number of classification techniques. Specifically we consider clustering-and-relabeling; Parzen's classifier; radial basis functions (RBF) networks; learning vector quantization (LVQ) type classifiers; and nearest neighbor rules. To classify an unlabeled point x the GNPC combines the degrees of similarity of x to a set of prototypes. Five questions are addressed for these GNPC families: (1) How many prototypes do we need? (2) How are the prototypes found? (3) How are their class labels obtained? (4) How are the similarities defined? (5) How are the similarities and label information combined? The classification performance of a set of GNPCs is illustrated on two benchmark data sets: IRIS and the 2-spirals data. We study the resubstitution error of the GNPC as a function of the number of prototypes. Our conclusions are that: (a) unsupervised selection (or extraction) of prototypes followed by relabeling is inferior to the techniques that use labels to guide them towards prototypes; (b) the edited nearest neighbor rule is a viable option for GNPC design which has not received the attention it deserves.

Three questions related to the nearest prototype classifier (NPC) are addressed: when is it better to construct the prototypes instead of selecting them as a subset of the given labeled data; how can we trade classification accuracy for a reduction in the number of prototypes; and how good is pure random search (RS) for selection of prototypes from the data? We compare the resubstitution performance of the NPC on the IRIS data set, where the prototypes are either extracted by replacement (R-prototypes) or by selection (S-prototypes). Results for the R-prototypes are taken from a previous study and are contrasted with S-prototype results obtained by a genetic algorithm (GA) or by RS. The best results reached by both algorithms (GA and RS), followed by resubstitution NPC, are two errors with sets of three S-prototypes. This compares favorably to the best result found with R-prototypes, viz., three errors with five R-prototypes. Based on our results, we recommend GA selection for the NPC. A by-product of this research is a counter example to minimality of a recently published minimal consistent set selection procedure.

An aggregation operator of quantitative opinions about the acceptance or rejection of a certain alternative is proposed. The main idea is to include a degree of consensus between the experts in computing the final value. We aim at strengthening the acceptance or rejection rate if the experts agree in their assessments. The proposed operator is directly applicable to the two-level classification paradigm where a pool of classifiers is used to infer the decision. The operator allows us to form a complex classification boundary in the space of experts' (classifiers') decisions expressed in terms of degrees of membership. The connective can easily be extended to define a classifier with a “refuse-to-decide” option.

A two-level fuzzy pattern recognition scheme is proposed which implements different classification rules at the second level in dependence on the degree of consensus between first-level decision makers. The first-level classification decisions are treated in a fuzzy manner, i.e. each one is supposed to provide class memberships to all classes for each object submitted to classification. The main aim is to increase the classification accuracy of automatic classifiers. The presumption behind the idea is that different rules may appear suitable for the cases of low, medium, or high consensus between first-level decisions and neither of these rules works sufficiently well in all cases. Some implementation details are specified in the text but the real application of the scheme is not confined to them only. A set of experimental results with real data from aviation medicine is presented which show the anticipated increase of the classification accuracy.

Developing accurate, reliable and easy to use diagnostic tests is based upon identifying a small set of highly discriminative biomarkers. This task can be cast as feature selection within a pattern recognition context. Medical data are usually of the "wide" type where the number of features is substantially larger than the number of instances. With the abundance of feature ranking methods, it is difficult to pick the most suitable one and choose a final consistent feature subset. Ensembles of ranking methods have been recommended for the task but their stability and accuracy have not been evaluated across different ranking methods. Here we present a case study consisting of 429 samples of exhaled air from smokers, 83% of whom suffer from COPD (chronic obstructive pulmonary disease). The task is to identify a discriminative subset of the 1929 volatile organic compounds (VOCs) measured through mass spectrometry. Using Pareto analysis, 16 feature ranking ensembles were evaluated with respect to three criteria: classification accuracy, area under the ROC curve and the stability of the ensemble selection. The t-statistic was rated the best among the 16 feature rankers, outperforming the currently favourite SVM ranker.

The author proposes a structure of a two-level fuzzy classifier that can use both data analysis and expert opinion. The correspondence between medical diagnostic tasks and fuzzy pattern recognition is discussed. The description of the scheme, with an experimental study on a real problem from aviation medicine are reported. The results provide experimental support of the improvement of the classification accuracy. >

A way to select a combination of feature subsets serving as inputs for a parallel classifier is described. A genetic algorithm with a properly modified fitness function is used. Experimental results with three sets of real data from internal, neonatal and aviation medicine are reported.

A software system was designed whose aim is to support everyday scientific research of physicians in different fields of medicine. DREAM is a shell-like tool which can be customized embedding in it the desirable structure of a particular medical problem. Various basic statistical analyses are provided along with the decision support capabilities. The decision aid is proposed in two steps — feature selection and classifier design. Genetic algorithm is implemented as the feature selection procedure. The classifier design option includes crisp and fuzzy k -Nearest Neighbors rule and a two-level classification scheme based on majority rule on the votes of several first-level k -Nearest Neighbors classifiers. The system's performance is illustrated with a database from aviation medicine.

Cryptanalysis attempts identify the weaknesses in the algorithms used to encrypt code or the methods used to generate keys. In this study, we use pattern recognition techniques for identification of encryption algorithms for block ciphers. The following block cipher algorithms, DES, IDEA, AES, and RC operating in ECB mode were considered. Eight different classification techniques which are: Naive Bayesian (NB), Support Vector Machine (SVM), neural network (MPL), Instance based learning (IBL), Bagging (Ba), AdaBoostM1, Rotaion Forest (RoFo), Decision Tree (C4.5) were used to identify the cipher text. This study aims to find the best classification algorithm to identify the cipher encryption method. The performance of each of the classifiers was presented, and the simulation results show that, in general, the RoFo classifier has the highest classification accuracy.

The advent of real-time fMRI pattern classification opens many avenues for interactive self-regulation where the brain’s response is better modelled by multivariate, rather than univariate techniques. Here we test three on-line linear classifiers, applied to a real fMRI dataset, collected as part of an experiment on the cortical response to emotional stimuli. We propose a random subspace ensemble as a fast and more accurate alternative to component classifiers. The on-line linear discriminant classifier (O-LDC) was found to be a better base classifier than the on-line versions of the perceptron and the balanced winnow.

Classification of brain images obtained through functional magnetic resonance imaging (fMRI) poses a serious challenge to pattern recognition and machine learning due to the extremely large feature-to-instance ratio. This calls for revision and adaptation of the current state-of-the-art classification methods. We investigate the suitability of the random subspace (RS) ensemble method for fMRI classification. RS samples from the original feature set and builds one (base) classifier on each subset. The ensemble assigns a class label by either majority voting or averaging of output probabilities. Looking for guidelines for setting the two parameters of the method-ensemble size and feature sample size-we introduce three criteria calculated through these parameters: usability of the selected feature sets, coverage of the set of ?important? features, and feature set diversity. Optimized together, these criteria work toward producing accurate and diverse individual classifiers. RS was tested on three fMRI datasets from single-subject experiments: the Haxby data (Haxby, 2001.) and two datasets collected in-house. We found that RS with support vector machines (SVM) as the base classifier outperformed single classifiers as well as some of the most widely used classifier ensembles such as bagging, AdaBoost, random forest, and rotation forest. The closest rivals were the single SVM and bagging of SVM classifiers. We use kappa-error diagrams to understand the success of RS.

Although diversity in classifier ensembles is desirable, its relationship with the ensemble accuracy is not straightforward. Here we derive a decomposition of the majority vote error into three terms: average individual accuracy, “good” diversity and “bad diversity”. The good diversity term is taken out of the individual error whereas the bad diversity term is added to it. We relate the two diversity terms to the majority vote limits defined previously (the patterns of success and failure). A simulation study demonstrates how the proposed decomposition can be used to gain insights about majority vote classifier ensembles.

We consider classification of sequential data in the presence of frequent and abrupt concept changes. The current practice is to use the data after the change to train a new classifier. However, if the window with the new data is too small, the classifier will be undertrained and hence less accurate that the "old'' classifier. Here we propose a method (called WR*) for resizing the training window after detecting a concept change. Experiments with synthetic and real data demonstrate the advantages of WR* over other window resizing methods.

Classical approaches for network traffic classification are based on port analysis and packet inspection. Recent studies indicate that network protocols can be recognised more accurately using the flow statistics of the TCP connection. We propose a classifier selection ensemble for a fast and accurate verification of network protocols. Using the requested port number, the classifier selector directs the decision to an ensemble member responsible for this port. The chosen ensemble member ramifies the decision further using the "sign pattern" of the first four packets. Finally, a decision tree classifier labels the flow as `accepted' or `rejected' using the sizes of the first four packets. The ensemble has modular architecture which allows further modules to be individually trained and added. The classifiers were cross-tested using designated training and testing data of network traffic traces from three institutions. The results show that accuracy need not be sacrificed for speed of classification, and that the protocol classification is robust from one network to another.

Two of the greatest obstacles in medical data processing can be summarized as 'multiclass and noncrisp membership of the patterns' and originate from the fact, that one patient can suffer in different degrees from more than one disease simultaneously. Some encouraging results have been obtained applying to these problems a fuzzy pattern recognition model instead of the purely statistical one. A criterion for evaluation of fuzzy classification accuracy has been developed. In this paper, its general form is presented and appropriately discussed. A composed form of the criterion is suggested, which consists of three components - Integral Index of Classification. Highest Discrepancy Index of Classification and Highest Coincidence Index of Classification. Three graphical examples have been designed in order to clear up the underlying idea. The implementation of the criterion is illustrated by experiments with real data from medical practice.

We extend the nearest prototype classifier to a generalized nearest prototype classifier (GNPC). The GNPC uses "soft" labeling of the prototypes in the classes, thereby encompassing a variety of classifiers. Based on how the prototypes are found we distinguish between presupervised and postsupervised GNPC designs. We derive the conditions for optimality (relative to the standard Bayes error rate) of two designs where prototypes represent: 1) the components of class-conditional mixture densities (presupervised design) or 2) the components of the unconditional mixture density (postsupervised design). An artificial data set and the "satimage" data set from the database ELENA are used to experimentally study the two approaches. A radial basis function (RBF) network is used as a representative of each GNPC type. Neither the theoretical nor the experimental results indicate clear reasons to prefer one of the approaches. The postsupervised GNPC design tends to be more robust and less accurate than the presupervised one.

We propose a strategy for updating the learning rate parameter of online linear classifiers for streaming data with concept drift. The change in the learning rate is guided by the change in a running estimate of the classification error. In addition, we propose an online version of the standard linear discriminant classifier (O-LDC) in which the inverse of the common covariance matrix is updated using the Sherman-Morrison-Woodbury formula. The adaptive learning rate was applied to four online linear classifier models on generated and real streaming data with concept drift. O-LDC was found to be better than balanced Winnow, the perceptron and a recently proposed online linear discriminant analysis.

Simple classifiers, including LDC, have often been praised for their robustness and accuracy. Here we consider an online version of LDC applied to streaming data with concept drift. The classifier is trained on a moving window containing the latest N observations. Current approaches to determining the window size are mostly heuristical. The talk presents a framework within which theoretical relationship can be sought between the window size and the classification error. The framework is based upon two ideas. First, past literature offers formal relationships between the size of the training set and the asymptotic accuracy for several classifier models. Such a relationship can be used as a guide to determining the moving window size. Second, after a sudden change, the "old" classifier may be better than an undertrained "new" classifier that uses only the data coming after the change. We state the optimal window size for the case of LDC applied to two Gaussian classes and a sudden change in the form of rotation and translation of the feature space. Simulation results are included in order to investigate the sensitivity of the theoretical results to violation of the underlying assumptions.