Search

Outlier detection, i.e., the task of detecting points that are markedly different from the data sample, is an important challenge in machine learning. When a model is built, these special points can skew the model training and result in less accurate predictions. Due to this fact, it is important to identify and remove them before building any supervised model and this is often the first step when dealing with a machine learning problem. Nowadays, there exists a very large number of outlier detector algorithms that provide good results, but their main drawbacks are their unsupervised nature together with the hyperparameters that must be properly set for obtaining good performance. In this work, a new supervised outlier estimator is proposed. This is done by pipelining an outlier detector with a following a supervised model, in such a way that the targets of the later supervise how all the hyperparameters involved in the outlier detector are optimally selected. This pipeline-based approach makes it very easy to combine different outlier detectors with different classifiers and regressors. In the experiments done, nine relevant outlier detectors have been combined with three regressors over eight regression problems as well as with two classifiers over another eight binary and multi-class classification problems. The usefulness of the proposal as an objective and automatic way to optimally determine detector hyperparameters has been proven and the effectiveness of the nine outlier detectors has also been analyzed and compared, The authors acknowledge financial support from the European Regional Development Fund and the Spanish State Research Agency of the Ministry of Economy, Industry, and Competitiveness under the projects TIN2016-76406-P (AEI/FEDER, UE) and PID2019-106827GB-I00. They also thank the UAM–ADIC Chair for Data Science and Machine Learning and gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at UAM

It is a common practice in the machine learning community to assume that the observed data are noise-free in the input attributes. Nevertheless, scenarios with input noise are common in real problems, as measurements are never perfectly accurate. If this input noise is not taken into account, a supervised machine learning method is expected to perform sub-optimally. In this paper, we focus on multi-class classification problems and use Gaussian processes (GPs) as the underlying classifier. Motivated by a data set coming from the astrophysics domain, we hypothesize that the observed data may contain noise in the inputs. Therefore, we devise several multi-class GP classifiers that can account for input noise. Such classifiers can be efficiently trained using variational inference to approximate the posterior distribution of the latent variables of the model. Moreover, in some situations, the amount of noise can be known before-hand. If this is the case, it can be readily introduced in the proposed methods. This prior information is expected to lead to better performance results. We have evaluated the proposed methods by carrying out several experiments, involving synthetic and real data. These include several data sets from the UCI repository, the MNIST data set and a data set coming from astrophysics. The results obtained show that, although the classification error is similar across methods, the predictive distribution of the proposed methods is better, in terms of the test log-likelihood, than the predictive distribution of a classifier based on GPs that ignores input noise, We would like to thank M. A. Sanchez-Conde, J. Coronado and V. Gammaldi for pointing our attention to the data set that motivated this work, as well as for the discussions concerning the data extraction. We thank as well E. Fernandez-Martınez, A. Suarez and C. M. Alaız-Gudin for useful discussions and feedback about the work. BZ especially acknowledges the hospitality of the Machine Learning group of UAM during the development of this project. BZ is supported by the Programa Atraccion de Talento de la Comunidad de Madrid under grant n. 2017-T2/TIC-5455, from the Spanish MINECO’s “Centro de Excelencia Severo Ochoa” Programme via grant SEV-2016-0597, and from the Comunidad de Madrid project SI1-PJI-2019-00294, of which BZ is the P.I. The authors gratefully acknowledge the use of the facilities of Centro de Computacion Cientıfica (CCC) at Universidad Autonoma de Madrid. The authors also acknowledge financial support from Spanish Plan Nacional I+D+i, grants TIN2016-76406-P. Finally, the authors acknowledge financial support from PID2019-106827GB-I00/AEI/10.13039/501100011033

Given the impact of renewable sources in the overall energy production, accurate predictions are becoming essential, with machine learning becoming a very important tool in this context. In many situations, the prediction problem can be divided into several tasks, more or less related between them but each with its own particularities. Multitask learning (MTL) aims to exploit this structure, training several models at the same time to improve on the results achievable either by a common model or by task-specific models. In this paper, we show how an MTL approach based on support vector regression can be applied to the prediction of photovoltaic and wind energy, problems where tasks can be defined according to different criteria. As shown experimentally with three different datasets, the MTL approach clearly outperforms the results of the common and specific models for photovoltaic energy, and are at the very least quite competitive for wind energy, The authors acknowledge financial support from Spain’s grants TIN2016-76406-P and PID2019-106827GB- I00/AEI/10.13039/501100011033. Work supported also by The first author is also supported by a FPU–UAM grant

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, The increasing presence of photovoltaic (PV) generation in the energy mix demands improved forecasting tools which can be updated in an almost continuous basis. Satellite-based information lends itself naturally to this purpose and here it is used to nowcast hourly PV energy production for horizons up to six hours over Peninsular Spain and two islands, Majorca in the Mediterranean Sea and Tenerife in the Atlantic Ocean. This paper compares a single model-based on same-day numerical weather prediction (NWP) with hourly refreshed models which either only use satellite-based measurements or which combine both NWP same-day forecasts and satellite data. As shown in the experiments, the satellite-NWP combination gives very good nowcasting results, clearly superior to those achievable separately by either approach, The authors would like to thank Red Elctrica de Espaa for useful discussions and making available PV energy data and gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at UAM, and also would like to thank the Agencia Española de Meteorologa, AEMET, and the ECMWF for access to the MARS repository

Kernel-based techniques have become a common way for describing the local and global relationships of data samples that are generated in real-world processes. In this research, we focus on a multi-scale kernel based technique named Auto-adaptive Laplacian Pyramids (ALP). This method can be useful for function approximation and interpolation. ALP is an extension of the standard Laplacian Pyramids model that incorporates a modified Leave-One-Out Cross Validation procedure, which makes the method stable and automatic in terms of parameters selection without extra cost. This paper introduces a new algorithm that extends ALP to fit datasets that are non-uniformly distributed. In particular, the optimal stopping criterion will be point-dependent with respect to the local noise level and the sample rate. Experimental results over real datasets highlight the advantages of the proposed multi-scale technique for modeling and learning complex, high dimensional data, They wish to thank Prof. Ronald R. Coifman for helpful remarks. They 525 also gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at Universidad Autónoma de Madrid. Funding: This work was supported by Spanish grants of the Ministerio de Ciencia, Innovación y Universidades [grant numbers: TIN2013-42351-P, TIN2015-70308-REDT, TIN2016-76406-P]; project CASI-CAM-CM supported by Madri+d 530 [grant number: S2013/ICE-2845]; project FACIL supported by Fundación BBVA (2016); and the UAM–ADIC Chair for Data Science and Machine Learning

We consider a model of power distribution in a social system where a set of agents plays a simple game on a graph: The probability of winning each round is proportional to the agent’s current power, and the winner gets more power as a result. We show that when the agents are distributed on simple one-dimensional and two-dimensional networks, inequality grows naturally up to a certain stationary value characterized by a clear division between a higher and a lower class of agents. High class agents are separated by one or several lower class agents which serve as a geometrical barrier preventing further flow of power between them. Moreover, we consider the effect of redistributive mechanisms, such as proportional (nonprogressive) taxation. Sufficient taxation will induce a sharp transition towards a more equal society, and we argue that the critical taxation level is uniquely determined by the system geometry. Interestingly, we find that the roughness and Shannon entropy of the power distributions are a very useful complement to the standard measures of inequality, such as the Gini index and the Lorenz curve, We acknowledge financial support from the Spanish Government through Grants No. FIS2015-69167-C2-1-P, No. FIS2015-66020-C2- 1-P, and No. PGC2018-094763-B-I00

Satellite-measured radiances are obviously of great interest for photovoltaic (PV) energy prediction. In this work we will use them together with clear sky irradiance estimates for the nowcasting of PV energy productions over peninsular Spain. We will feed them directly into two linear Machine Learning models, Lasso and linear Support Vector Regression (SVR), and two highly non-linear ones, Deep Neural Networks (in particular, Multilayer Perceptrons, MLPs) and Gaussian SVRs. We shall also use a simple clear sky-based persistence model for benchmarking purposes. We consider prediction horizons of up to 6 h, with Gaussian SVR being statistically better than the other models at each horizon, since its errors increase slowly with time (with an average of 1.92% for the first three horizons and of 2.89% for the last three). MLPs performance is close to that of the Gaussian SVR for the longer horizons (with an average of 3.1%) but less so at the initial ones (average of 2.26%), being nevertheless significantly better than the linear models. As it could be expected, linear models give weaker results (in the initial horizons, Lasso and linear SVR have already an error of 3.21% and 3.46%, respectively), but we will take advantage of the spatial sparsity provided by Lasso to try to identify the concrete areas with a larger influence on PV energy nowcasts., With partial support from Spain’s Grants TIN2013-42351-P, TIN2016-76406-P, TIN2015-70308-REDT and S2013/ICE-2845 CASI-CAM-CM. Work supported also by project FACIL–Ayudas Fundación BBVA a Equipos de Investigación Científica 2016, and the UAM–ADIC Chair for Data Science and Machine Learning. The second author was also supported by the FPU–MEC Grant AP-2012-5163.

This is the author’s version of a work that was accepted for publication in Applied Soft Computing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Applied Soft Computing, Vol 36 (2015) DOI 10.1016/j.asoc.2015.06.053, A memetic approach that combines a genetic algorithm (GA) and quadratic programming is used to address the problem of optimal portfolio selection with cardinality constraints and piecewise linear transaction costs. The framework used is an extension of the standard Markowitz mean–variance model that incorporates realistic constraints, such as upper and lower bounds for investment in individual assets and/or groups of assets, and minimum trading restrictions. The inclusion of constraints that limit the number of assets in the final portfolio and piecewise linear transaction costs transforms the selection of optimal portfolios into a mixed-integer quadratic problem, which cannot be solved by standard optimization techniques. We propose to use a genetic algorithm in which the candidate portfolios are encoded using a set representation to handle the combinatorial aspect of the optimization problem. Besides specifying which assets are included in the portfolio, this representation includes attributes that encode the trading operation (sell/hold/buy) performed when the portfolio is rebalanced. The results of this hybrid method are benchmarked against a range of investment strategies (passive management, the equally weighted portfolio, the minimum variance portfolio, optimal portfolios without cardinality constraints, ignoring transaction costs or obtained with L1 regularization) using publicly available data. The transaction costs and the cardinality constraints provide regularization mechanisms that generally improve the out-of-sample performance of the selected portfolios.

This is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Neurocomputing, VOL 163, (2015) DOI 10.1016/j.neucom.2014.08.090, The growing interest in big data problems implies the need for unsupervised methods for data visualization and dimensionality reduction. Diffusion Maps (DM) is a recent technique that can capture the lower dimensional geometric structure underlying the sample patterns in a way which can be made to be independent of the sampling distribution. Moreover, DM allows us to define an embedding whose Euclidean metric relates to the sample's intrinsic one which, in turn, enables a principled application of k-means clustering. In this work we give a self-contained review of DM and discuss two methods to compute the DM embedding coordinates to new out-of-sample data. Then, we will apply them on two meteorological data problems that involve time and spatial compression of numerical weather forecasts and show how DM is capable to, first, greatly reduce the initial dimension while still capturing relevant information in the original data and, also, how the sample-derived DM embedding coordinates can be extended to new patterns., The authors acknowledge partial support from Spain's grant TIN2010-21575-C02-01 and the UAM{ADIC Chair for Machine Learning. The first author is also supported by an FPI{UAM grant and kindly thanks the Applied Mathematics Department of Yale University for receiving her during her visits.

This is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Neurocomputing, VOL 160 (2015) DOI 10.1016/j.neucom.2014.12.086, Subsampling is used to generate bagging ensembles that are accurate and robust to class-label noise. The effect of using smaller bootstrap samples to train the base learners is to make the ensemble more diverse. As a result, the classification margins tend to decrease. In spite of having small margins, these ensembles can be robust to class-label noise. The validity of these observations is illustrated in a wide range of synthetic and real-world classification tasks. In the problems investigated, subsampling significantly outperforms standard bagging for different amounts of class-label noise. By contrast, the effectiveness of subsampling in random forest is problem dependent. In these types of ensembles the best overall accuracy is obtained when the random trees are built on bootstrap samples of the same size as the original training data. Nevertheless, subsampling becomes more effective as the amount of class-label noise increases., The authors acknowledge financial support from Spanish Plan Nacional I+D+i Grant TIN2013-42351-P and from Comunidad de Madrid Grant S2013/ICE-2845 CASI-CAM-CM.

Many important linear sparse models have at its core the Lasso problem, for which the GLMNet algorithm is often considered as the current state of the art. Recently M. Jaggi has observed that Constrained Lasso (CL) can be reduced to an SVM-like problem, for which the LIBSVM library provides very efficient algorithms. This suggests that it could also be used advantageously to solve CL. In this work we will refine Jaggi’s arguments to reduce CL as well as constrained Elastic Net to a Nearest Point Problem, which in turn can be rewritten as an appropriate ν-SVM problem solvable by LIBSVM. We will also show experimentally that the well-known LIBSVM library results in a faster convergence than GLMNet for small problems and also, if properly adapted, for larger ones. Screening is another ingredient to speed up solving Lasso. Shrinking can be seen as the simpler alternative of SVM to screening and we will discuss how it also may in some cases reduce the cost of an SVM-based CL solution, With partial support from Spanish government grants TIN2013-42351-P, TIN2016-76406-P, TIN2015-70308-REDT and S2013/ICE-2845 CASI-CAM-CM; work also supported by project FACIL–Ayudas Fundación BBVA a Equipos de Investigación Científica 2016 and the UAM–ADIC Chair for Data Science and Machine Learning. The first author is also supported by the FPU–MEC grant AP-2012-5163. We gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at UAM and thank Red Eléctrica de España for kindly supplying wind energy data

General noise cost functions have been recently proposed for support vector regression (SVR). When applied to tasks whose underlying noise distribution is similar to the one assumed for the cost function, these models should perform better than classical є-SVR. On the other hand, uncertainty estimates for SVR have received a somewhat limited attention in the literature until now and still have unaddressed problems. Keeping this in mind, three main goals are addressed here. First, we propose a framework that uses a combination of general noise SVR models with naive online R minimization algorithm (NORMA) as optimization method, and then gives nonconstant error intervals dependent upon input data aided by the use of clustering techniques. We give theoretical details required to implement this framework for Laplace, Gaussian, Beta, Weibull and Marshall–Olkin generalized exponential distributions. Second, we test the proposed framework in two real-world regression problems using data of two public competitions about solar energy. Results show the validity of our models and an improvement over classical є-SVR. Finally, in accordance with the principle of reproducible research, we make sure that data and model implementations used for the experiments are easily and publicly accessible., With partial support from Spain’s grants TIN2013-42351-P, TIN2016-76406-P, TIN2015-70308-REDT, as well as S2013/ICE-2845 CASI-CAM-CM. This work was supported also by project FACIL–Ayudas Fundación BBVA a Equipos de Investigación Científica 2016 and the UAM–ADIC Chair for Data Science and Machine Learning. We gratefully acknowledge the use of the facilities of Centro de Computación Científica, CCC, at Universidad Autónoma de Madrid, UAM

This is the author’s version of a work that was accepted for publication in Pattern Recognition. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition 46.5 (2013): 1323 – 1336, DOI: 10.1016/j.patcog.2012.10.021, We propose to determine the size of a parallel ensemble by estimating the minimum number of classifiers that are required to obtain stable aggregate predictions. Assuming that majority voting is used, a statistical description of the convergence of the ensemble prediction to its asymptotic (infinite size) limit is given. The analysis of the voting process shows that for most test instances the ensemble prediction stabilizes after only a few classifiers are polled. By contrast, a small but non-negligible fraction of these instances require large numbers of classifier queries to reach stable predictions. Specifically, the fraction of instances whose stable predictions require more than T classifiers for T ≫ 1 has a universal form and is proportional to T−1/2. The ensemble size is determined as the minimum number of classifiers that are needed to estimate the infinite ensemble prediction at an average confidence level , close to one. This approach differs from previous proposals, which are based on determining the size for which the prediction error (not the predictions themselves) stabilizes. In particular, it does not require estimates of the generalization performance of the ensemble, which can be unreliable. It has general validity because it is based solely on the statistical description of the convergence of majority voting to its asymptotic limit. Extensive experiments using representative parallel ensembles (bagging and random forest) illustrate the application of the proposed framework in a wide range of classification problems. These experiments show that the optimal ensemble size is very sensitive to the particular classification problem considered., The authors acknowledge financial support from the Spanish Dirección General de Investigación, project TIN2010-21575-C02-02.

This is the author’s version of a work that was accepted for publication in Chemometrics and Intelligent Laboratory Systems. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Chemometrics and Intelligent Laboratory Systems, VOL 157, (2016) DOI 10.1016/j.chemolab.2016.07.004, A method for online decorrelation of chemical sensor signals from the e ects of envi- ronmental humidity and temperature variations is proposed. The goal is to improve the accuracy of electronic nose measurements for continuous monitoring by processing data from simultaneous readings of environmental humidity and temperature. The electronic nose setup built for this study included eight metal-oxide sensors, temperature and humid- ity sensors with a wireless communication link to external computer. This wireless electronic nose was used to monitor air for two years in the residence of one of the authors and it col- lected data continuously during 537 days with a sampling rate of 1 samples per second. To estimate the e ects of variations in air humidity and temperature on the chemical sensors signals, we used a standard energy band model for an n-type metal-oxide (MOX) gas sensor. The main assumption of the model is that variations in sensor conductivity can be expressed as a nonlinear function of changes in the semiconductor energy bands in the presence of external humidity and temperature variations. Fitting this model to the collected data, we con rmed that the most statistically signi cant factors are humidity changes and correlated changes of temperature and humidity. This simple model achieves excellent accuracy with a coe cient of determination R2 close to 1. To show how the humidity-temperature correc- tion model works for gas discrimination, we constructed a model for online discrimination among banana, wine and baseline response. This shows that pattern recognition algorithms improve performance and reliability by including the ltered signal of the chemical sensors., This work has been supported by the California Institute for Telecommunications and Information Technology (CALIT2) under Grant Number 2014CSRO 136. JF acknowledges the support of the Marie Curie Actions and the Agency for Business Competitiveness of the Government of Catalonia (ACCI O) for the grant TECSPR15-1-0031; and the Spanish MINECO program, grant TEC2014-59229-R (SIGVOL). RH, TM, and IR-L acknowledge the partial support by 3a Convocatoria de Proyectos de Cooperacion Interuniversitaria UAM{Banco Santander con EEUU. NR would like to acknowledge partial support by ONR grant N000141612252. TM acknowledges CNPq grant 234817/2014- 3 for partial support. We are also thankful to Flavia Huerta who collected data examples during the summer of 2015.

This is the author’s version of a work that was accepted for publication in Pattern Recognition. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition 44.7 (2011): 1426 – 1434, DOI: 10.1016/j.patcog.2010.12.021, In this paper we introduce a framework for making statistical inference on the asymptotic prediction of parallel classification ensembles. The validity of the analysis is fairly general. It only requires that the individual classifiers are generated in independent executions of some randomized learning algorithm, and that the final ensemble prediction is made via majority voting. Given an unlabeled test instance, the predictions of the classifiers in the ensemble are obtained sequentially. As the individual predictions become known, Bayes’ theorem is used to update an estimate of the probability that the class predicted by the current ensemble coincides with the classification of the corresponding ensemble of infinite size. Using this estimate, the voting process can be halted when the confidence on the asymptotic prediction is sufficiently high. An empirical investigation in several benchmark classification problems shows that most of the test instances require querying only a small number of classifiers to converge to the infinite ensemble prediction with a high degree of confidence. For these instances, the difference between the generalization error of the finite ensemble and the infinite ensemble limit is very small, often negligible., The authors acknowledge support from the Spanish Ministerio de Ciencia e Innovación, projects TIN2007-66862-C02-02 and TIN2010-21575-C02-02.

This is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Neurocomputing 74.12-13 (2011) DOI: 10.1016/j.neucom.2011.03.001, Identifying the optimal subset of regressors in a regression bagging ensemble is a difficult task that has exponential cost in the size of the ensemble. In this article we analyze two approximate techniques especially devised to address this problem. The first strategy constructs a relaxed version of the problem that can be solved using Semidefinite Programming. The second one is based on modifying the order of aggregation of the regressors. Ordered Aggregation is a simple forward selection algorithm that incorporates at each step the regressor that reduces the training error of the current subensemble the most. Both techniques can be used to identify subensembles that are close to the optimal ones, which can be obtained by exhaustive search at a larger computational cost. Experiments in a wide variety of synthetic and real-world regression problems show that pruned ensembles composed of only 20% of the initial regressors often have better generalization performance than the original bagging ensembles. These improvements are due to a reduction in the bias and the covariance components of the generalization error. Subensembles obtained using either SDP or Ordered Aggregation generally outperform subensembles obtained by other ensemble pruning methods and ensembles generated by the Adaboost.R2 algorithm, negative correlation learning or regularized linear stacked generalization. Ordered Aggregation has a slightly better overall performance than SDP in the problems investigated. However, the difference is not statistically significant. Ordered Aggregation has the further advantage that it produces a nested sequence of near-optimal subensembles of increasing size with no additional computational cost., The authors acknowledge support from the Spanish Ministerio de Ciencia e Innovación, Project TIN2010-21575-C02-02.

Rigorous and proper linking of financial databases is a necessary step to test trading strategies incorporating multimodal sources of information. This paper proposes a machine learning solution to match companies in heterogeneous financial databases. Our method, named Financial Attribute Selection Distance (FASD), has two stages, each of them corresponding to one of the two interrelated tasks commonly involved in heterogeneous database matching problems: schema matching and entity matching. FASD's schema matching procedure is based on the Kullback-Leibler divergence of string and numeric attributes. FASD's entity matching solution relies on learning a company distance flexible enough to deal with the numeric and string attribute links found by the schema matching algorithm and incorporate different string matching approaches such as edit-based and token-based metrics. The parameters of the distance are optimized using the F-score as cost function. FASD is able to match the joint COMPUSTAT/CRSP and Institutional Brokers' Estimate System (I/B/E/S) databases with a F-score over 0.94 using only a hundred of manually labeled company links., This research is based upon work supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the Federal Bureau of Investigations, Finance Division. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. P. Martín and A. Sierra, "Improving Power System Static Security Margins by Means of a Real Coded Genetic Algorithm," in IEEE Transactions on Power Systems, vol. 31, no. 3, pp. 1915-1924, May 2016. doi: 10.1109/TPWRS.2015.2439579, This paper introduces a new method of removing thermal overloads and voltage limits in an electric power system by means of the Evolution of Corrective and Preventive control Actions (ECPA). The goal is to find the minimum number of control actions that solve the identified limit violations at minimum cost. A recombination operator based on form theory allows the codification of control actions in a natural and simple way. ECPA has been tested on the IEEE 30-bus and the IEEE 118-bus systems. The limit violations are solved at minimum cost and with fewer control actions on average than alternative methods.

This is the author’s version of a work that was accepted for publication in Pattern Recognition Letters. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition Letters 28.1 (2007): 156 – 165, DOI: 10.1016/j.patrec.2006.06.018, Boosting is used to determine the order in which classifiers are aggregated in a bagging ensemble. Early stopping in the aggregation of the classifiers in the ordered bagging ensemble allows the identification of subensembles that require less memory for storage, classify faster and can improve the generalization accuracy of the original bagging ensemble. In all the classification problems investigated pruned ensembles with 20 % of the original classifiers show statistically significant improvements over bagging. In problems where boosting is superior to bagging, these improvements are not sufficient to reach the accuracy of the corresponding boosting ensembles. However, ensemble pruning preserves the performance of bagging in noisy classification tasks, where boosting often has larger generalization errors. Therefore, pruned bagging should generally be preferred to complete bagging and, if no information about the level of noise is available, it is a robust alternative to AdaBoost., The authors acknowledge financial support from the Spanish Dirección General de Investigación, project TIN2004-07676-C02-02.

The concept of pointwise Fisher consistency (or classification calibration) states necessary and sufficient conditions to have Bayes consistency when a classifier minimizes a surrogate loss function instead of the 0-1 loss. We present a family of multiclass hinge loss functions defined by a continuous control parameter. representing the margin of the positive points of a given class. The parameter. allows shifting from classification uncalibrated to classification calibrated loss functions. Though previous results suggest that increasing the margin of positive points has positive effects on the classification model, other approaches have failed to give increasing weight to the positive examples without losing the classification calibration property. Our lambda-based loss function can give unlimited weight to the positive examples without breaking the classification calibration property. Moreover, when embedding these loss functions into the Support Vector Machine's framework (lambda-SVM), the parameter. defines different regions for the Karush-Kuhn-Tucker conditions. A large margin on positive points also facilitates faster convergence of the Sequential Minimal Optimization algorithm, leading to lower training times than other classification calibrated methods. lambda-SVM allows easy implementation, and its practical use in different datasets not only supports our theoretical analysis, but also provides good classification performance and fast training times., The authors acknowledge the referees' comments and suggestions that helped to improve the manuscript. This research is based upon work supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the Federal Bureau of Investigations, Finance Division. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. I.R-L acknowledges partial support by Spain's grants TIN2013-42351-P (MINECO) and S2013/ICE-2845 CASI-CAM-CM (Comunidad de Madrid). The authors gratefully acknowledge the use of the facilities of Centro de Computacion Cientifica (CCC) at Universidad Autonoma de Madrid.

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. G. Martínez-Muñoz and E. Pulido, "Using a SPOC to flip the classroom," 2015 IEEE Global Engineering Education Conference (EDUCON), Tallinn, 2015, pp. 431-436. doi: 10.1109/EDUCON.2015.7096007, The benefits of using SPOC platforms into face-to-face education are yet to be completely analysed. In this work we propose to use SPOCs and video contents to flip the classroom. The objective is to improve the involvement and satisfaction of students with the course, to reduce the drop-out rates and to improve the face-to-face course success rate. We apply these ideas to an undergraduate first year course on Data Structures and Algorithms. The study is validated by collecting data from two consecutive editions of the course, one in which the flipped classroom model and videos were used and other in which they were not. The gathered data included online data about the students' interaction with the SPOC materials and offline data collected during lectures and exams. In the edition where the SPOC materials were available, we have observed a correlation between the students' final marks and their percentage rate of video accesses with respect to the total number of accesses, which indicates a better academic performance for students who prefer videos over documents., Authors acknowledge financial support from the Spanish Dirección General de Investigación project TIN2013-42351-P and from Comunidad de Madrid project CASI-CAM S2013/ICE-2845.

This is the author’s version of a work that was accepted for publication in Pattern Recognition. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition 38.10 (2005): 1483 – 1494, DOI: 10.1016/j.patcog.2005.02.020, Ensembles that combine the decisions of classi ers generated by using perturbed versions of the training set where the classes of the training examples are randomly switched can produce a signi cant error reduction, provided that large numbers of units and high class switching rates are used. The classi ers generated by this procedure have statistically uncorrelated errors in the training set. Hence, the ensembles they form exhibit a similar dependence of the training error on ensemble size, independently of the classi cation problem. In particular, for binary classi cation problems, the classi cation performance of the ensemble on the training data can be analysed in terms of a Bernoulli process. Experiments on several UCI datasets demonstrate the improvements in classi cation accuracy that can be obtained using these class-switching ensembles., This work has been supported by the Spanish "Dirección General de Investigación", project TIC2001-0572-C02-02.

The dataset includes the acquired time series of a chemical detection platform exposed to different gas conditions in a turbulent wind tunnel. The chemo-sensory elements were sampling directly the environment. In contrast to traditional approaches that include measurement chambers, open sampling systems are sensitive to dispersion mechanisms of gaseous chemical analytes, namely diffusion, turbulence, and advection, making the identification and monitoring of chemical substances more challenging. The sensing platform included 72 metal-oxide gas sensors that were positioned at 6 different locations of the wind tunnel. At each location, 10 distinct chemical gases were released in the wind tunnel, the sensors were evaluated at 5 different operating temperatures, and 3 different wind speeds were generated in the wind tunnel to induce different levels of turbulence. Moreover, each configuration was repeated 20 times, yielding a dataset of 18,000 measurements. The dataset was collected over a period of 16 months. The data is related to "On the performance of gas sensor arrays in open sampling systems using Inhibitory Support Vector Machines", by Vergara et al.[1]. The dataset can be accessed publicly at the UCI repository upon citation of [1]: http://archive.ics.uci.edu/ml/datasets/Gas+sensor+arrays+in+open+sampling+settings., This work has been supported by the California Institute for Telecommunications and Information Technology (CALIT2) under Grant number 2014 CSRO 136.

Multi-task feature selection methods often make the hypothesis that learning tasks share relevant and irrelevant features. However, this hypothesis may be too restrictive in practice. For example, there may be a few tasks with specific relevant and irrelevant features (outlier tasks). Similarly, a few of the features may be relevant for only some of the tasks (outlier features). To account for this, we propose a model for multi-task feature selection based on a robust prior distribution that introduces a set of binary latent variables to identify outlier tasks and outlier features. Expectation propagation can be used for efficient approximate inference under the proposed prior. Several experiments show that a model based on the new robust prior provides better predictive performance than other benchmark methods., Daniel Hernández-Lobato gratefully acknowledges the use of the facilities of Centro de Computacin Científica (CCC) at Universidad Autónoma de Madrid. This author also acknowledges financial support from Spanish Plan Nacional I+D+i, Grant TIN2013-42351-P, and from Comunidad de Madrid, Grant S2013/ICE-2845 CASI-CAM-CM. José Miguel Hernández-Lobato acknowledges financial support from the Rafael del Pino Fundation.

To address drift in chemical sensing, an extensive dataset was collected over a period of three years. An array of 16 metal-oxide gas sensors was exposed to six different volatile organic compounds at different concentration levels under tightly-controlled operating conditions. Moreover, the generated dataset is suitable to tackle a variety of challenges in chemical sensing such as sensor drift, sensor failure or system calibration. The data is related to "Chemical gas sensor drift compensation using classifier ensembles", by Vergara et al. [1], and "On the calibration of sensor arrays for pattern recognition using the minimal number of experiments", by Rodriguez-Lujan et al. [2] The dataset can be accessed publicly at the UCI repository upon citation of: http://archive.ics.uci.edu/ml/datasets/Gas+Sensor+Array+Drift+Dataset+at+Different+Concentrations., This work has been supported by the California Institute for Telecommunications and Information Technology (CALIT2) under Grant number 2014 CSRO 136.

Lauwerys BR, Hernández-Lobato D, Gramme P, Ducreux J, Dessy A, Focant I, et al. (2015) Heterogeneity of Synovial Molecular Patterns in Patients with Arthritis. PLoS ONE 10(4): e0122104. doi:10.1371/journal.pone.0122104, Objectives: Early diagnosis of rheumatoid arthritis (RA) is an unmet medical need in the field of rheumatology. Previously, we performed high-density transcriptomic studies on synovial biopsies from patients with arthritis, and found that synovial gene expression profiles were significantly different according to the underlying disorder. Here, we wanted to further explore the consistency of the gene expression signals in synovial biopsies of patients with arthritis, using low-density platforms. Methods: Low-density assays (cDNA microarray and microfluidics qPCR) were designed, based on the results of the high-density microarray data. Knee synovial biopsies were obtained from patients with RA, spondyloarthropathies (SA) or osteoarthritis (OA) (n = 39), and also from patients with initial undifferentiated arthritis (UA) (n = 49). Results: According to high-density microarray data, several molecular pathways are differentially expressed in patients with RA, SA and OA: T and B cell activation, chromatin remodelling, RAS GTPase activation and extracellular matrix regulation. Strikingly, disease activity (DAS28-CRP) has a significant influence on gene expression patterns in RA samples. Using the low-density assays, samples from patients with OA are easily discriminated from RA and SA samples. However, overlapping molecular patterns are found, in particular between RA and SA biopsies. Therefore, prediction of the clinical diagnosis based on gene expression data results in a diagnostic accuracy of 56.8%, which is increased up to 98.6%by the addition of specific clinical symptoms in the prediction algorithm. Similar observations are made in initial UA samples, in which overlapping molecular patterns also impact the accuracy of the diagnostic algorithm. When clinical symptoms are added, the diagnostic accuracy is strongly improved. Conclusions: Gene expression signatures are overall different in patients with OA, RA and SA, but overlapping molecular signatures are found in patients with these conditions. Therefore, an accurate diagnosis in patients with UA requires a combination of gene expression and clinical data., This work was supported by grants from the Région Wallonne (BioWin), and the Fondation Saint-Luc (Cliniques Universitaires Saint-Luc, Brussels, Belgium). JD is funded by a unrestricted UCB grant (« Chaire UCB sur les rhumatismes inflammatoires et systémiques »). BRL is a part-time research fellow (« Clinicien-chercheur ») of the Fonds National de la Recherche Scientifique (Communauté française de Belgique). DNAlytics provided support in the form of salaries for authors PG and TH, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.

A chemical detection platform composed of 8 chemo-resistive gas sensors was exposed to turbulent gas mixtures generated naturally in a wind tunnel. The acquired time series of the sensors are provided. The experimental setup was designed to test gas sensors in realistic environments. Traditionally, chemical detection systems based on chemo-resistive sensors include a gas chamber to control the sample air flow and minimize turbulence. Instead, we utilized a wind tunnel with two independent gas sources that generate two gas plumes. The plumes get naturally mixed along a turbulent flow and reproduce the gas concentration fluctuations observed in natural environments. Hence, the gas sensors can capture the spatio-temporal information contained in the gas plumes. The sensor array was exposed to binary mixtures of ethylene with either methane or carbon monoxide. Volatiles were released at four different rates to induce different concentration levels in the vicinity of the sensor array. Each configuration was repeated 6 times, for a total of 180 measurements. The data is related to "Chemical Discrimination in Turbulent Gas Mixtures with MOX Sensors Validated by Gas Chromatography-Mass Spectrometry", by Fonollosa et al. [1]. The dataset can be accessed publicly at the UCI repository upon citation of [1]: http://archive.ics.uci.edu/ml/datasets/Gas+senso+rarray+exposed+to+turbulent+gas+mixtures., This work has been supported by the California Institute for Telecommunications and Information Technology (CALIT2) under Grant Number 2014 CSRO 136.

This is the author’s version of a work that was accepted for publication in Pattern Recognition Letters. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition Letters, 19, 8, (1998) DOI: 10.1016/S0167-8655(98)00042-7, Surprisingly simple local learning algorithms are known to outperform many other global non linear machines Unfortunately these algorithms are computationally costly A means of assembling both learning ap proaches is proposed in this letter and shown to enhance performance

The final publication is available at Springer via http://dx.doi.org/10.1007/s11042-011-0913-z, Spoken term detection (STD) is the task of searching for occurrences of spoken terms in audio archives. It relies on robust confidence estimation to make a hit/false alarm (FA) decision. In order to optimize the decision in terms of the STD evaluation metric, the confidence has to be discriminative. Multi-layer perceptrons (MLPs) and support vector machines (SVMs) exhibit good performance in producing discriminative confidence; however they are severely limited by the continuous objective functions, and are therefore less capable of dealing with complex decision tasks. This leads to a substantial performance reduction when measuring detection of out-of-vocabulary (OOV) terms, where the high diversity in term properties usually leads to a complicated decision boundary. In this paper we present a new discriminative confidence estimation approach based on evolutionary discriminant analysis (EDA). Unlike MLPs and SVMs, EDA uses the classification error as its objective function, resulting in a model optimized towards the evaluation metric. In addition, EDA combines heterogeneous projection functions and classification strategies in decision making, leading to a highly flexible classifier that is capable of dealing with complex decision tasks. Finally, the evolutionary strategy of EDA reduces the risk of local minima. We tested the EDA-based confidence with a state-of-the-art phoneme-based STD system on an English meeting domain corpus, which employs a phoneme speech recognition system to produce lattices within which the phoneme sequences corresponding to the enquiry terms are searched. The test corpora comprise 11 hours of speech data recorded with individual head-mounted microphones from 30 meetings carried out at several institutes including ICSI; NIST; ISL; LDC; the Virginia Polytechnic Institute and State University; and the University of Edinburgh. The experimental results demonstrate that EDA considerably outperforms MLPs and SVMs on both classification and confidence measurement in STD, and the advantage is found to be more significant on OOV terms than on in-vocabulary (INV) terms. In terms of classification performance, EDA achieved an equal error rate (EER) of 11% on OOV terms, compared to 34% and 31% with MLPs and SVMs respectively; for INV terms, an EER of 15% was obtained with EDA compared to 17% obtained with MLPs and SVMs. In terms of STD performance for OOV terms, EDA presented a significant relative improvement of 1.4% and 2.5% in terms of average term-weighted value (ATWV) over MLPs and SVMs respectively., This work was partially supported by the French Ministry of Industry (Innovative Web call) under contract 09.2.93.0966, ‘Collaborative Annotation for Video Accessibility’ (ACAV) and by ‘The Adaptable Ambient Living Assistant’ (ALIAS) project funded through the joint national Ambient Assisted Living (AAL) programme.

This is an electronic version of the paper presented at the 27 Annual Conference on Neural Information Processing Systems, held in Lake Tahoe on 2013, A probabilistic model based on the horseshoe prior is proposed for learning dependencies in the process of identifying relevant features for prediction. Exact inference is intractable in this model. However, expectation propagation offers an approximate alternative. Because the process of estimating feature selection dependencies may suffer from over-fitting in the model proposed, additional data from a multi-task learning scenario are considered for induction. The same model can be used in this setting with few modifications. Furthermore, the assumptions made are less restrictive than in other multi-task methods: The different tasks must share feature selection dependencies, but can have different relevant features and model coefficients. Experiments with real and synthetic data show that this model performs better than other multi-task alternatives from the literature. The experiments also show that the model is able to induce suitable feature selection dependencies for the problems considered, only from the training data.

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-38679-4_9, Proceedings of 12th International Work-Conference on Artificial Neural Networks, IWANN 2013, Puerto de la Cruz, Tenerife, Spain, June 12-14, 2013, Part I, The prediction and management of wind power ramps is currently receiving large attention as it is a crucial issue for both system operators and wind farm managers. However, this is still an issue far from being solved and in this work we will address it as a classification problem working with delay vectors of the wind power time series and applying local Mahalanobis K-NN search with metrics derived from Anisotropic Diffusion methods. The resulting procedures clearly outperform a random baseline method and yield good sensitivity but more work is needed to improve on specificity and, hence, precision., With partial support from Spain's grant TIN2010-21575- C02-01 and the UAM-ADIC Chair for Machine Learning. The rst author is also supported by an FPI-UAM grant and kindly thanks the Applied Mathematics Department of Yale University for receiving her during her visits. The second author is supported by the FPU-MEC grant AP2008-00167.

This is an electronic version of the paper presented at the Annual Conference on Neural Information Processing Systems, held in Lake Tahoe on 2013, The estimation of dependencies between multiple variables is a central problem in the analysis of financial time series. A common approach is to express these dependencies in terms of a copula function. Typically the copula function is assumed to be constant but this may be innacurate when there are covariates that could have a large influence on the dependence structure of the data. To account for this, a Bayesian framework for the estimation of conditional copulas is proposed. In this framework the parameters of a copula are non-linearly related to some arbitrary conditioning variables. We evaluate the ability of our method to predict time-varying dependencies on several equities and currencies and observe consistent performance gains compared to static copula models and other time-varying copula methods., We thank David López-Paz and Andrew Gordon Wilson for interesting discussions. José Miguel Hernández-Lobato acknowledges support from Infosys Labs, Infosys Limited. Daniel Hernández- Lobato acknowledges support from the Spanish Dirección General de Investigación, project ALLS (TIN2010-21575-C02-02).

We describe a Bayesian method for group feature selection in linear regression problems. The method is based on a generalized version of the standard spike-and-slab prior distribution which is often used for individual feature selection. Exact Bayesian inference under the prior considered is infeasible for typical regression problems. However, approximate inference can be carried out efficiently using Expectation Propagation (EP). A detailed analysis of the generalized spike-and-slab prior shows that it is well suited for regression problems that are sparse at the group level. Furthermore, this prior can be used to introduce prior knowledge about specific groups of features that are a priori believed to be more relevant. An experimental evaluation compares the performance of the proposed method with those of group LASSO, Bayesian group LASSO, automatic relevance determination and additional variants used for group feature selection. The results of these experiments show that a model based on the generalized spike-and-slab prior and the EP algorithm has state-of-the-art prediction performance in the problems analyzed. Furthermore, this model is also very useful to carry out sequential experimental design (also known as active learning), where the data instances that are most informative are iteratively included in the training set, reducing the number of instances needed to obtain a particular level of prediction accuracy., Daniel Hernández-Lobato and Pierre Dupont acknowledge support from the Spanish Dirección General de Investigación, project ALLS (TIN2010-21575-C02-02).

Theoretical studies of localization, anomalous diffusion and ergodicity breaking require solving the electronic structure of disordered systems. We use free probability to approximate the ensemble- averaged density of states without exact diagonalization. We present an error analysis that quantifies the accuracy using a generalized moment expansion, allowing us to distinguish between different approximations. We identify an approximation that is accurate to the eighth moment across all noise strengths, and contrast this with the perturbation theory and isotropic entanglement theory., Comment: 5 pages, 3 figures, submitted to Phys. Rev. Lett

This article is published in International Journal of Nanomedicine, Issue 7, 2012, 623-630., The engineering of surface patterns is a powerful tool for analyzing cellular communication factors involved in the processes of adhesion, migration, and expansion, which can have a notable impact on therapeutic applications including tissue engineering. In this regard, the main objective of this research was to fabricate patterned and textured surfaces at micron- and nanoscale levels, respectively, with very different chemical and topographic characteristics to control cell–substrate interactions. For this task, one-dimensional (1-D) and two-dimensional (2-D) patterns combining silicon and nanostructured porous silicon were engineered by ion beam irradiation and subsequent electrochemical etch. The experimental results show that under the influence of chemical and morphological stimuli, human mesenchymal stem cells polarize and move directionally toward or away from the particular stimulus. Furthermore, a computational model was developed aiming at understanding cell behavior by reproducing the surface distribution and migration of human mesenchymal stem cells observed experimentally.

This is an electronic version of the paper presented at the European Symposium on Artificial Neural Networks, held in Bruges on 2012, In randomized parallel ensembles the class label predictions for a particular instance by different ensemble classifiers are independent random variables. Taking advantage of this independence we design a statistical test to identify instances near the decision borders, which are difficult to classify because of their proximity to these borders. For these instances, the performance of the ensemble is poor and approaches random guessing. The validity of this analysis and the usefulness of the proposed statistical test are illustrated in several real-world classification problems.

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-33266-1_70, Proceedings of 22nd International Conference on Artificial Neural Networks, Lausanne, Switzerland, September 11-14, 2012, In this work we will apply Diffusion Maps (DM), a recent technique for dimensionality reduction and clustering, to build local models for wind energy forecasting. We will compare ridge regression models for K–means clusters obtained over DM features, against the models obtained for clusters constructed over the original meteorological data or principal components, and also against a global model. We will see that a combination of the DM model for the low wind power region and the global model elsewhere outperforms other options., With partial support from grant TIN2010-21575-C02-01 of Spain’s Ministerio de Economía y Competitividad and the UAM–ADIC Chair for Machine Learning in Modelling and Prediction. The first author is also supported by an FPI-UAM grant and kindly thanks the Applied Mathematics Department of Yale University for receiving her during a visit. The second author is supported by the FPU-MEC grant AP2008-00167. We also thank Red Eléctrica de España, Spain’s TSO, for providing historic wind energy data

The aim of a crisp cluster validity index is to quantify the quality of a given data partition. It allows to select the best partition out of a set of potential ones, and to determine the number of clusters. Recently, negentropy-based cluster validation has been introduced. This new approach seems to perform better than other state of the art techniques, and its computation is quite simple. However, like many other cluster validation approaches, it presents problems when some partition regions have a small number of points. Different heuristics have been proposed to cope with this problem. In this article we systematically analyze the performance of different negentropy-based validation approaches, including a new heuristic, in clustering problems of increasing dimensionality, and compare them to reference criteria such as AIC and BIC. Our results on synthetic data suggest that the newly proposed negentropy-based validation strategy can outperform AIC and BIC when the ratio of the number of points to the dimension is not high, which is a very common situation in most real applications., The authors thank the financial support from DGUI-CAM/UAM (Project CCG10-UAM/TIC-5864)

The transition between a regime in which thermodynamic relations apply only to ensembles of small systems coupled to a large environment and a regime in which they can be used to characterize individual macroscopic systems is analyzed in terms of the change in behavior of the Jarzynski estimator of equilibrium free energy differences from nonequilibrium work measurements. Given a fixed number of measurements, the Jarzynski estimator is unbiased for sufficiently small systems. In these systems, the directionality of time is poorly defined and configurations that dominate the empirical average, but which are in fact typical of the reverse process, are sufficiently well sampled. As the system size increases the arrow of time becomes better defined. The dominant atypical fluctuations become rare and eventually cannot be sampled with the limited resources that are available. Asymptotically, only typical work values are measured. The Jarzynski estimator becomes maximally biased and approaches the exponential of minus the average work, which is the result that is expected from standard macroscopic thermodynamics. In the proper scaling limit, this regime change can be described in terms of a phase transition in variants of the random energy model (REM). This correspondence is explicitly demonstrated in several examples of physical interest: near-equilibrium processes in which the work distribution is Gaussian, the sudden compression of an ideal gas and adiabatic quasi-static volume changes in a dilute real gas., Comment: 29 pages, 5 figures, accepted for publication in Physical Review E (2012)

This is an electronic version of the paper presented at the 19th European Symposium on Artificial Neural Networks, held in Bruges on 2011, Least-Squares Support Vector Machines (LS-SVMs) have been successfully applied in many classification and regression tasks. Their main drawback is the lack of sparseness of the final models. Thus, a procedure to sparsify LS-SVMs is a frequent desideratum. In this paper, we adapt to the LS-SVM case a recent work for sparsifying classical SVM classifiers, which is based on an iterative approximation to the L0-norm. Experiments on real-world classification and regression datasets illustrate that this adaptation achieves very sparse models, without significant loss of accuracy compared to standard LS-SVMs or SVMs.

Identifying a subset of features that preserves classification accuracy is a problem of growing importance, because of the increasing size and dimensionality of real-world data sets. We propose a new feature selection method, named Quadratic Programming Feature Selection (QPFS), that reduces the task to a quadratic optimization problem. In order to limit the computational complexity of solving the optimization problem, QPFS uses the Nystr¨om method for approximate matrix diagonalization. QPFS is thus capable of dealing with very large data sets, for which the use of other methods is computationally expensive. In experiments with small and medium data sets, the QPFS method leads to classification accuracy similar to that of other successful techniques. For large data sets, QPFS is superior in terms of computational efficiency., I.R.-L. is supported by an FPU grant from Universidad Autónoma de Madrid, and partially supported by the Universidad Autónoma de Madrid-IIC Chair. R.H. acknowledges partial support by ONR N00014-07-1-0741

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-12127-2_11, Proceedings of 9th International Workshop, MCS 2010, Cairo, Egypt, April 7-9, 2010., This article introduces a double pruning algorithm that can be used to reduce the storage requirements, speed-up the classification process and improve the performance of parallel ensembles. A key element in the design of the algorithm is the estimation of the class label that the ensemble assigns to a given test instance by polling only a fraction of its classifiers. Instead of applying this form of dynamical (instance-based) pruning to the original ensemble, we propose to apply it to a subset of classifiers selected using standard ensemble pruning techniques. The pruned subensemble is built by first modifying the order in which classifiers are aggregated in the ensemble and then selecting the first classifiers in the ordered sequence. Experiments in benchmark problems illustrate the improvements that can be obtained with this technique. Specifically, using a bagging ensemble of 101 CART trees as a starting point, only the 21 trees of the pruned ordered ensemble need to be stored in memory. Depending on the classification task, on average, only 5 to 12 of these 21 classifiers are queried to compute the predictions. The generalization performance achieved by this double pruning algorithm is similar to pruned ordered bagging and significantly better than standard bagging.

This is an electronic version of the paper presented at the 18th European Symposium on Artificial Neural Networks, held in Bruges on 2010, Least Squares Support Vector Machines (LS-SVMs) were proposed by replacing the inequality constraints inherent to L1-SVMs with equality constraints. So far this idea has only been suggested for a least squares (L2) loss. We describe how this can also be done for the sumof-slacks (L1) loss, yielding a new classifier (Least 1-Norm SVMs) which gives similar models in terms of complexity and accuracy and that may also be more robust than LS-SVMs with respect to outliers., With partial support of Spain’s TIN 2007–66862 project and Cátedra IIC en Modelado y Predicción. The first author is kindly supported by FPU-MICINN grant reference AP2007– 00142.

This is the author’s version of a work that was accepted for publication in Pattern Recognition. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Pattern Recognition 43.1 (2010): 143 – 152 , DOI: 10.1016/j.patcog.2009.05.010, The performance of m-out-of-n bagging with and without replacement in terms of the sampling ratio (m/n) is analyzed. Standard bagging uses resampling with replacement to generate bootstrap samples of equal size as the original training set mwor = n. Without-replacement methods typically use half samples mwr = n/2. These choices of sampling sizes are arbitrary and need not be optimal in terms of the classification performance of the ensemble. We propose to use the out-of-bag estimates of the generalization accuracy to select a near-optimal value for the sampling ratio. Ensembles of classifiers trained on independent samples whose size is such that the out-of-bag error of the ensemble is as low as possible generally improve the performance of standard bagging and can be efficiently built.

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-04274-4_10, Proceedings of 19th International Conference, Limassol, Cyprus, September 14-17, 2009., Recent research has shown that the provisional count of votes of an ensemble of classifiers can be used to estimate the probability that the final ensemble prediction coincides with the current majority class. For a given instance, querying can be stopped when this probability is above a specified threshold. This instance-based ensemble pruning procedure can be efficiently implemented if these probabilities are pre-computed and stored in a lookup table. However, the size of the table and the cost of computing the probabilities grow very rapidly with the number of classes of the problem. In this article we introduce a number of computational optimizations that can be used to make the construction of the lookup table feasible. As a result, the application of instance-based ensemble pruning is extended to multi-class problems. Experiments in several UCI multi-class problems show that instance-based pruning speeds-up classification by a factor between 2 and 10 without any significant variation in the prediction accuracy of the ensemble.

This is an electronic version of the paper presented at the 17th European Symposium on Artificial Neural Networks, held in Bruges on 2009, In this work we will give explicit formulae for the application of Rosen’s gradient projection method to SVM training that leads to a very simple implementation. We shall experimentally show that the method provides good descent directions that result in less training iterations, particularly when large precision is wanted. However, a naive kernelization may end up in a procedure requiring more KOs than SMO and further work is needed to arrive at an efficient implementation., With partial support of Spain’s TIN 2007–66862 project and Cátedra UAM–IIC en Modelado y Predicción. The first author is kindly supported by FPU-MICINN grant reference AP2007–00142.

This is an electronic version of the paper presented at the European Wind Energy Conference and Exhibition, held in Marseille on 2009, High and low production regimes are in principle different enough as to warrant the use of regime–specific models for the prediction of wind energy production. The simplest way to identify them may be the use of concrete wind/production thresholds. The computation of these thresholds requires an estimate of the most likely future regime and in this work we consider both NWP wind speed forecasts and also the production forecasts of a global full operation range model. As we shall illustrate over the aggregated wind energy production of a very large area of Spain, a production threshold–based approach gives consistently better results than the alternative, wind speed based, procedure.

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. G. Martínez-Muñoz, D. Hernández-Lobato and A. Suárez, "An analysis of ensemble pruning techniques based on ordered aggregation", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 245-249, February 2009, Several pruning strategies that can be used to reduce the size and increase the accuracy of bagging ensembles are analyzed. These heuristics select subsets of complementary classifiers that, when combined, can perform better than the whole ensemble. The pruning methods investigated are based on modifying the order of aggregation of classifiers in the ensemble. In the original bagging algorithm, the order of aggregation is left unspecified. When this order is random, the generalization error typically decreases as the number of classifiers in the ensemble increases. If an appropriate ordering for the aggregation process is devised, the generalization error reaches a minimum at intermediate numbers of classifiers. This minimum lies below the asymptotic error of bagging. Pruned ensembles are obtained by retaining a fraction of the classifiers in the ordered ensemble. The performance of these pruned ensembles is evaluated in several benchmark classification tasks under different training conditions. The results of this empirical investigation show that ordered aggregation can be used for the efficient generation of pruned ensembles that are competitive, in terms of performance and robustness of classification, with computationally more costly methods that directly select optimal or near-optimal subensembles., The authors acknowledge support form the Spanish Ministerio de Educación y Ciencia under Project TIN2007-66862-C02-02

In this paper we introduce a fast algorithm that can detect the most unusual part of a digital image. The most unusual part of a given shape is de ned as a part of the image that has the maximal distance to all non intersecting shapes with the same form. The method is tested on two and three-dimensional images and have shown very good results without any prede ned model. The results can be used to scan large image databases, as for example medical databases, The work is financially supported by Spanish Grants TIN 2004 07676-G01-01, TIN 2007 66862 (K.K.) and DGI.M.CyT.FIS2005- 1729 (E.K.)

This is an electronic version of the paper presented at the European Wind Energy Conference, held in Brussels on 2008, After discussing the computational cost drawbacks of Multilayer Perceptrons (MLPs) for wide area prediction of aggregated wind farm output and the need to use Principal Component (PC) analysis for dimensionality reduction, we shall review Support Vector Regression (SVR) as an alternative for such a task. We will numerically demonstrate that SVR may offer a better forecasting performance with training times that are much smaller than those of an MLP and PC combination, and similar to those of an MLP for which a precomputed PC projection matrix is used.