Search

The traditional extreme learning machine (ELM) approach is based on a random assignment of the hidden weight values, while the linear coefficients of the output layer are determined analytically. This brief presents an analysis based on geometric properties of the sampling points used to assign the weight values, investigating the replacement of random generation of such values with low-discrepancy sequences (LDSs). Such sequences are a family of sampling methods commonly employed for numerical integration, yielding a more efficient covering of multidimensional sets with respect to random sequences, without the need for any computationally intensive procedure. In particular, we prove that the universal approximation property of the ELM is guaranteed when LDSs are employed, and how an efficient covering affects the convergence positively. Furthermore, since LDSs are generated deterministically, the results do not have a probabilistic nature. Simulation results confirm, in practice, the good theoretical properties given by the combination of ELM with LDSs. [ABSTRACT FROM AUTHOR]

Local linear regression models, a kind of nonparametric structures that locally perform a linear estimation of the target function, are analyzed in the context of empirical risk minimization (ERM) for function learning. The analysis is carried out with emphasis on geometric properties of the available data. In particular, the discrepancy of the observation points used both to build the local regression models and compute the empirical risk is considered. This allows to treat indifferently the case in which the samples come from a random external source and the one in which the input space can be freely explored. Both consistency of the ERM procedure and approximating capabilities of the estimator are analyzed, proving conditions to ensure convergence. Since the theoretical analysis shows that the estimation improves as the discrepancy of the observation points becomes smaller, low-discrepancy sequences, a family of sampling methods commonly employed for efficient numerical integration, are also analyzed. Simulation results involving two different examples of function learning are provided. [ABSTRACT FROM PUBLISHER]

This brief presents an analysis of the performance of kernel smoothing models used to estimate an unknown target function, addressing the case where the choice of the training set is part of the learning process. In particular, we consider a choice of the points at which the function is observed based on low-discrepancy sequences, which is a family of sampling methods commonly employed for efficient numerical integration. We prove that, under suitable regularity assumptions, consistency of the empirical risk minimization is guaranteed with a good rate of convergence of the estimation error, as well as the convergence of the approximation error. Simulation results confirm, in practice, the good theoretical properties given by the combination of kernel smoothing models with low-discrepancy sampling. [ABSTRACT FROM AUTHOR]