1. Reservoir Computing Universality With Stochastic Inputs
- Author
-
Juan-Pablo Ortega and Lukas Gonon
- Subjects
FOS: Computer and information sciences ,Polynomial ,Computer Networks and Communications ,Computer science ,Computer Science - Emerging Technologies ,computer science ,Dynamical Systems (math.DS) ,Echo state network (ESN) ,machine learning ,reservoir computing ,stochastic input ,uniform system approximation ,universality ,02 engineering and technology ,Artificial Intelligence ,FOS: Mathematics ,0202 electrical engineering, electronic engineering, information engineering ,Uniform boundedness ,Applied mathematics ,Neural and Evolutionary Computing (cs.NE) ,Mathematics - Dynamical Systems ,Artificial neural network ,Stochastic process ,Probability (math.PR) ,Reservoir computing ,Computer Science - Neural and Evolutionary Computing ,Computer Science Applications ,Universality (dynamical systems) ,Emerging Technologies (cs.ET) ,020201 artificial intelligence & image processing ,Mathematics - Probability ,Software - Abstract
The universal approximation properties with respect to $L ^p $-type criteria of three important families of reservoir computers with stochastic discrete-time semi-infinite inputs is shown. First, it is proved that linear reservoir systems with either polynomial or neural network readout maps are universal. More importantly, it is proved that the same property holds for two families with linear readouts, namely, trigonometric state-affine systems and echo state networks, which are the most widely used reservoir systems in applications. The linearity in the readouts is a key feature in supervised machine learning applications. It guarantees that these systems can be used in high-dimensional situations and in the presence of large datasets. The $L ^p $ criteria used in this paper allow the formulation of universality results that do not necessarily impose almost sure uniform boundedness in the inputs or the fading memory property in the filter that needs to be approximated., 11 pages
- Published
- 2020