Search

Abstract Diversity conveys advantages in nature, yet homogeneous neurons typically comprise the layers of artificial neural networks. Here we construct neural networks from neurons that learn their own activation functions, quickly diversify, and subsequently outperform their homogeneous counterparts on image classification and nonlinear regression tasks. Sub-networks instantiate the neurons, which meta-learn especially efficient sets of nonlinear responses. Examples include conventional neural networks classifying digits and forecasting a van der Pol oscillator and physics-informed Hamiltonian neural networks learning Hénon–Heiles stellar orbits and the swing of a video recorded pendulum clock. Such learned diversity provides examples of dynamical systems selecting diversity over uniformity and elucidates the role of diversity in natural and artificial systems.

A particle confined to an impassable box is a paradigmatic and exactly solvable one-dimensional quantum system modeled by an infinite square well potential. Here, we explore some of its infinitely many generalizations to two dimensions, including particles confined to rectangle-, ellipse-, triangle-, and cardioid-shaped boxes using physics-informed neural networks. In particular, we generalize an unsupervised learning algorithm to find the particles’ eigenvalues and eigenfunctions, even in cases where the eigenvalues are degenerate. During training, the neural network adjusts its weights and biases, one of which is the energy eigenvalue, so that its output approximately solves the stationary Schrödinger equation with normalized and mutually orthogonal eigenfunctions. The same procedure solves the Helmholtz equation for the harmonics and vibration modes of waves on drumheads or transverse magnetic modes of electromagnetic cavities. Related applications include quantum billiards, quantum chaos, and Laplacian spectra.

We quantify how incorporating physics into neural network design can significantly improve the learning and forecasting of dynamical systems, even nonlinear systems of many dimensions. We train conventional and Hamiltonian neural networks on increasingly difficult dynamical systems and compute their forecasting errors as the number of training data and number of system dimensions vary. A map-building perspective elucidates the superiority of Hamiltonian neural networks. The results clarify the critical relation among data, dimension, and neural network learning performance.

We develop a framework to uncover and analyse dynamical anomalies from massive, nonlinear and non-stationary time series data. The framework consists of three steps: preprocessing of massive datasets to eliminate erroneous data segments, application of the empirical mode decomposition and Hilbert transform paradigm to obtain the fundamental components embedded in the time series at distinct time scales, and statistical/scaling analysis of the components. As a case study, we apply our framework to detecting and characterizing high-frequency oscillations (HFOs) from a big database of rat electroencephalogram recordings. We find a striking phenomenon: HFOs exhibit on–off intermittency that can be quantified by algebraic scaling laws. Our framework can be generalized to big data-related problems in other fields such as large-scale sensor data and seismic data analysis.

The core element of machine learning is a flexible, universal function approximator that can be trained and fit into the data. One of the main challenges in modern machine learning is to understand the role of nonlinearity and complexity in these universal function approximators. In this research, we focus on nonlinear complex systems, and show their capability in representation and learning of different functions. Complex nonlinear dynamics and chaos naturally yield an almost infinite diversity of dynamical behaviors and functions. Physical, biological and engineered systems can utilize this diversity to implement adaptive, robust behaviors and operations. A nonlinear dynamical system can be considered as an embodiment of a collection of different possible behaviors or functions, from which different behaviors or functions can be chosen as a response to different conditions or problems. This process of selection can be manual in the sense that one can manually pick and choose the right function through directly setting parameters. Alternatively, we can automate the process and allow the system itself learn how to do it. This creates an approach to machine learning, wherein the nonlinear dynamics represents and embodies different possible functions, and it learns through training how to pick the right function from this function space. We report on how we utilized nonlinear dynamics and chaos to design and fabricate nonlinear dynamics based, morphable hardware in silicon as a physical embodiment for different possible functions. We demonstrate how this flexible, morphable hardware learns through learning and searching algorithms such as genetic algorithm to implement different desired functions. In this approach, we combine two powerful natural and biological phenomenon, Darwinian evolution and nonlinear dynamics and chaos, as a dynamics-oriented approach to designing intelligent, adaptive systems with applications. Nonlinear dynamics embodies different functions at the hardware level, while an evolutionary method is utilized in order to find the parameters to implement the right function.

Certain nonlinear systems can switch between dynamical attractors occupying different regions of phase space, under variation of parameters or initial states. In this work we exploit this feature to obtain reliable logic operations. With logic output 0/1 mapped to dynamical attractors bounded in distinct regions of phase space, and logic inputs encoded by a very small bias parameter, we explicitly demonstrate that the system hops consistently in response to an external input stream, operating effectively as a reliable logic gate. This system offers the advantage that very low-amplitude inputs yield highly amplified outputs. Additionally, different dynamical variables in the system yield complementary logic operations in parallel. Further, we show that in certain parameter regions noise aids the reliability of logic operations, and is actually necessary for obtaining consistent outputs. This leads us to a generalization of the concept of Logical Stochastic Resonance to attractors more complex than fixed point states, such as periodic or chaotic attractors. Lastly, the results are verified in electronic circuit experiments, demonstrating the robustness of the phenomena. So we have combined the research directions of Chaos Computing and Logical Stochastic Resonance here, and this approach has potential to be realized in wide-ranging systems.