Search

We demonstrate a data-driven technique for adaptive control in dynamical systems that exploits the reservoir computing method. We show that a reservoir computer can be trained to predict a system parameter from the time series data. Subsequently, a control signal based on the predicted parameter can be used as feedback to the dynamical system to lead it to a target state. Our results show that the dynamical system can be controlled throughout a wide range of attractor types. One set of training data consisting of only a few time series corresponding to the known parameter values enables our scheme to control a dynamical system to an arbitrary target attractor starting from any other initial attractor. In addition to numerical results, we implement our scheme in real-world systems like on a R\"{o}ssler system realized in an electronic circuit to demonstrate the effectiveness of our approach.

Graph neural networks (GNNs) with unsupervised learning can solve large-scale combinatorial optimization problems (COPs) with efficient time complexity, making them versatile for various applications. However, since this method maps the combinatorial optimization problem to the training process of a graph neural network, and the current mainstream backpropagation-based training algorithms are prone to fall into local minima, the optimization performance is still inferior to the current state-of-the-art (SOTA) COP methods. To address this issue, inspired by possibly chaotic dynamics of real brain learning, we introduce a chaotic training algorithm, i.e. chaotic graph backpropagation (CGBP), which introduces a local loss function in GNN that makes the training process not only chaotic but also highly efficient. Different from existing methods, we show that the global ergodicity and pseudo-randomness of such chaotic dynamics enable CGBP to learn each optimal GNN effectively and globally, thus solving the COP efficiently. We have applied CGBP to solve various COPs, such as the maximum independent set, maximum cut, and graph coloring. Results on several large-scale benchmark datasets showcase that CGBP can outperform not only existing GNN algorithms but also SOTA methods. In addition to solving large-scale COPs, CGBP as a universal learning algorithm for GNNs, i.e. as a plug-in unit, can be easily integrated into any existing method for improving the performance.

In-memory computing (IMC) architectures mitigate the von Neumann bottleneck encountered in traditional deep learning accelerators. Its energy efficiency can realize deep learning-based edge applications. However, because IMC is implemented using analog circuits, inherent non-idealities in the hardware pose significant challenges. This paper presents physical neural networks (PNNs) for constructing physical models of IMC. PNNs can address the synaptic current's dependence on membrane potential, a challenge in charge-domain IMC systems. The proposed model is mathematically equivalent to spiking neural networks with reversal potentials. With a novel technique called differentiable spike-time discretization, the PNNs are efficiently trained. We show that hardware non-idealities traditionally viewed as detrimental can enhance the model's learning performance. This bottom-up methodology was validated by designing an IMC circuit with non-ideal characteristics using the sky130 process. When employing this bottom-up approach, the modeling error reduced by an order of magnitude compared to conventional top-down methods in post-layout simulations., Comment: 53 pages, 20 figures

Synchronous firing of neurons is thought to play important functional roles such as feature binding and switching of cognitive states. Although synchronization has mainly been investigated using model neurons with simple connection topology so far, real neural networks have more complex structures. Here we examine behavior of pulse-coupled leaky integrate-and-fire neurons with various network structures. We first show that the dispersion of the number of connections for neurons influences dynamical behavior even if other major topological statistics are kept fixed. The rewiring probability parameter representing the randomness of networks bridges two spatially opposite frameworks: precise local synchrony and rough global synchrony. Finally, cooperation of the global connections and the local clustering property, which is prominent in small-world networks, reinforces synchrony of distant neuronal groups receiving coherent inputs., Comment: 4 figures

Dynamics of class II neurons, firing frequencies of which are strongly regulated by the inherent neuronal property, have been extensively studied since the formulation of the Hodgkin--Huxley model in 1952. However, how class II neurons process stimulus information and what kind of external information and internal structure firing patterns of neurons represent are vaguely understood in contrast to firing rate coding by class I neurons. Here we show that the FitzHugh--Nagumo class II neuron simultaneously filters inputs based on the input frequency and represent the signal strength by interspike intervals. In this sense, the class II neuron works as an AM processor that passes the information on the carrier and on the temporal waveform of signals., Comment: 3 figures

Cooperation is commonly found in ecological and social systems even when it apparently seems that individuals can benefit from selfish behavior. We investigate how cooperation emerges with the spatial prisoner's dilemma played in a class of networks ranging from regular lattices to random networks. We find that, among these networks, small-world topology is the optimal structure when we take into account the speed at which cooperative behavior propagates. Our results may explain why the small-world properties are self-organized in real networks., Comment: 4 figures

Early warning signals (EWSs) for complex dynamical systems aim to anticipate tipping points, or large regime shifts, before they occur. While signals computed from time series data, such as temporal variance and lagged autocorrelation functions, are useful for this task, they are costly to obtain in practice because they need many samples over time to calculate. Spatial EWSs use just a single sample per spatial location and aggregate the samples over space rather than time to try to mitigate this limitation. However, although many complex systems in nature and society form diverse networks, the performance of spatial EWSs is mostly unknown for general networks because the vast majority of studies of spatial EWSs have been on regular lattice networks. Therefore, we have carried out a comprehensive investigation of four major spatial EWSs on various networks. We find that the winning EWS depends on tipping scenarios, while spatial skewness tends to excel when tipping occurs via a saddle-node bifurcation, which is a commonly studied scenario in the literature. We also find that spatial EWSs behave in a drastically different manner between the square lattice and complex networks and tend to be more reliable for the latter than the former according to a standard performance measure. The present results encourage further studies of spatial EWSs on complex networks.

Detecting and quantifying causality is a focal topic in the fields of science, engineering, and interdisciplinary studies. However, causal studies on non-intervention systems attract much attention but remain extremely challenging. To address this challenge, we propose a framework named Interventional Dynamical Causality (IntDC) for such non-intervention systems, along with its computational criterion, Interventional Embedding Entropy (IEE), to quantify causality. The IEE criterion theoretically and numerically enables the deciphering of IntDC solely from observational (non-interventional) time-series data, without requiring any knowledge of dynamical models or real interventions in the considered system. Demonstrations of performance showed the accuracy and robustness of IEE on benchmark simulated systems as well as real-world systems, including the neural connectomes of C. elegans, COVID-19 transmission networks in Japan, and regulatory networks surrounding key circadian genes.

Chaotic dynamics are ubiquitous in nature and useful in engineering, but their geometric design can be challenging. Here, we propose a method using reservoir computing to generate chaos with a desired shape by providing a periodic orbit as a template, called a skeleton. We exploit a bifurcation of the reservoir to intentionally induce unsuccessful training of the skeleton, revealing inherent chaos. The emergence of this untrained attractor, resulting from the interaction between the skeleton and the reservoir's intrinsic dynamics, offers a novel semi-supervised framework for designing chaos., Comment: 6 pages, 4 figures (excluding supplementary material)

Collecting time series data spatially distributed in many locations is often important for analyzing climate change and its impacts on ecosystems. However, comprehensive spatial data collection is not always feasible, requiring us to predict climate variables at some locations. This study focuses on a prediction of climatic elements, specifically near-surface temperature and pressure, at a target location apart from a data observation point. Our approach uses two prediction methods: reservoir computing (RC), known as a machine learning framework with low computational requirements, and vector autoregression models (VAR), recognized as a statistical method for analyzing time series data. Our results show that the accuracy of the predictions degrades with the distance between the observation and target locations. We quantitatively estimate the distance in which effective predictions are possible. We also find that in the context of climate data, a geographical distance is associated with data correlation, and a strong data correlation significantly improves the prediction accuracy with RC. In particular, RC outperforms VAR in predicting highly correlated data within the predictive range. These findings suggest that machine learning-based methods can be used more effectively to predict climatic elements in remote locations by assessing the distance to them from the data observation point in advance. Our study on low-cost and accurate prediction of climate variables has significant value for climate change strategies.

Realizing smooth traffic flow is important for achieving carbon neutrality. Adaptive traffic signal control, which considers traffic conditions, has thus attracted attention. However, it is difficult to ensure optimal vehicle flow throughout a large city using existing control methods because of their heavy computational load. Here, we propose a control method called AMPIC (Adaptive Model Predictive Ising Controller) that guarantees both scalability and optimality. The proposed method employs model predictive control to solve an optimal control problem at each control interval with explicit consideration of a predictive model of vehicle flow. This optimal control problem is transformed into a combinatorial optimization problem with binary variables that is equivalent to the so-called Ising problem. This transformation allows us to use an Ising solver, which has been widely studied and is expected to have fast and efficient optimization performance. We performed numerical experiments using a microscopic traffic simulator for a realistic city road network. The results show that AMPIC enables faster vehicle cruising speed with less waiting time than that achieved by classical control methods, resulting in lower CO2 emissions. The model predictive approach with a long prediction horizon thus effectively improves control performance. Systematic parametric studies on model cities indicate that the proposed method realizes smoother traffic flows for large city road networks. Among Ising solvers, D-Wave's quantum annealing is shown to find near-optimal solutions at a reasonable computational cost., Comment: 18 pages, 8 figures

Chaos-based reinforcement learning (CBRL) is a method in which the agent's internal chaotic dynamics drives exploration. This approach offers a model for considering how the biological brain can create variability in its behavior and learn in an exploratory manner. At the same time, it is a learning model that has the ability to automatically switch between exploration and exploitation modes and the potential to realize higher explorations that reflect what it has learned so far. However, the learning algorithms in CBRL have not been well-established in previous studies and have yet to incorporate recent advances in reinforcement learning. This study introduced Twin Delayed Deep Deterministic Policy Gradients (TD3), which is one of the state-of-the-art deep reinforcement learning algorithms that can treat deterministic and continuous action spaces, to CBRL. The validation results provide several insights. First, TD3 works as a learning algorithm for CBRL in a simple goal-reaching task. Second, CBRL agents with TD3 can autonomously suppress their exploratory behavior as learning progresses and resume exploration when the environment changes. Finally, examining the effect of the agent's chaoticity on learning shows that extremely strong chaos negatively impacts the flexible switching between exploration and exploitation.

Recent studies on StyleGAN variants show promising performances for various generation tasks. In these models, latent codes have traditionally been manipulated and searched for the desired images. However, this approach sometimes suffers from a lack of photorealism in generated images due to a lack of knowledge about the geometry of the trained latent space. In this paper, we show a simple unsupervised method that provides well-trained local latent subspace, enabling latent code navigation while preserving the photorealism of the generated images. Specifically, the method identifies densely mapped latent spaces and restricts latent manipulations within the local latent subspace. Experimental results demonstrate that images generated within the local latent subspace maintain photorealism even when the latent codes are significantly and repeatedly manipulated. Moreover, experiments show that the method can be applied to latent code optimization for various types of style-based models. Our empirical evidence of the method will benefit applications in style-based models.

The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs, which is important not only because of its similarity with information processing in the brain, but also from an engineering point of view. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. The first is the membrane potential-aware SSR (M-SSR) method, which has been derived as an extreme form of the loss function of the membrane potential value. The second is the firing condition-aware SSR (F-SSR) method, which is a regularization function obtained from the firing conditions. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.

Reservoir computing (RC) can efficiently process time-series data by transferring the input signal to randomly connected recurrent neural networks (RNNs), which are referred to as a reservoir. The high-dimensional representation of time-series data in the reservoir significantly simplifies subsequent learning tasks. Although this simple architecture allows fast learning and facile physical implementation, the learning performance is inferior to that of other state-of-the-art RNN models. In this paper, to improve the learning ability of RC, we propose self-modulated RC (SM-RC), which extends RC by adding a self-modulation mechanism. The self-modulation mechanism is realized with two gating variables: an input gate and a reservoir gate. The input gate modulates the input signal, and the reservoir gate modulates the dynamical properties of the reservoir. We demonstrated that SM-RC can perform attention tasks where input information is retained or discarded depending on the input signal. We also found that a chaotic state emerged as a result of learning in SM-RC. This indicates that self-modulation mechanisms provide RC with qualitatively different information-processing capabilities. Furthermore, SM-RC outperformed RC in NARMA and Lorentz model tasks. In particular, SM-RC achieved a higher prediction accuracy than RC with a reservoir 10 times larger in the Lorentz model tasks. Because the SM-RC architecture only requires two additional gates, it is physically implementable as RC, providing a new direction for realizing edge AI.

Non-orthogonal multiple access (NOMA) technique is important for achieving a high data rate in next-generation wireless communications. A key challenge to fully utilizing the effectiveness of the NOMA technique is the optimization of the resource allocation (RA), e.g., channel and power. However, this RA optimization problem is NP-hard, and obtaining a good approximation of a solution with a low computational complexity algorithm is not easy. To overcome this problem, we propose the coherent Ising machine (CIM) based optimization method for channel allocation in NOMA systems. The CIM is an Ising system that can deliver fair approximate solutions to combinatorial optimization problems at high speed (millisecond order) by operating optimization algorithms based on mutually connected photonic neural networks. The performance of our proposed method was evaluated using a simulation model of the CIM. We compared the performance of our proposed method to simulated annealing, a conventional-NOMA pairing scheme, deep Q learning based scheme, and an exhaustive search scheme. Simulation results indicate that our proposed method is superior in terms of speed and the attained optimal solutions., Comment: 16 pages, 10 figures. This work has been submitted to the IEEE for possible publication

We propose a novel backpropagation algorithm for training spiking neural networks (SNNs) that encodes information in the relative multiple spike timing of individual neurons without single-spike restrictions. The proposed algorithm inherits the advantages of conventional timing-based methods in that it computes accurate gradients with respect to spike timing, which promotes ideal temporal coding. Unlike conventional methods where each neuron fires at most once, the proposed algorithm allows each neuron to fire multiple times. This extension naturally improves the computational capacity of SNNs. Our SNN model outperformed comparable SNN models and achieved as high accuracy as non-convolutional artificial neural networks. The spike count property of our networks was altered depending on the time constant of the postsynaptic current and the membrane potential. Moreover, we found that there existed the optimal time constant with the maximum test accuracy. That was not seen in conventional SNNs with single-spike restrictions on time-to-fast-spike (TTFS) coding. This result demonstrates the computational properties of SNNs that biologically encode information into the multi-spike timing of individual neurons. Our code would be publicly available., Comment: 10 pages, 5 figures

We experimentally demonstrate that networks of identical photonic spiking neurons based on coupled degenerate parametric oscillators can show various chimera states, in which, depending on their local synchronization and desynchronization, different kinds of spiking dynamics can develop in a self-organized manner. Even when only a static interaction is implemented, through synchronized inputs from connected neurons, the spiking mode of photonic neurons can be spontaneously and adaptively changed between the Class-I and Class-II modes classified by A. L. Hodgkin. This spontaneous spiking-mode shift induces a significant change in the spiking frequency despite the all neurons having the same natural spiking frequency, which encourages the generation of chimera states. Controllability and self-organized flexibility of the spiking modes in the present system allow us to create an experimental platform to explore the nature of chimera states in neuromorphic spiking neural networks., Comment: 26 pages, 12 figures

Herding is a deterministic algorithm used to generate data points that can be regarded as random samples satisfying input moment conditions. The algorithm is based on the complex behavior of a high-dimensional dynamical system and is inspired by the maximum entropy principle of statistical inference. In this paper, we propose an extension of the herding algorithm, called entropic herding, which generates a sequence of distributions instead of points. Entropic herding is derived as the optimization of the target function obtained from the maximum entropy principle. Using the proposed entropic herding algorithm as a framework, we discuss a closer connection between herding and the maximum entropy principle. Specifically, we interpret the original herding algorithm as a tractable version of entropic herding, the ideal output distribution of which is mathematically represented. We further discuss how the complex behavior of the herding algorithm contributes to optimization. We argue that the proposed entropic herding algorithm extends the application of herding to probabilistic modeling. In contrast to original herding, entropic herding can generate a smooth distribution such that both efficient probability density calculation and sample generation become possible. To demonstrate the viability of these arguments in this study, numerical experiments were conducted, including a comparison with other conventional methods, on both synthetic and real data., Comment: 28 pages, 12 figures. This is the first version of the journal paper submitted to Statistics and Computing. This has not undergone peer review or any post-submission improvements or corrections. The Version of Record of this contribution is published in Statistics and Computing, and is available online at https://doi.org/10.1007/s11222-022-10199-8

Machine learning approaches have recently been leveraged as a substitute or an aid for physical/mathematical modeling approaches to dynamical systems. To develop an efficient machine learning method dedicated to modeling and prediction of multiscale dynamics, we propose a reservoir computing (RC) model with diverse timescales by using a recurrent network of heterogeneous leaky integrator (LI) neurons. We evaluate computational performance of the proposed model in two time series prediction tasks related to four chaotic fast-slow dynamical systems. In a one-step-ahead prediction task where input data are provided only from the fast subsystem, we show that the proposed model yields better performance than the standard RC model with identical LI neurons. Our analysis reveals that the timescale required for producing each component of target multiscale dynamics is appropriately and flexibly selected from the reservoir dynamics by model training. In a long-term prediction task, we demonstrate that a closed-loop version of the proposed model can achieve longer-term predictions compared to the counterpart with identical LI neurons depending on the hyperparameter setting., Comment: 7 pages, 6 figures, supplemental material (10 pages)

The spiking neural network (SNN) has been attracting considerable attention not only as a mathematical model for the brain, but also as an energy-efficient information processing model for real-world applications. In particular, SNNs based on temporal coding are expected to be much more efficient than those based on rate coding, because the former requires substantially fewer spikes to carry out tasks. As SNNs are continuous-state and continuous-time models, it is favorable to implement them with analog VLSI circuits. However, the construction of the entire system with continuous-time analog circuits would be infeasible when the system size is very large. Therefore, mixed-signal circuits must be employed, and the time discretization and quantization of the synaptic weights are necessary. Moreover, the analog VLSI implementation of SNNs exhibits non-idealities, such as the effects of noise and device mismatches, as well as other constraints arising from the analog circuit operation. In this study, we investigated the effects of the time discretization and/or weight quantization on the performance of SNNs. Furthermore, we elucidated the effects the lower bound of the membrane potentials and the temporal fluctuation of the firing threshold. Finally, we propose an optimal approach for the mapping of mathematical SNN models to analog circuits with discretized time., Comment: corrected typos

Facing a global epidemic of new infectious diseases such as COVID-19, non-pharmaceutical interventions (NPIs), which reduce transmission rates without medical actions, are being implemented around the world to mitigate spreads. One of the problems in assessing the effects of NPIs is that different NPIs have been implemented at different times based on the situation of each country; therefore, few assumptions can be shared about how the introduction of policies affects the patient population. Mathematical models can contribute to further understanding these phenomena by obtaining analytical solutions as well as numerical simulations. In this study, an NPI was introduced into the SIR model for a conceptual study of infectious diseases under the condition that the transmission rate was reduced to a fixed value only once within a finite time duration, and its effect was analyzed numerically and theoretically. It was analytically shown that the maximum fraction of infected individuals and the final size could be larger if the intervention starts too early. The analytical results also suggested that more individuals may be infected at the peak of the second wave with a stronger intervention. This study provides quantitative relationship between the strength of a one-shot intervention and the reduction in the number of patients with no approximation. This suggests the importance of the strength and time of NPIs, although detailed studies are necessary for the implementation of NPIs in complicated real-world environments as the model used in this study is based on various simplifications.

Nonlinear dynamics of spiking neural networks has recently attracted much interest as an approach to understand possible information processing in the brain and apply it to artificial intelligence. Since information can be processed by collective spiking dynamics of neurons, the fine control of spiking dynamics is desirable for neuromorphic devices. Here we show that photonic spiking neurons implemented with paired nonlinear optical oscillators can be controlled to generate two modes of bio-realistic spiking dynamics by changing the optical pump amplitude. When they are coupled in a network, we found that the interaction between the photonic neurons induces an effective change in the pump amplitude depending on the order parameter that characterizes synchronization. The experimental results show that the effective change causes spontaneous modification of the spiking modes and firing rates of clustered neurons, and such collective dynamics can be utilized to realize efficient heuristics for solving NP-hard combinatorial optimization problems., Comment: 43 pages, 9 figures

The development of physical simulators, called Ising machines, that sample from low energy states of the Ising Hamiltonian has the potential to drastically transform our ability to understand and control complex systems. However, most of the physical implementations of such machines have been based on a similar concept that is closely related to relaxational dynamics such as in simulated, mean-field, chaotic, and quantum annealing. We show that nonrelaxational dynamics that is associated with broken detailed balance and positive entropy production rate can accelerate the sampling of low energy states compared to that of conventional methods. By implementing such dynamics on field programmable gate array, we show that the nonrelaxational dynamics that we propose, called chaotic amplitude control, exhibits a scaling with problem size of the time to finding optimal solutions and its variance that is significantly smaller than that of relaxational schemes recently implemented on Ising machines.

Reservoir computing (RC) is a machine learning algorithm that can learn complex time series from data very rapidly based on the use of high-dimensional dynamical systems, such as random networks of neurons, called "reservoirs." To implement RC in edge computing, it is highly important to reduce the amount of computational resources that RC requires. In this study, we propose methods that reduce the size of the reservoir by inputting the past or drifting states of the reservoir to the output layer at the current time step. These proposed methods are analyzed based on information processing capacity, which is a performance measure of RC proposed by Dambre et al. (2012). In addition, we evaluate the effectiveness of the proposed methods on time-series prediction tasks: the generalized Henon-map and NARMA. On these tasks, we found that the proposed methods were able to reduce the size of the reservoir up to one tenth without a substantial increase in regression error. Because the applications of the proposed methods are not limited to a specific network structure of the reservoir, the proposed methods could further improve the energy efficiency of RC-based systems, such as FPGAs and photonic systems.

The spread of intelligent transportation systems in urban cities has caused heavy computational loads, requiring a novel architecture for managing large-scale traffic. In this study, we develop a method for globally controlling traffic signals arranged on a square lattice by means of a quantum annealing machine, namely the D-Wave quantum annealer. We first formulate a signal optimization problem that minimizes the imbalance of traffic flows in two orthogonal directions. Then we reformulate this problem as an Ising Hamiltonian, which is fully compatible with quantum annealers. The new control method is compared with a conventional local control method for a large 50-by-50 city, and the results exhibit the superiority of our global control method in suppressing traffic imbalance over wide parameter ranges. Furthermore, the solutions to the global control method obtained with the quantum annealing machine are better than those obtained with conventional simulated annealing. In addition, we prove analytically that the local and the global control methods converge at the limit where cars have equal probabilities for turning and going straight. These results are verified with numerical experiments., Comment: This is a pre-print of an article published in Scientific Reports. The final authenticated version is available online at: https://doi.org/10.1038/s41598-021-82740-0

We propose a new digital-to-analog converter (DAC) for realizing a synapse circuit of mixed-signal spiking neural networks. We named this circuit "time-domain DAC (TDAC)". This produces weights for converting a digital input code into voltage using one current waveform. Therefore, a TDAC is more compact than a conventional DAC that comprises many current sources and resistors. Moreover, a TDAC with leak resistance reproduces biological plausible synaptic responses that are expressed as alpha functions or dual exponential equations. We will show numerical analysis results of a TDAC and circuit simulation results of a circuit designed by the TSMC 40 nm CMOS process.

Spiking neural networks (SNNs) are brain-inspired mathematical models with the ability to process information in the form of spikes. SNNs are expected to provide not only new machine-learning algorithms, but also energy-efficient computational models when implemented in VLSI circuits. In this paper, we propose a novel supervised learning algorithm for SNNs based on temporal coding. A spiking neuron in this algorithm is designed to facilitate analog VLSI implementations with analog resistive memory, by which ultra-high energy efficiency can be achieved. We also propose several techniques to improve the performance on a recognition task, and show that the classification accuracy of the proposed algorithm is as high as that of the state-of-the-art temporal coding SNN algorithms on the MNIST dataset. Finally, we discuss the robustness of the proposed SNNs against variations that arise from the device manufacturing process and are unavoidable in analog VLSI implementation. We also propose a technique to suppress the effects of variations in the manufacturing process on the recognition performance.

Data-driven flood forecasting methods are useful, especially for the rivers that lack hydrological information to build physical models. Although these former methods can forecast river stages using only past water levels and rainfall data, they cannot handle previously unexperienced water levels easily, and require a large amount of data to build accurate models. Here, we focus on phase-space reconstruction approaches, and develop a practical data-driven forecasting method to overcome the existing problems. The proposed method can handle the unexperienced water levels and provide forecasts using only a small number of water rise events. We apply the method to data from actual rivers, and it achieved the best forecast performance among existing methods, including a physical runoff model, a data-driven multi-layer perceptron, and a conventional method based on phase-space reconstruction. In addition, the proposed method also forecasted the exceedance of the evacuation warning level 6 h earlier for steep rivers. Given its performance and maintainability, the proposed method can be applied to many actual rivers for early evacuation.

Clustering algorithms are a cornerstone of machine learning applications. Recently, a quantum algorithm for clustering based on the k-means algorithm has been proposed by Kerenidis, Landman, Luongo and Prakash. Based on their work, we propose a quantum expectation-maximization (EM) algorithm for Gaussian mixture models (GMMs). The robustness and quantum speedup of the algorithm is demonstrated. We also show numerically the advantage of GMM over k-means for non-trivial cluster data., Comment: 10 pages, 9 figures

Delay embedding---a method for reconstructing dynamical systems by delay coordinates---is widely used to forecast nonlinear time series as a model-free approach. When multivariate time series are observed, several existing frameworks can be applied to yield a single forecast combining multiple forecasts derived from various embeddings. However, the performance of these frameworks is not always satisfactory because they randomly select embeddings or use brute force and do not consider the diversity of the embeddings to combine. Herein, we develop a forecasting framework that overcomes these existing problems. The framework exploits various "suboptimal embeddings" obtained by minimizing the in-sample error via combinatorial optimization. The framework achieves the best results among existing frameworks for sample toy datasets and a real-world flood dataset. We show that the framework is applicable to a wide range of data lengths and dimensions. Therefore, the framework can be applied to various fields such as neuroscience, ecology, finance, fluid dynamics, weather, and disaster prevention.

A temporal point process is a mathematical model for a time series of discrete events, which covers various applications. Recently, recurrent neural network (RNN) based models have been developed for point processes and have been found effective. RNN based models usually assume a specific functional form for the time course of the intensity function of a point process (e.g., exponentially decreasing or increasing with the time since the most recent event). However, such an assumption can restrict the expressive power of the model. We herein propose a novel RNN based model in which the time course of the intensity function is represented in a general manner. In our approach, we first model the integral of the intensity function using a feedforward neural network and then obtain the intensity function as its derivative. This approach enables us to both obtain a flexible model of the intensity function and exactly evaluate the log-likelihood function, which contains the integral of the intensity function, without any numerical approximations. Our model achieves competitive or superior performances compared to the previous state-of-the-art methods for both synthetic and real datasets.

The relation between the work performed to a system and the change of its free energy during a certain process is important in nonequilibrium statistical mechanics. In particular, the work relation with measurement and feedback control has attracted much attention, because it resolved the paradox concerning Maxwell's demon. Most studies, however, assume that their target systems are isolated or isothermal. In this paper, by considering a nonisothermal system, we generalize the Sagawa-Ueda-Jarzynski relation, which involves measurement and feedback control, and apply it to a realistic model. Furthermore, when the temperature profile is quadratic, we see that the system is governed by Tsallis statistical mechanics. In addition, we show that our formulation provides the generalized version of the second law of information thermodynamics and a set of work relations for isothermal systems., Comment: 7 pages, 1 figure, published in PRE

The relaxation of binary spins to analog values has been the subject of much debate in the field of statistical physics, neural networks, and more recently quantum computing, notably because the benefits of using an analog state for finding lower energy spin configurations are usually offset by the negative impact of the improper mapping of the energy function that results from the relaxation. We show that it is possible to destabilize trapping sets of analog states that correspond to local minima of the binary spin Hamiltonian by extending the phase space to include error signals that correct amplitude inhomogeneity of the analog spin states and controlling the divergence of their velocity. Performance of the proposed analog spin system in finding lower energy states is competitive against state-of-the-art heuristics., Comment: Revised version. Supplementary material included

Multiple types of fluctuations impact the collective dynamics of power grids and thus challenge their robust operation. Fluctuations result from processes as different as dynamically changing demands, energy trading, and an increasing share of renewable power feed-in. Here we analyze principles underlying the dynamics and statistics of power grid frequency fluctuations. Considering frequency time series for a range of power grids, including grids in North America, Japan and Europe, we find a substantial deviation from Gaussianity best described as L\'evy-stable and q-Gaussian distributions. We present a coarse framework to analytically characterize the impact of arbitrary noise distributions as well as a superstatistical approach which systematically interprets heavy tails and skewed distributions. We identify energy trading as a substantial contribution to today's frequency fluctuations and effective damping of the grid as a controlling factor enabling reduction of fluctuation risks, with enhanced effects for small power grids., Comment: Accepted version of Nature Energy article (see doi for original publication)

Recently, the coherent Ising machine (CIM) as a degenerate optical parametric oscillator (DOPO) network has been researched to solve Ising combinatorial optimization problems. We formulate a theoretical model for the CIM with discrete-time measurement feedback processes, and perform numerical simulations for the simplest network, composed of two degenerate optical parametric oscillator pulses with the anti-ferromagnetic mutual coupling. We evaluate the extent to which quantum coherence exists during the optimization process.

The quantum theory of coherent Ising machines, based on degenerate optical parametric oscillators and measurement-feedback circuits, is developed using the positive $P({\alpha},{\beta})$ representation of the density operator and the master equation. The theory is composed of the c-number stochastic differential equations for describing open dissipative quantum dynamics and the replicator dynamics equations for handling measurement-induced collapse of the density operator. We apply the present theory to simulate two simple Ising spin models and elucidate the unique features of this computing machine., Comment: 9 pages, 5 figures

The coherent Ising machine is expected to find a near-optimal solution in various combinatorial optimization problems, which has been experimentally confirmed with optical parametric oscillators (OPOs) and a field programmable gate array (FPGA) circuit. The similar mathematical models were proposed three decades ago by J. J. Hopfield, et al. in the context of classical neural networks. In this article, we compare the computational performance of both models., Comment: 11 pages, 4 figures, submitted to IOP-QST

The asymptotic attractors of a nonlinear dynamical system play a key role in the long-term physically observable behaviors of the system. The study of attractors and the search for distinct types of attractor have been a central task in nonlinear dynamics. In smooth dynamical systems, an attractor is often enclosed completely in its basin of attraction with a finite distance from the basin boundary. Recent works have uncovered that, in neuronal networks, unstable attractors with a remote basin can arise, where almost every point on the attractor is locally transversely repelling. Herewith we report our discovery of a class of attractors: partially unstable attractors, in pulse-coupled integrate-and-fire networks subject to a periodic forcing. The defining feature of such an attractor is that it can simultaneously possess locally stable and unstable sets, both of positive measure. Exploiting the structure of the key dynamical events in the network, we develop a symbolic analysis that can fully explain the emergence of the partially unstable attractors. To our knowledge, such exotic attractors have not been reported previously, and we expect them to arise commonly in biological networks whose dynamics are governed by pulse (or spike) generation., Comment: 14 pages