Search

Reservoir computing, a machine learning framework used for modeling the brain, can predict temporal data with little observations and minimal computational resources. However, it is difficult to accurately reproduce the long-term target time series because the reservoir system becomes unstable. This predictive capability is required for a wide variety of time-series processing, including predictions of motor timing and chaotic dynamical systems. This study proposes oscillation-driven reservoir computing (ODRC) with feedback, where oscillatory signals are fed into a reservoir network to stabilize the network activity and induce complex reservoir dynamics. The ODRC can reproduce long-term target time series more accurately than conventional reservoir computing methods in a motor timing and chaotic time-series prediction tasks. Furthermore, it generates a time series similar to the target in the unexperienced period, that is, it can learn the abstract generative rules from limited observations. Given these significant improvements made by the simple and computationally inexpensive implementation, the ODRC would serve as a practical model of various time series data. Moreover, we will discuss biological implications of the ODRC, considering it as a model of neural oscillations and their cerebellar processors.

In this work, we propose a novel method for modeling numerous speakers, which enables expressing the overall characteristics of speakers in detail like a trained multi-speaker model without additional training on the target speaker's dataset. Although various works with similar purposes have been actively studied, their performance has not yet reached that of trained multi-speaker models due to their fundamental limitations. To overcome previous limitations, we propose effective methods for feature learning and representing target speakers' speech characteristics by discretizing the features and conditioning them to a speech synthesis model. Our method obtained a significantly higher similarity mean opinion score (SMOS) in subjective similarity evaluation than seen speakers of a high-performance multi-speaker model, even with unseen speakers. The proposed method also outperforms a zero-shot method by significant margins. Furthermore, our method shows remarkable performance in generating new artificial speakers. In addition, we demonstrate that the encoded latent features are sufficiently informative to reconstruct an original speaker's speech completely. It implies that our method can be used as a general methodology to encode and reconstruct speakers' characteristics in various tasks., Comment: ICML 2024

Single-stage text-to-speech models have been actively studied recently, and their results have outperformed two-stage pipeline systems. Although the previous single-stage model has made great progress, there is room for improvement in terms of its intermittent unnaturalness, computational efficiency, and strong dependence on phoneme conversion. In this work, we introduce VITS2, a single-stage text-to-speech model that efficiently synthesizes a more natural speech by improving several aspects of the previous work. We propose improved structures and training mechanisms and present that the proposed methods are effective in improving naturalness, similarity of speech characteristics in a multi-speaker model, and efficiency of training and inference. Furthermore, we demonstrate that the strong dependence on phoneme conversion in previous works can be significantly reduced with our method, which allows a fully end-to-end single-stage approach., Comment: Interspeech 2023

Automatic modulation classification (AMC) is a technology that identifies a modulation scheme without prior signal information and plays a vital role in various applications, including cognitive radio and link adaptation. With the development of deep learning (DL), DL-based AMC methods have emerged, while most of them focus on reducing computational complexity in a centralized structure. This centralized learning-based AMC (CentAMC) violates data privacy in the aspect of direct transmission of client-side raw data. Federated learning-based AMC (FedeAMC) can bypass this issue by exchanging model parameters, but causes large resultant latency and client-side computational load. Moreover, both CentAMC and FedeAMC are vulnerable to large-scale noise occured in the wireless channel between the client and the server. To this end, we develop a novel AMC method based on a split learning (SL) framework, coined SplitAMC, that can achieve high accuracy even in poor channel conditions, while guaranteeing data privacy and low latency. In SplitAMC, each client can benefit from data privacy leakage by exchanging smashed data and its gradient instead of raw data, and has robustness to noise with the help of high scale of smashed data. Numerical evaluations validate that SplitAMC outperforms CentAMC and FedeAMC in terms of accuracy for all SNRs as well as latency., Comment: to be presented at IEEE VTC2023-Spring

An increase in power consumption necessitates a low-power circuit technology to extend Moore's law. Low-power transistors, such as tunnel field-effect transistors (TFETs), negative-capacitance field-effect transistors (NC-FETs), and Dirac-source field-effect transistors (DS-FETs), have been realised to break the thermionic limit of the subthreshold swing (SS). However, a low-power diode rectifier, which breaks the thermionic limit of an ideality factor (n) of 1 at room temperature, has not been proposed yet. In this study, we have realised a DS Schottky diode, which exhibits a steep-slope characteristic curve, by utilising the linear density of states (DOSs) of graphene. For the developed DS Schottky diode, n<1 for more than two decades of drain current with a minimum value of 0.8, and the rectifying ratio is large (100000). The realisation of a DS Schottky diode paves the way for the development of low-power electronic circuits., Comment: 28 pages, 14 figures, submitted to Nature Communications

Sutton, Szepesv\'{a}ri and Maei introduced the first gradient temporal-difference (GTD) learning algorithms compatible with both linear function approximation and off-policy training. The goal of this paper is (a) to propose some variants of GTDs with extensive comparative analysis and (b) to establish new theoretical analysis frameworks for the GTDs. These variants are based on convex-concave saddle-point interpretations of GTDs, which effectively unify all the GTDs into a single framework, and provide simple stability analysis based on recent results on primal-dual gradient dynamics. Finally, numerical comparative analysis is given to evaluate these approaches.

Binarization of neural network models is considered as one of the promising methods to deploy deep neural network models on resource-constrained environments such as mobile devices. However, Binary Neural Networks (BNNs) tend to suffer from severe accuracy degradation compared to the full-precision counterpart model. Several techniques were proposed to improve the accuracy of BNNs. One of the approaches is to balance the distribution of binary activations so that the amount of information in the binary activations becomes maximum. Based on extensive analysis, in stark contrast to previous work, we argue that unbalanced activation distribution can actually improve the accuracy of BNNs. We also show that adjusting the threshold values of binary activation functions results in the unbalanced distribution of the binary activation, which increases the accuracy of BNN models. Experimental results show that the accuracy of previous BNN models (e.g. XNOR-Net and Bi-Real-Net) can be improved by simply shifting the threshold values of binary activation functions without requiring any other modification., Comment: CVPR 2021, 10 pages, 10 figures

Manipulating materials with atomic-scale precision is essential for the development of next-generation material design toolbox. Tremendous efforts have been made to advance the compositional, structural, and spatial accuracy of material deposition and patterning. The family of 2D materials provides an ideal platform to realize atomic-level material architectures. The wide and rich physics of these materials have led to fabrication of heterostructures, superlattices, and twisted structures with breakthrough discoveries and applications. Here, we report a novel atomic-scale material design tool that selectively breaks and forms chemical bonds of 2D materials at room temperature, called atomic-layer substitution (ALS), through which we can substitute the top layer chalcogen atoms within the 3-atom-thick transition-metal dichalcogenides using arbitrary patterns. Flipping the layer via transfer allows us to perform the same procedure on the other side, yielding programmable in-plane multi-heterostructures with different out-of-plane crystal symmetry and electric polarization. First-principle calculations elucidate how the ALS process is overall exothermic in energy and only has a small reaction barrier, facilitating the reaction to occur at room temperature. Optical characterizations confirm the fidelity of this design approach, while TEM shows the direct evidence of Janus structure and suggests the atomic transition at the interface of designed heterostructure. Finally, transport and Kelvin probe measurements on MoXY (X,Y=S,Se; X and Y corresponding to the bottom and top layers) lateral multi-heterostructures reveal the surface potential and dipole orientation of each region, and the barrier height between them. Our approach for designing artificial 2D landscape down to a single layer of atoms can lead to unique electronic, photonic and mechanical properties previously not found in nature.

Integrated gradients are widely employed to evaluate the contribution of input features in classification models because it satisfies the axioms for attribution of prediction. This method, however, requires an appropriate baseline for reliable determination of the contributions. We propose a compensated integrated gradients method that does not require a baseline. In fact, the method compensates the attributions calculated by integrated gradients at an arbitrary baseline using Shapley sampling. We prove that the method retrieves reliable attributions if the processes of input features in a classifier are mutually independent, and they are identical like shared weights in convolutional neural networks. Using three electroencephalogram datasets, we experimentally demonstrate that the attributions of the proposed method are more reliable than those of the original integrated gradients, and its computational complexity is much lower than that of Shapley sampling., Comment: Machine Learning for Health (ML4H) Workshop at NeurIPS 2018 arXiv:1811.07216