Search

An agent employing reinforcement learning takes inputs (state variables) from an environment and performs actions that affect the environment in order to achieve some objective. Rewards (positive or negative) guide the agent toward improved future actions. This paper builds on prior clustering neural network research by constructing an agent with biologically plausible neo-Hebbian three-factor synaptic learning rules, with a reward signal as the third factor (in addition to pre- and post-synaptic spikes). The classic cart-pole problem (balancing an inverted pendulum) is used as a running example throughout the exposition. Simulation results demonstrate the efficacy of the approach, and the proposed method may eventually serve as a low-level component of a more general method.

The macrocolumn is a key component of a neuromorphic computing system that interacts with an external environment under control of an agent. Environments are learned and stored in the macrocolumn as labeled directed graphs where edges connect features and labels indicate the relative displacements between them. Macrocolumn functionality is first defined with a state machine model. This model is then implemented with a neural network composed of spiking neurons. The neuron model employs active dendrites and mirrors the Hawkins/Numenta neuron model. The architecture is demonstrated with a research benchmark in which an agent employs a macrocolumn to first learn and then navigate 2-d environments containing pseudo-randomly placed features., Comment: This is a major revision. Neuron outputs are encoded as the body potential. Winner-take-all inhibition then compares body potentials to determine a winner. At the end of each cycle, a non-zero WTA output is converted to a binary spike. This method remains consistent with temporal neuron operation internal to a cycle, with only a single bit of temporal precision being maintained between cycles

A Temporal Neural Network (TNN) architecture for implementing efficient online reinforcement learning is proposed and studied via simulation. The proposed T-learning system is composed of a frontend TNN that implements online unsupervised clustering and a backend TNN that implements online reinforcement learning. The reinforcement learning paradigm employs biologically plausible neo-Hebbian three-factor learning rules. As a working example, a prototype implementation of the cart-pole problem (balancing an inverted pendulum) is studied via simulation.

We present a stable and systematically improvable quantum Monte Carlo (QMC) approach to calculating excited-state energies, which we implement using our fast randomized iteration method for the full configuration interaction problem (FCI-FRI). Unlike previous excited-state quantum Monte Carlo methods, our approach, which is an asymmetric variant of subspace iteration, avoids the use of dot products of random vectors and instead relies upon trial vectors to maintain orthogonality and estimate eigenvalues. By leveraging recent advances, we apply our method to calculate ground- and excited-state energies of strongly correlated molecular systems in large active spaces, including the carbon dimer with 8 electrons in 108 orbitals (8e,108o), an oxo-Mn(salen) transition metal complex (28e,28o), ozone (18e,87o), and butadiene (22e,82o). In the majority of these test cases, our approach yields total excited-state energies that agree with those from state-of-the-art methods -- including heat-bath CI, the density matrix renormalization group approach, and FCIQMC -- to within sub-milliHartree accuracy. In all cases, estimated excitation energies agree to within about 0.1 eV., Comment: 16 pages, 5 figures, 3 tables

This document is focused on computing systems implemented in technologies that communicate and compute with temporal transients. Although described in general terms, implementations of spiking neural networks are of primary interest. As background, an algebra for constructing temporal networks is summarized. Then, a system organization consisting of synchronized segments is described. The segments are feedforward internally with feedback between segments. A synchronizing clock resets network segments at the end of each computation step or cycle. In its basic form, the synchronizing clock merely performs a reset function. In the context of neural networks, this satisfies biological plausibility. However, functional completeness is restricted. This restriction is removed by allowing use of the synchronizing clock as an additional function input that acts as a temporal reference value.

In this paper, we study the nuclear gradients of heat bath configuration interaction self-consistent field (HCISCF) wave functions and use them to optimize molecular geometries for various molecules. We show that the HCISCF nuclear gradients are fairly insensitive to the size of the "selected" variational space, which allows us to reduce the computational cost without introducing significant error. The ability of HCISCF to treat larger active spaces combined with the flexibility for users to control the computational cost makes the method very attractive for studying strongly correlated systems which require a larger active space than possible with complete active space self-consistent field (CASSCF). Finally, we study the realistic catalyst, Fe(PDI), and highlight some of the challenges this system poses for density functional theory (DFT). We demonstrate how HCISCF can clarify the energetic stability of geometries obtained from DFT when the results are strongly dependent on the functional. We also use the HCISCF gradients to optimize geometries for this species and study the adiabatic singlet-triplet gap. During geometry optimization, we find that multiple near-degenerate local minima exist on the triplet potential energy surface.

Temporal Neural Networks (TNNs) are spiking neural networks that use time as a resource to represent and process information, similar to the mammalian neocortex. In contrast to compute-intensive deep neural networks that employ separate training and inference phases, TNNs are capable of extremely efficient online incremental/continual learning and are excellent candidates for building edge-native sensory processing units. This work proposes a microarchitecture framework for implementing TNNs using standard CMOS. Gate-level implementations of three key building blocks are presented: 1) multi-synapse neurons, 2) multi-neuron columns, and 3) unsupervised and supervised online learning algorithms based on Spike Timing Dependent Plasticity (STDP). The proposed microarchitecture is embodied in a set of characteristic scaling equations for assessing the gate count, area, delay and power for any TNN design. Post-synthesis results (in 45nm CMOS) for the proposed designs are presented, and their online incremental learning capability is demonstrated., Comment: To be published in ISVLSI 2021. arXiv admin note: substantial text overlap with arXiv:2009.00457

A long-standing proposition is that by emulating the operation of the brain's neocortex, a spiking neural network (SNN) can achieve similar desirable features: flexible learning, speed, and efficiency. Temporal neural networks (TNNs) are SNNs that communicate and process information encoded as relative spike times (in contrast to spike rates). A TNN architecture is proposed, and, as a proof-of-concept, TNN operation is demonstrated within the larger context of online supervised classification. First, through unsupervised learning, a TNN partitions input patterns into clusters based on similarity. The TNN then passes a cluster identifier to a simple online supervised decoder which finishes the classification task. The TNN learning process adjusts synaptic weights by using only signals local to each synapse, and clustering behavior emerges globally. The system architecture is described at an abstraction level analogous to the gate and register transfer levels in conventional digital design. Besides features of the overall architecture, several TNN components are new to this work. Although not addressed directly, the overall research objective is a direct hardware implementation of TNNs. Consequently, all the architecture elements are simple, and processing is done at very low precision., Comment: 13 pages, 10 figures

Temporal Neural Networks (TNNs) use time as a resource to represent and process information, mimicking the behavior of the mammalian neocortex. This work focuses on implementing TNNs using off-the-shelf digital CMOS technology. A microarchitecture framework is introduced with a hierarchy of building blocks including: multi-neuron columns, multi-column layers, and multi-layer TNNs. We present the direct CMOS gate-level implementation of the multi-neuron column model as the key building block for TNNs. Post-synthesis results are obtained using Synopsys tools and the 45 nm CMOS standard cell library. The TNN microarchitecture framework is embodied in a set of characteristic equations for assessing the total gate count, die area, compute time, and power consumption for any TNN design. We develop a multi-layer TNN prototype of 32M gates. In 7 nm CMOS process, it consumes only 1.54 mm^2 die area and 7.26 mW power and can process 28x28 images at 107M FPS (9.34 ns per image). We evaluate the prototype's performance and complexity relative to a recent state-of-the-art TNN model., Comment: Submission Under Review for an IEEE Conference

A computational paradigm based on neuroscientific concepts is proposed and shown to be capable of online unsupervised clustering. Because it is an online method, it is readily amenable to streaming realtime applications and is capable of dynamically adjusting to macro-level input changes. All operations, both training and inference, are localized and efficient. The paradigm is implemented as a cognitive column that incorporates five key elements: 1) temporal coding, 2) an excitatory neuron model for inference, 3) winner-take-all inhibition, 4) a column architecture that combines excitation and inhibition, 5) localized training via spike timing de-pendent plasticity (STDP). These elements are described and discussed, and a prototype column is given. The prototype column is simulated with a semi-synthetic benchmark and is shown to have performance characteristics on par with classic k-means. Simulations reveal the inner operation and capabilities of the column with emphasis on excitatory neuron response functions and STDP implementations., Comment: Submitted to 53rd IEEE/ACM International Symposium on Microarchitecture

PYSCF is a Python-based general-purpose electronic structure platform that both supports first-principles simulations of molecules and solids, as well as accelerates the development of new methodology and complex computational workflows. The present paper explains the design and philosophy behind PYSCF that enables it to meet these twin objectives. With several case studies, we show how users can easily implement their own methods using PYSCF as a development environment. We then summarize the capabilities of PYSCF for molecular and solid-state simulations. Finally, we describe the growing ecosystem of projects that use PYSCF across the domains of quantum chemistry, materials science, machine learning and quantum information science.

The space-time (s-t) algebra provides a mathematical model for communication and computation using values encoded as events in discretized linear (Newtonian) time. Consequently, the input-output behavior of s-t algebra and implemented functions are consistent with the flow of time. The s-t algebra and functions are formally defined. A network design framework for s-t functions is described, and the design of temporal neural networks, a form of spiking neural networks, is discussed as an extended case study. Finally, the relationship with Allen's interval algebra is briefly discussed.

We introduce NetKet, a comprehensive open source framework for the study of many-body quantum systems using machine learning techniques. The framework is built around a general and flexible implementation of neural-network quantum states, which are used as a variational ansatz for quantum wave functions. NetKet provides algorithms for several key tasks in quantum many-body physics and quantum technology, namely quantum state tomography, supervised learning from wave-function data, and ground state searches for a wide range of customizable lattice models. Our aim is to provide a common platform for open research and to stimulate the collaborative development of computational methods at the interface of machine learning and many-body physics.

We use the recently-developed Heat-bath Configuration Interaction (HCI) algorithm as an efficient active-space solver to perform multi-configuration self-consistent field calculations (HCISCF) with large active spaces. We give a detailed derivation of the theory and show that difficulties associated with non-variationality of the HCI procedure can be overcome by making use of the Lagrangian formulation to calculate the HCI relaxed two body reduced density matrix. HCISCF is then used to study the electronic structure of butadiene, pentacene, and Fe-porphyrin. One of the most striking results of our work is that the converged active space orbitals obtained from HCISCF are relatively insensitive to the accuracy of the HCI calculation. This allows us to obtain nearly converged CASSCF energies with an estimated error of less than 1 mHa using the orbitals obtained from the HCISCF procedure in which the integral transformation is the dominant cost. For example, an HCISCF calculation on Fe-Porphyrin model complex with an active space of (44e, 44o) took only 412 seconds per iteration on a single node containing 28 cores, out of which 185 seconds were spent in the HCI calculation and the remaining 227 seconds were mainly used for integral transformation. Finally, we also show that active-space orbitals can be optimized using HCISCF to substantially speed up the convergence of the HCI energy to the Full CI limit because HCI is not invariant to unitary transformations within the active space., Comment: 14 pages, 4 figures

We report spectropolarimetric observations of the quasar E1821+643 (z=0.297), which suggest that it may be an example of gravitational recoil due to anisotropic emission of gravitational waves following the merger of a supermassive black hole (SMBH) binary. In total flux, the broad Balmer lines are redshifted by ~1000 km/s relative to the narrow lines and have highly red asymmetric profiles, whereas in polarized flux the broad H_alpha line exhibits a blueshift of similar magnitude and a strong blue asymmetry. We show that these observations are consistent with a scattering model in which the broad-line region has two components, moving with different bulk velocities away from the observer and towards a scattering region at rest in the host galaxy. If the high velocity system is identified as gas bound to the SMBH, this implies that the SMBH is itself moving with a velocity ~2100 km/s relative to the host galaxy. We discuss some implications of the recoil hypothesis and also briefly consider whether our observations can be explained in terms of scattering of broad-line emission originating from the active component of an SMBH binary, or from an outflowing wind., Comment: 18 pages, 4 figures, accepted for publication in the Astrophysical Journal as a Letter