Search

Atomic layer deposition (ALD) is a promising technique to functionalize particle surfaces for energy applications including energy storage, catalysis, and decarbonization. In this work, we present a set of models of ALD particle coating to explore the transition from lab scale to manufacturing. Our models encompass the main particle coating manufacturing approaches including rotary bed, fluidized bed, and continuously vibrating reactors. These models provide key metrics, such as throughput and precursor utilization, required to evaluate the scalability of ALD manufacturing approaches and their feasibility in the context of energy applications. Our results show that designs that force the precursor to flow through fluidized particles transition faster to a transport-limited regime where throughput is maximized. They also exhibit higher precursor utilization. In the context of continuous processes, our models indicate that it is possible to achieve self-extinguishing processes with almost 100% precursor utilization. A comparison with past experimental results of ALD in fluidized bed reactors shows excellent qualitative and quantitative agreement.

Lifelong learning - an agent's ability to learn throughout its lifetime - is a hallmark of biological learning systems and a central challenge for artificial intelligence (AI). The development of lifelong learning algorithms could lead to a range of novel AI applications, but this will also require the development of appropriate hardware accelerators, particularly if the models are to be deployed on edge platforms, which have strict size, weight, and power constraints. Here, we explore the design of lifelong learning AI accelerators that are intended for deployment in untethered environments. We identify key desirable capabilities for lifelong learning accelerators and highlight metrics to evaluate such accelerators. We then discuss current edge AI accelerators and explore the future design of lifelong learning accelerators, considering the role that different emerging technologies could play.

The ability to learn continuously from an incoming data stream without catastrophic forgetting is critical to designing intelligent systems. Many approaches to continual learning rely on stochastic gradient descent and its variants that employ global error updates, and hence need to adopt strategies such as memory buffers or replay to circumvent its stability, greed, and short-term memory limitations. To address this limitation, we have developed a biologically inspired lightweight neural network architecture that incorporates synaptic plasticity mechanisms and neuromodulation and hence learns through local error signals to enable online continual learning without stochastic gradient descent. Our approach leads to superior online continual learning performance on Split-MNIST, Split-CIFAR-10, and Split-CIFAR-100 datasets compared to other memory-constrained learning approaches and matches that of the state-of-the-art memory-intensive replay-based approaches. We further demonstrate the effectiveness of our approach by integrating key design concepts into other backpropagation-based continual learning algorithms, significantly improving their accuracy. Our results provide compelling evidence for the importance of incorporating biological principles into machine learning models and offer insights into how we can leverage them to design more efficient and robust systems for online continual learning.

In this work we have extended AutoML inspired approaches to the exploration and optimization of neuromorphic architectures. Through the integration of a parallel asynchronous model-based search approach with a simulation framework to simulate spiking architectures, we are able to efficiently explore the configuration space of neuromorphic architectures and identify the subset of conditions leading to the highest performance in a targeted application. We have demonstrated this approach on an exemplar case of real time, on-chip learning application. Our results indicate that we can effectively use optimization approaches to optimize complex architectures, therefore providing a viable pathway towards application-driven codesign.

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future., Comment: To appear in Neural Networks

We have developed a model for online continual or lifelong reinforcement learning (RL) inspired on the insect brain. Our model leverages the offline training of a feature extraction and a common general policy layer to enable the convergence of RL algorithms in online settings. Sharing a common policy layer across tasks leads to positive backward transfer, where the agent continuously improved in older tasks sharing the same underlying general policy. Biologically inspired restrictions to the agent's network are key for the convergence of RL algorithms. This provides a pathway towards efficient online RL in resource-constrained scenarios.

In this work we explore the application of deep neural networks to the optimization of atomic layer deposition processes based on thickness values obtained at different points of an ALD reactor. We introduce a dataset designed to train neural networks to predict saturation times based on the dose time and thickness values measured at different points of the reactor for a single experimental condition. We then explore different artificial neural network configurations, including depth (number of hidden layers) and size (number of neurons in each layers) to better understand the size and complexity that neural networks should have to achieve high predictive accuracy. The results obtained show that trained neural networks can accurately predict saturation times without requiring any prior information on the surface kinetics. This provides a viable approach to minimize the number of experiments required to optimize new ALD processes in a known reactor. However, the datasets and training procedure depend on the reactor geometry.

We have developed a simulation tool to model self-limited processes such as atomic layer deposition and atomic layer etching inside reactors of arbitrary geometry. In this work, we have applied this model to two standard types of cross-flow reactors: a cylindrical reactor and a model 300 mm wafer reactor, and explored both ideal and non-ideal self-limited kinetics. For the cylindrical tube reactor the full simulation results agree well with analytic expressions obtained using a simple plug flow model, though the presence of axial diffusion tends to soften growth profiles with respect to the plug flow case. Our simulations also allowed us to model the output of in-situ techniques such as quartz crystal microbalance and mass spectrometry, providing a way of discriminating between ideal and non-ideal surface kinetics using in-situ measurements. We extended the simulations to consider two non-ideal self-limited processes: soft-saturating processes characterized by a slow reaction pathway, and processes where surface byproducts can compete with the precursor for the same pool of adsorption sites, allowing us to quantify their impact in the thickness variability across 300 mm wafer substrates.

Neuromorphic architectures are ideally suited for the implementation of smart sensors able to react, learn, and respond to a changing environment. Our work uses the insect brain as a model to understand how heterogeneous architectures, incorporating different types of neurons and encodings, can be leveraged to create systems integrating input processing, evaluation, and response. Here we show how the combination of time and rate encodings can lead to fast sensors that are able to generate a hypothesis on the input in only a few cycles and then use that hypothesis as secondary input for more detailed analysis., Comment: Paper accepted in GOMACTech 2018

The ability to learn continuously from an incoming data stream without catastrophic forgetting is critical for designing intelligent systems. Many existing approaches to continual learning rely on stochastic gradient descent and its variants. However, these algorithms have to implement various strategies, such as memory buffers or replay, to overcome well-known shortcomings of stochastic gradient descent methods in terms of stability, greed, and short-term memory. To that end, we develop a biologically-inspired light weight neural network architecture that incorporates local learning and neuromodulation to enable input processing over data streams and online learning. Next, we address the challenge of hyperparameter selection for tasks that are not known in advance by implementing transfer metalearning: using a Bayesian optimization to explore a design space spanning multiple local learning rules and their hyperparameters, we identify high performing configurations in classical single task online learning and we transfer them to continual learning tasks with task-similarity considerations. We demonstrate the efficacy of our approach on both single task and continual learning setting. For the single task learning setting, we demonstrate superior performance over other local learning approaches on the MNIST, Fashion MNIST, and CIFAR-10 datasets. Using high performing configurations metalearned in the single task learning setting, we achieve superior continual learning performance on Split-MNIST, and Split-CIFAR-10 data as compared with other memory-constrained learning approaches, and match that of the state-of-the-art memory-intensive replay-based approaches.

In this work we explore recurrent representations of leaky integrate and fire neurons operating at a timescale equal to their absolute refractory period. Our coarse time scale approximation is obtained using a probability distribution function for spike arrivals that is homogeneously distributed over this time interval. This leads to a discrete representation that exhibits the same dynamics as the continuous model, enabling efficient large scale simulations and backpropagation through the recurrent implementation. We use this approach to explore the training of deep spiking neural networks including convolutional, all-to-all connectivity, and maxpool layers directly in Pytorch. We found that the recurrent model leads to high classification accuracy using just 4-long spike trains during training. We also observed a good transfer back to continuous implementations of leaky integrate and fire neurons. Finally, we applied this approach to some of the standard control problems as a first step to explore reinforcement learning using neuromorphic chips., Comment: Paper accepted in ICONS 2020

We report the optical, electrical, and structural properties of Si doped $\beta$-Ga$_2$O$_3$ films grown on (010)-oriented $\beta$-Ga$_2$O$_3$ substrate via HVPE. Our results show that, despite growth rates that are more than one order of magnitude faster than MOCVD, films with mobility values of up to 95 cm$^2$V$^{-1}$s$^{-1}$ at a carrier concentration of 1.3$\times$10$^{17}$ cm$^{-3}$ can be achieved using this technique, with all Si-doped samples showing n-type behavior with carrier concentrations in the range of 10$^{17}$ to 10$^{19}$ cm$^{-3}$. All samples showed similar room temperature photoluminescence, with only the samples with the lowest carrier concentration showing the presence of a blue luminescence, and the Raman spectra exhibiting only phonon modes that belong to $\beta$-Ga$_2$O$_3$, indicating that the Ga$_2$O$_3$ films are phase pure and of high crystal quality. We further evaluated the epitaxial quality of the films by carrying out grazing incidence X-ray scattering measurements, which allowed us to discriminate the bulk and film contributions. Finally, MOS capacitors were fabricated using ALD HfO$_2$ to perform C-V measurements. The carrier concentration and dielectric values extracted from the C-V characteristics are in good agreement with Hall probe measurements. These results indicate that HVPE has a strong potential to yield device-quality $\beta$-Ga$_2$O$_3$ films that can be utilized to develop vertical devices for high-power electronics applications.

The ability to learn and adapt in real time is a central feature of biological systems. Neuromorphic architectures demonstrating such versatility can greatly enhance our ability to efficiently process information at the edge. A key challenge, however, is to understand which learning rules are best suited for specific tasks and how the relevant hyperparameters can be fine-tuned. In this work, we introduce a conceptual framework in which the learning process is integrated into the network itself. This allows us to cast meta-learning as a mathematical optimization problem. We employ DeepHyper, a scalable, asynchronous model-based search, to simultaneously optimize the choice of meta-learning rules and their hyperparameters. We demonstrate our approach with two different datasets, MNIST and FashionMNIST, using a network architecture inspired by the learning center of the insect brain. Our results show that optimal learning rules can be dataset-dependent even within similar tasks. This dependency demonstrates the importance of introducing versatility and flexibility in the learning algorithms. It also illuminates experimental findings in insect neuroscience that have shown a heterogeneity of learning rules within the insect mushroom body.

We have characterized the growth of ZnO using sequential infiltration synthesis (SiS) on PS-b-PMMA block copolymers (BCP) of spherical and cylindrical sub-20nm morphologies and studied how the photoluminescence of these nanostructures varies per its seed layer. Investigation of these structures was done using atomic force microscopy (AFM), spectrofluorometry, Raman spectroscopy, and scanning electron microscopy (SEM). We report blue-shifted photoemission at 335 nm (3.70 eV), suggesting quantum confinement effects. This UV-photoluminescence can be translated into emitter sizes of roughly 1.5nm in radius. Furthermore, samples of ZnO prepared with an alumina seed layer showed additional defect state photoemission at 470 nm and 520 nm for spherical and cylindrical BCP morphologies, respectively. Defect photoemission was not observed in samples prepared without a seed layer. Raman and EXAFS data suggest lack of long range order between the ZnO nanostructures during early stages of infiltrated ZnO growth and therefore supports the blue shift emission is due to confinement. Our work demonstrates that ZnO nanostructures grown on PS-b-PMMA via infiltration are advantageous in uniformity and size, and exhibit unique fluorescence properties. These observations suggest that infiltrated ZnO in PS-b-PMMA nanostructures lends itself to a new regime of applications in photonics and quantum materials., Comment: 12 pages, 10 figures

We describe an instrument that exploits the ongoing revolution in synchrotron sources, optics, and detectors to enable in situ studies of metal-organic vapor phase epitaxy (MOVPE) growth of III-nitride materials using coherent x-ray methods. The system includes high-resolution positioning of the sample and detector including full rotations, an x-ray transparent chamber wall for incident and diffracted beam access over a wide angular range, and minimal thermal sample motion, giving the sub-micron positional stability and reproducibility needed for coherent x-ray studies. The instrument enables surface x-ray photon correlation spectroscopy, microbeam diffraction, and coherent diffraction imaging of atomic-scale surface and film structure and dynamics during growth, to provide fundamental understanding of MOVPE processes., Comment: 13 pages, 13 figures, published in RSI v2 revision, clarified several points for reader from suggestions of referees, fixed typo Please cite paper as : G. Ju, et al., Review of Scientific Instruments 88, 035113 (2017)