Search

The broad adoption of machine learning (ML)-based automated and autonomous experiments (AE) in physical characterization and synthesis requires development of strategies for understanding and intervention in the experimental workflow. Here, we introduce and realize strategies for post-acquisition forensic analysis applied to the deep kernel learning based AE scanning probe microscopy. This approach yields real-time and post-acquisition indicators of the progression of an active learning process interacting with an experimental system. We further illustrate that this approach can be extended towards human-in-the-loop autonomous experiments, where human operators make high-level decisions at high latencies setting the policies for AE, and the ML algorithm performs low-level fast decisions. The proposed approach is universal and can be extended to other physical and chemical imaging techniques and applications such as combinatorial library analysis. The full forensic analysis notebook is publicly available on GitHub at https://github.com/yongtaoliu/Forensics-DKL-BEPS., Comment: 24 pages, 8 figures

Electron and scanning probe microscopy produce vast amounts of data in the form of images or hyperspectral data, such as EELS or 4D STEM, that contain information on a wide range of structural, physical, and chemical properties of materials. To extract valuable insights from these data, it is crucial to identify physically separate regions in the data, such as phases, ferroic variants, and boundaries between them. In order to derive an easily interpretable feature analysis, combining with well-defined boundaries in a principled and unsupervised manner, here we present a physics augmented machine learning method which combines the capability of Variational Autoencoders to disentangle factors of variability within the data and the physics driven loss function that seeks to minimize the total length of the discontinuities in images corresponding to latent representations. Our method is applied to various materials, including NiO-LSMO, BiFeO3, and graphene. The results demonstrate the effectiveness of our approach in extracting meaningful information from large volumes of imaging data. The fully notebook containing implementation of the code and analysis workflow is available at https://github.com/arpanbiswas52/PaperNotebooks, Comment: 20 pages, 7 figures in main text, 4 figures in Supp Mat

Experimental science is enabled by the combination of synthesis, imaging, and functional characterization. Synthesis of a new material is typically followed by a set of characterization methods aiming to provide feedback for optimization or discover fundamental mechanisms. However, the sequence of synthesis and characterization methods and their interpretation, or research workflow, has traditionally been driven by human intuition and is highly domain specific. Here we explore concepts of scientific workflows that emerge at the interface between theory, characterization, and imaging. We discuss the criteria by which these workflows can be constructed for special cases of multi-resolution structural imaging and structural and functional characterization. Some considerations for theory-experiment workflows are provided. We further pose that the emergence of user facilities and cloud labs disrupt the classical progression from ideation, orchestration, and execution stages of workflow development and necessitate development of universal frameworks for workflow design, including universal hyper-languages describing laboratory operation, reward functions and their integration between domains, and policy development for workflow optimization. These tools will enable knowledge-based workflow optimization, enable lateral instrumental networks, sequential and parallel orchestration of characterization between dissimilar facilities, and empower distributed research., Comment: 33 pages; 8 figures

Discovery of the molecular candidates for applications in drug targets, biomolecular systems, catalysts, photovoltaics, organic electronics, and batteries, necessitates development of machine learning algorithms capable of rapid exploration of the chemical spaces targeting the desired functionalities. Here we introduce a novel approach for the active learning over the chemical spaces based on hypothesis learning. We construct the hypotheses on the possible relationships between structures and functionalities of interest based on a small subset of data and introduce them as (probabilistic) mean functions for the Gaussian process. This approach combines the elements from the symbolic regression methods such as SISSO and active learning into a single framework. The primary focus of constructing this framework is to approximate physical laws in an active learning regime toward a more robust predictive performance, as traditional evaluation on hold-out sets in machine learning doesn't account for out-of-distribution effects and may lead to a complete failure on unseen chemical space. Here, we demonstrate it for the QM9 dataset, but it can be applied more broadly to datasets from both domains of molecular and solid-state materials sciences.

Electronic transport and hysteresis in metal halide perovskites (MHPs) are key to the applications in photovoltaics, light emitting devices, and light and chemical sensors. These phenomena are strongly affected by the materials microstructure including grain boundaries, ferroic domain walls, and secondary phase inclusions. Here, we demonstrate an active machine learning framework for 'driving' an automated scanning probe microscope (SPM) to discover the microstructures responsible for specific aspects of transport behavior in MHPs. In our setup, the microscope can discover the microstructural elements that maximize the onset of conduction, hysteresis, or any other characteristic that can be derived from a set of current-voltage spectra. This approach opens new opportunities for exploring the origins of materials functionality in complex materials by SPM and can be integrated with other characterization techniques either before (prior knowledge) or after (identification of locations of interest for detail studies) functional probing., Comment: 19 pages; 7 figures

Underlying the rapidly increasing photovoltaic efficiency and stability of metal halide perovskites (MHPs) is the advance in the understanding of the microstructure of polycrystalline MHP thin film. Over the past decade, intense efforts have aimed to understand the effect of microstructure on MHP properties, including chemical heterogeneity, strain disorder, phase impurity, etc. It has been found that grain and grain boundary (GB) are tightly related to lots of microscale and nanoscale behavior in MHP thin film. Atomic force microscopy (AFM) is widely used to observe grain and boundary structures in topography and subsequently to study the correlative surface potential and conductivity of these structures. For now, most AFM measurements have been performed in imaging mode to study the static behavior, in contrast, AFM spectroscopy mode allows us to investigate the dynamic behavior of materials, e.g. conductivity under sweeping voltage. However, a major limitation of AFM spectroscopy measurements is that it requests manual operation by human operators, as such only limited data can be obtained, hindering systematic investigations of these microstructures. In this work, we designed a workflow combining the conductive AFM measurement with a machine learning (ML) algorithm to systematically investigate grain boundaries in MHPs. The trained ML model can extract GBs locations from the topography image, and the workflow drives the AFM probe to each GB location to perform a current-voltage (IV) curve automatically. Then, we are able to IV curves at all GB locations, allowing us to systematically understand the property of GBs. Using this method, we discover that the GB junction points are more photoactive, while most previous works only focused on the difference between GB and grains., Comment: 19 pages, 8 figures

Advanced electron microscopy workflows require an ecosystem of microscope instruments and computing systems possibly located at different sites to conduct remotely steered and automated experiments. Current workflow executions involve manual operations for steering and measurement tasks, which are typically performed from control workstations co-located with microscopes; consequently, their operational tempo and effectiveness are limited. We propose an approach based on separate data and control channels for such an ecosystem of Scanning Transmission Electron Microscopes (STEM) and computing systems, for which no general solutions presently exist, unlike the neutron and light source instruments. We demonstrate automated measurement transfers and remote steering of Nion STEM physical instruments over site networks. We propose a Virtual Infrastructure Twin (VIT) of this ecosystem, which is used to develop and test our steering software modules without requiring access to the physical instrument infrastructure. Additionally, we develop a VIT for a multiple laboratory scenario, which illustrates the applicability of this approach to ecosystems connected over wide-area networks, for the development and testing of software modules and their later field deployment., Comment: 11 pages, 16 figures, accepted at IEEE eScience 2022 conference

We pose that microscopy offers an ideal real-world experimental environment for the development and deployment of active Bayesian and reinforcement learning methods. Indeed, the tremendous progress achieved by machine learning (ML) and artificial intelligence over the last decade has been largely achieved via the utilization of static data sets, from the paradigmatic MNIST to the bespoke corpora of text and image data used to train large models such as GPT3, DALLE and others. However, it is now recognized that continuous, minute improvements to state-of-the-art do not necessarily translate to advances in real-world applications. We argue that a promising pathway for the development of ML methods is via the route of domain-specific deployable algorithms in areas such as electron and scanning probe microscopy and chemical imaging. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing. Favorable environment characteristics of scanning and electron microscopy include low risk, extensive availability of domain-specific priors and rewards, relatively small effects of exogeneous variables, and often the presence of both upstream first principles as well as downstream learnable physical models for both statics and dynamics. Recent developments in programmable interfaces, edge computing, and access to APIs facilitating microscope control, all render the deployment of ML codes on operational microscopes straightforward. We discuss these considerations and hope that these arguments will lead to creating a novel set of development targets for the ML community by accelerating both real-world ML applications and scientific progress.

The advent of modern, high-speed electron detectors has made the collection of multidimensional hyperspectral transmission electron microscopy datasets, such as 4D-STEM, a routine. However, many microscopists find such experiments daunting since such datasets' analysis, collection, long-term storage, and networking remain challenging. Some common issues are the large and unwieldy size of the said datasets, often running into several gigabytes, non-standardized data analysis routines, and a lack of clarity about the computing and network resources needed to utilize the electron microscope fully. However, the existing computing and networking bottlenecks introduce significant penalties in each step of these experiments, and thus, real-time analysis-driven automated experimentation for multidimensional TEM is exceptionally challenging. One solution is integrating microscopy with edge computing, where moderately powerful computational hardware performs the preliminary analysis before handing off the heavier computation to HPC systems. In this perspective, we trace the roots of computation in modern electron microscopy, demonstrate deep learning experiments running on an edge system, and discuss the networking requirements for tying together microscopes, edge computers, and HPC systems., Comment: Perspective on automated microscopy. 3 figures

Machine learning (ML) algorithms are showing a growing trend in helping the scientific communities across different disciplines and institutions to address large and diverse data problems. However, many available ML tools are programmatically demanding and computationally costly. The MLExchange project aims to build a collaborative platform equipped with enabling tools that allow scientists and facility users who do not have a profound ML background to use ML and computational resources in scientific discovery. At the high level, we are targeting a full user experience where managing and exchanging ML algorithms, workflows, and data are readily available through web applications. Since each component is an independent container, the whole platform or its individual service(s) can be easily deployed at servers of different scales, ranging from a personal device (laptop, smart phone, etc.) to high performance clusters (HPC) accessed (simultaneously) by many users. Thus, MLExchange renders flexible using scenarios -- users could either access the services and resources from a remote server or run the whole platform or its individual service(s) within their local network., Comment: The accepted version with DOI and IEEE copyright notice in the first page

Understanding the mechanisms and spatial correlations of crystallographic symmetry breaking in ferroelectric materials is essential to tuning their functional properties. While optical second harmonic generation (SHG) has long been utilized in ferroelectric studies, its capability for probing complex polar materials has yet to be fully realized. Here, we develop a SHG spectral imaging method implemented on a home-designed laser-scanning SHG microscope, and demonstrate its application for a model system of (K,Na)NbO3 single crystals. Supervised model fitting analysis produces comprehensive information about the polarization vector orientations and relative fractions of constituent domain variants as well as their thermal evolution across the polymorphic phase transitions. We observe an unexpected persistence of the orthorhombic phase at low temperatures, pointing to the phase competitions. Besides, we show that unsupervised matrix decomposition analysis can quickly and faithfully reveal domain configurations without a priori knowledge about specific material systems. The SHG spectral imaging method can be readily extended to other ferroelectric materials with potentials to be further enhanced., Comment: 16 pages, 4 figures

We demonstrate the application of machine learning for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. We show that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that we can use hyperspectral darkfield microscopy to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems., Comment: 12 pages, 5 figures

The robust approach for real-time analysis of the scanning transmission electron microscopy (STEM) data streams, based on the ensemble learning and iterative training (ELIT) of deep convolutional neural networks, is implemented on an operational microscope, enabling the exploration of the dynamics of specific atomic configurations under electron beam irradiation via an automated experiment in STEM. Combined with beam control, this approach allows studying beam effects on selected atomic groups and chemical bonds in a fully automated mode. Here, we demonstrate atomically precise engineering of single vacancy lines in transition metal dichalcogenides and the creation and identification of topological defects graphene. The ELIT-based approach opens the pathway toward the direct on-the-fly analysis of the STEM data and engendering real-time feedback schemes for probing electron beam chemistry, atomic manipulation, and atom by atom assembly.

Ferroelectricity in binary oxides including hafnia and zirconia have riveted the attention of the scientific community due to highly unconventional physical mechanisms and the potential for integration of these materials into semiconductor workflows. Over the last decade, it has been argued that behaviors such as wake-up phenomena and an extreme sensitivity to electrode and processing conditions suggests that ferroelectricity in these materials is strongly coupled with additional mechanisms, with possible candidates including the ionic subsystem or strain. Here we argue that the properties of these materials emerge due to the interplay between the bulk competition between ferroelectric and structural instabilities, similar to that in classical antiferroelectrics, coupled with non-local screening mediated by the finite density of states at surfaces and internal interfaces. Via decoupling of electrochemical and electrostatic controls realized via environmental and ultra-high vacuum PFM, we show that these materials demonstrate a rich spectrum of ferroic behaviors including partial pressure- and temperature-induced transitions between FE and AFE behaviors. These behaviors are consistent with an antiferroionic model and suggest novel strategies for hafnia-based device optimization.

Ferroelectric domain walls have emerged as one of the most fascinating objects in condensed matter physics due to the broad variability of functional behaviors they exhibit. However, the vast majority of domain walls studies have been focused on bias-induced dynamics and transport behaviors. Here, we introduce the scanning probe microscopy approach based on piezoresponse force microscopy (PFM) with a dynamically heated probe, combining local heating and local biasing of the material. This approach is used to explore the thermal polarization dynamics in soft Sn2P2S6 ferroelectrics, and allows for the exploration of phase transitions at individual domain walls. The strong and weak modulation regimes for the thermal PFM are introduced. The future potential applications of heated probe approach for functional SPM measurements including piezoelectric, elastic, microwave, and transport measurements are discussed.

We report the development and experimental implementation of the automated experiment workflows for the identification of the best predictive channel for a phenomenon of interest in spectroscopic measurements. The approach is based on the combination of ensembled deep kernel learning for probabilistic predictions and a basic reinforcement learning policy for channel selection. It allows the identification of which of the available observational channels, sampled sequentially, are most predictive of selected behaviors, and hence have the strongest correlations. We implement this approach for multimodal imaging in Piezoresponse Force Microscopy (PFM), with the behaviors of interest manifesting in piezoresponse spectroscopy. We illustrate the best predictive channel for polarization-voltage hysteresis loop and frequency-voltage hysteresis loop areas is amplitude in the model samples. The same workflow and code are universal and applicable for any multimodal imaging and local characterization methods., Comment: 17 pages, 5 figures

We develop and implement an automated experiment in multimodal imaging to probe structural, chemical, and functional behaviors in complex materials and elucidate the dominant physical mechanisms that control device function. Here the emergence of non-linear electromechanical responses in piezoresponse force microscopy (PFM) is explored. Non-linear responses in PFM can originate from multiple mechanisms, including intrinsic material responses often controlled by domain structure, surface topography that affects the mechanical phenomena at the tip-surface junction, and, potentially, the presence of surface contaminants. Using an automated experiment to probe the origins of non-linear behavior in model ferroelectric lead titanate (PTO) and ferroelectric Al0.93B0.07N films, it was found that PTO showed asymmetric nonlinear behavior across a/c domain walls and a broadened high nonlinear response region around c/c domain walls. In contrast, for Al0.93B0.07N, well-poled regions showed high linear piezoelectric responses paired with low non-linear responses and regions that were multidomain indicated low linear responses and high nonlinear responses. We show that formulating dissimilar exploration strategies in deep kernel learning as alternative hypotheses allows for establishing the preponderant physical mechanisms behind the non-linear behaviors, suggesting that this approach automated experiments can potentially discern between competing physical mechanisms. This technique can also be extended to electron, probe, and chemical imaging., Comment: 18 pages, 5 figures

Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g., balancing the Kullback Leibler (KL) and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolution during training. Because of the inefficiency of exhaustive search in a high-dimensional hyperparameter space for the expensive to train models, here we explored a latent Bayesian optimization (zBO) approach for the hyperparameter trajectory optimization for the unsupervised and semi-supervised ML and demonstrate for joint-VAE with rotational invariances. We demonstrate an application of this method for finding joint discrete and continuous rotationally invariant representations for MNIST and experimental data of a plasmonic nanoparticles material system. The performance of the proposed approach has been discussed extensively, where it allows for any high dimensional hyperparameter tuning or trajectory optimization of other ML models., Comment: 32 pages, including 11 figures in the main text and Appendixes with 2 figures. arXiv admin note: text overlap with arXiv:2108.12889

Automated chemical synthesis, materials fabrication, and spectroscopic physical measurements often bring forth the challenge of process trajectory optimization, i.e., discovering the time dependence of temperature, electric field, or pressure that gives rise to optimal properties. Due to the high dimensionality of the corresponding vectors, these problems are not directly amenable to Bayesian Optimization (BO). Here we propose an approach based on the combination of the generative statistical models, specifically variational autoencoders, and Bayesian optimization. Here, the set of potential trajectories is formed based on best practices in the field, domain intuition, or human expertise. The variational autoencoder is used to encode the thus generated trajectories as a latent vector, and also allows for the generation of trajectories via sampling from latent space. In this manner, Bayesian Optimization of the process is realized in the latent space of the system, reducing the problem to a low-dimensional one. Here we apply this approach to a ferroelectric lattice model and demonstrate that this approach allows discovering the field trajectories that maximize curl in the system. The analysis of the corresponding polarization and curl distributions allows the relevant physical mechanisms to be decoded., Comment: 22 pages and 9 figures

The functionality of ferroelastic domain walls in ferroelectric materials is explored in real-time via the in-situ implementation of computer vision algorithms in scanning probe microscopy (SPM) experiment. The robust deep convolutional neural network (DCNN) is implemented based on a deep residual learning framework (Res) and holistically-nested edge detection (Hed), and ensembled to minimize the out-of-distribution drift effects. The DCNN is implemented for real-time operations on SPM, converting the data stream into the semantically segmented image of domain walls and the corresponding uncertainty. We further demonstrate the pre-selected experimental workflows on thus discovered domain walls, and report alternating high- and low- polarization dynamic (out-of-plane) ferroelastic domain walls in a (PbTiO3) PTO thin film and high polarization dynamic (out-of-plane) at short ferroelastic walls (compared with long ferroelastic walls) in a lead zirconate titanate (PZT) thin film. This work establishes the framework for real-time DCNN analysis of data streams in scanning probe and other microscopies and highlights the role of out-of-distribution effects and strategies to ameliorate them in real time analytics., Comment: 21 pages, 7 figures

Recent progress in machine learning methods, and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs), have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian Process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian Processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. The discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods, and can be particularly impactful for destructive or irreversible measurements., Comment: 39 pages, 10 figures

The rapid development of machine learning (ML) methods has fundamentally affected numerous applications ranging from computer vision, biology, and medicine to accounting and text analytics. Until now, it was the availability of large and often labeled data sets that enabled significant breakthroughs. However, the adoption of these methods in classical physical disciplines has been relatively slow, a tendency that can be traced to the intrinsic differences between correlative approaches of purely data-based ML and the causal hypothesis-driven nature of physical sciences. Furthermore, anomalous behaviors of classical ML necessitate addressing issues such as explainability and fairness of ML. We also note the sequence in which deep learning became mainstream in different scientific disciplines - starting from medicine and biology and then towards theoretical chemistry, and only after that, physics - is rooted in the progressively more complex level of descriptors, constraints, and causal structures available for incorporation in ML architectures. Here we put forth that over the next decade, physics will become a new data, and this will continue the transition from dot-coms and scientific computing concepts of the 90ies to big data of 2000-2010 to deep learning of 2010-2020 to physics-enabled scientific ML.

Active learning methods are rapidly becoming the integral component of automated experiment workflows in imaging, materials synthesis, and computation. The distinctive aspect of many experimental scenarios is the presence of multiple information channels, including both the intrinsic modalities of the measurement system and the exogenous environment and noise signals. One of the key tasks in experimental studies is hence establishing which of these channels is predictive of the behaviors of interest. Here we explore the problem of discovery of the optimal predictive channel for structure-property relationships (in microscopy) using deep kernel learning for modality selection in an active experiment setting. We further pose that this approach can be directly applicable to similar active learning tasks in automated synthesis and the discovery of quantitative structure-activity relations in molecular systems.

Electron-beam (e-beam) manipulation of single dopant atoms in an aberration-corrected scanning transmission electron microscope is emerging as a method for directed atomic motion and atom-by-atom assembly. Until now, the dopant species have been limited to atoms closely matched to carbon in terms of ionic radius and capable of strong covalent bonding with carbon atoms in the graphene lattice. In situ dopant insertion into a graphene lattice has thus far been demonstrated only for Si, which is ubiquitously present as a contaminant in this material. Here, we achieve in situ manipulation of Pt atoms and their insertion into the graphene host matrix using the e-beam deposited Pt on graphene as a host system. We further demonstrate a mechanism for stabilization of the Pt atom, enabled through the formation of Si-stabilized Pt heteroatomic clusters attached to the graphene surface. This study provides evidence toward the universality of the e-beam assembly approach, opening a pathway for exploring cluster chemistry through direct assembly., Comment: 23 pages, 6 figures

Recent advances in electron, scanning probe, optical, and chemical imaging and spectroscopy yield bespoke data sets containing the information of structure and functionality of complex systems. In many cases, the resulting data sets are underpinned by low-dimensional simple representations encoding the factors of variability within the data. The representation learning methods seek to discover these factors of variability, ideally further connecting them with relevant physical mechanisms. However, generally the task of identifying the latent variables corresponding to actual physical mechanisms is extremely complex. Here, we explore an approach based on conditioning the data on the known (continuous) physical parameters, and systematically compare it with the previously introduced approach based on the invariant variational autoencoders. The conditional variational autoencoders (cVAE) approach does not rely on the existence of the invariant transforms, and hence allows for much greater flexibility and applicability. Interestingly, cVAE allows for limited extrapolation outside of the original domain of the conditional variable. However, this extrapolation is limited compared to the cases when true physical mechanisms are known, and the physical factor of variability can be disentangled in full. We further show that introducing the known conditioning results in the simplification of the latent distribution if the conditioning vector is correlated with the factor of variability in the data, thus allowing to separate relevant physical factors. We initially demonstrate this approach using 1D and 2D examples on a synthetic dataset and then extend it to the analysis of experimental data on ferroelectric domain dynamics visualized via Piezoresponse Force Microscopy., Comment: 20 pages, 8 figures

The polar states of uniaxial ferroelectric nanoparticles interacting with a surface system of electronic and ionic charges with a broad distribution of mobilities is explored, which corresponds to the experimental case of nanoparticles in solution or ambient conditions. The nonlinear interactions between the ferroelectric dipoles and surface charges with slow relaxation dynamics in an external field lead to the emergence of a broad range of paraelectric-like, antiferroelectric-like ionic, and ferroelectric-like ferroionic states. The crossover between these states can be controlled not only by the static characteristics of the surface charges, but also by their relaxation dynamics in the applied field. Obtained results are not only promising for advanced applications of ferroelectric nanoparticles in nanoelectronics and optoelectronics, they also offer strategies for experimental verification., Comment: 35 pages, 13 figures

The design of materials structure for optimizing functional properties and potentially, the discovery of novel behaviors is a keystone problem in materials science. In many cases microstructural models underpinning materials functionality are available and well understood. However, optimization of average properties via microstructural engineering often leads to combinatorically intractable problems. Here, we explore the use of the reinforcement learning (RL) for microstructure optimization targeting the discovery of the physical mechanisms behind enhanced functionalities. We illustrate that RL can provide insights into the mechanisms driving properties of interest in a 2D discrete Landau ferroelectrics simulator. Intriguingly, we find that non-trivial phenomena emerge if the rewards are assigned to favor physically impossible tasks, which we illustrate through rewarding RL agents to rotate polarization vectors to energetically unfavorable positions. We further find that strategies to induce polarization curl can be non-intuitive, based on analysis of learned agent policies. This study suggests that RL is a promising machine learning method for material design optimization tasks, and for better understanding the dynamics of microstructural simulations., Comment: 5 figures

We report the development and implementation of a hypothesis learning based automated experiment, in which the microscope operating in the autonomous mode identifies the physical laws behind the material's response. Specifically, we explore the bias induced transformations that underpin the functionality of broad classes of devices and functional materials from batteries and memristors to ferroelectrics and antiferroelectrics. Optimization and design of these materials require probing the mechanisms of these transformations on the nanometer scale as a function of the broad range of control parameters such as applied potential and time, often leading to experimentally intractable scenarios. At the same time, often the behaviors of these systems are understood within potentially competing theoretical models, or hypotheses. Here, we develop a hypothesis list that covers the possible limiting scenarios for the domain growth, including thermodynamic, domain wall pinning, and screening limited. We further develop and experimentally implement the hypothesis driven automated experiment in Piezoresponse Force Microscopy, autonomously identifying the mechanisms of the bias induced domain switching. This approach can be applied for a broad range of physical and chemical experiments with relatively low dimensional control parameter space and for which the possible competing models of the system behavior that ideally cover the full range of physical eventualities are known or can be created. These include other scanning probe microscopy modalities such as force distance curve measurements and nanoindentation, as well as materials synthesis and optimization., Comment: 25 pages, 6 figures

Over the last two decades, Electron Energy Loss Spectroscopy (EELS) imaging with a scanning transmission electron microscope (STEM) has emerged as a technique of choice for visualizing complex chemical, electronic, plasmonic, and phononic phenomena in complex materials and structures. The availability of the EELS data necessitates the development of methods to analyze multidimensional datasets with complex spatial and energy structures. Traditionally, the analysis of these data sets has been based on analysis of individual spectra, one at a time, whereas the spatial structure and correlations between individual spatial pixels containing the relevant information of the physics of underpinning processes have generally been ignored and analyzed only via the visualization as 2D maps. Here we develop a machine learning-based approach and workflows for the analysis of spatial structures in 3D EELS data sets using a combination of dimensionality reduction and multichannel rotationally-invariant variational autoencoders. This approach is illustrated for the analysis of both the plasmonic phenomena in a system of nanowires and in the core excitations in functional oxides using low loss and core loss EELS, respectively. The code developed in this manuscript is open sourced and freely available and provided as a Jupyter notebook for the interested reader here., Comment: Associated Jupyter Notebook: https://github.com/kevinroccapriore/Multichannel-rVAE

Local current-voltage characteristics for low-dimensional transition metal dichalcogenides (LD-TMD), as well as the reconstruction of their local density of states (LDOS) from scanning tunneling microscopy (STM) experiments is of fundamental interest and can be useful for advanced applications. Most of existing models are either hardly applicable for the LD-TMD of complex shape (e.g., those based on Simmons approach), or necessary for solving an ill-defined integral equation to deconvolute the unknown LDOS (e.g., those based on Tersoff approach). Using a serial expansion of Tersoff formulae, we propose a flexible method how to reconstruct the LDOS from local current-voltage characteristics measured in STM experiments. We established a set of key physical parameters, which characterize the tunneling current of a STM probe - sample contact and the sample LDOS expanded in Gaussian functions. Using a direct variational method coupled with a probabilistic analysis, we determine these parameters from the STM experiments for MoS2 nanoflakes with different number of layers. The main result is the reconstruction of the LDOS in a relatively wide energy range around a Fermi level, which allows insight in the local band structure of LD-TMDs. The reconstructed LDOS reveals pronounced size effects for the single-layer, bi-layer and three-layer MoS$_2$ nanoflakes, which we relate with low dimensionality and strong bending/corrugation of the nanoflakes. We hope that the proposed elaboration of the Tersoff approach allowing LDOS reconstruction will be of urgent interest for quantitative description of STM experiments, as well as useful for the microscopic physical understanding of the surface, strain and bending contribution to LD-TMDs electronic properties., Comment: 26 pages, 6 figures, 3 appendices

Combination of local heating and biasing at the tip-surface junction in temperature-assisted piezoresponse force microscopy (tPFM) opens the pathway for probing local temperature induced phase transitions in ferroics, exploring the temperature dependence of polarization dynamics in ferroelectrics, and potentially discovering coupled phenomena driven by strong temperature- and electric field gradients. Here, we analyze the signal formation mechanism in tPFM and explore the interplay between thermal- and bias-induced switching in model ferroelectric materials. We further explore the contributions of the flexoelectric and thermopolarization effects to the local electromechanical response, and demonstrate that the latter can be significant for "soft" ferroelectrics. These results establish the framework for quantitative interpretation of tPFM observations, predict the emergence the non-trivial switching and relaxation phenomena driven by non-local thermal gradient-induced polarization switching, and open a pathway for exploring the physics of thermopolarization effects in various non-centrosymmetric and centrosymmetric materials., Comment: 36 pages, 7 figures, 2 Appendixes

Machine learning is rapidly becoming an integral part of experimental physical discovery via automated and high-throughput synthesis, and active experiments in scattering and electron/probe microscopy. This, in turn, necessitates the development of active learning methods capable of exploring relevant parameter spaces with the smallest number of steps. Here we introduce an active learning approach based on co-navigation of the hypothesis and experimental spaces. This is realized by combining the structured Gaussian Processes containing probabilistic models of the possible system's behaviors (hypotheses) with reinforcement learning policy refinement (discovery). This approach closely resembles classical human-driven physical discovery, when several alternative hypotheses realized via models with adjustable parameters are tested during an experiment. We demonstrate this approach for exploring concentration-induced phase transitions in combinatorial libraries of Sm-doped BiFeO3 using Piezoresponse Force Microscopy, but it is straightforward to extend it to higher-dimensional parameter spaces and more complex physical problems once the experimental workflow and hypothesis-generation are available., Comment: Fixed typo in Eq. 1. Expanded the introduction part. The code reproducing Algorithm 1 is available at https://github.com/ziatdinovmax/hypoAL

We theoretically explore mechanisms that can potentially give rise to the steady-state and transient negative capacitance in a uniaxial ferroelectric film stabilized by a dielectric layer. The analytical expressions for the steady-state capacitance of a single-domain and poly-domain states are derived within Landau-Ginzburg-Devonshire approach and used to study the state stability vs. the domain splitting as a function of dielectric layer thickness. Analytical expressions for the critical thickness of the dielectric layer, polarization amplitude, equilibrium domain period and susceptibility are obtained and corroborated by finite element modelling. We further explore the possible effects of nonlinear screening by two types of screening charges at the ferroelectric-dielectric interface and show that if at least one of the screening charges is very slow, the total polarization dynamics can exhibit complex time- and voltage dependent behaviors that can be interpreted as an observable negative capacitance. In this setting, the transient negative capacitance effect is accompanied by almost zero dielectric susceptibility in a wide voltage range and low frequencies. These results may help to elucidate the observation of the transient negative capacitance in thin ferroelectric films, and identify materials systems that can give rise to the behavior., Comment: 46 pages, 10 figes, and supplementary materials on 17 pages

Automated experiments in 4D Scanning Transmission Electron Microscopy are implemented for rapid discovery of local structures, symmetry-breaking distortions, and internal electric and magnetic fields in complex materials. Deep kernel learning enables active learning of the relationship between local structure and a 4D-STEM based descriptors. With this, efficient and "intelligent" probing of dissimilar structural elements to discover desired physical functionality is made possible. This approach allows effective navigation of the sample in an automated fashion guided by either a pre-determined physical phenomenon, such as strongest electric field magnitude, or in an exploratory fashion. We verify the approach first on pre-acquired 4D-STEM data, and further implement it experimentally on an operational STEM. The experimental discovery workflow is demonstrated using graphene, and subsequently extended towards a lesser-known layered 2D van der Waal material, MnPS3. This approach establishes a paradigm for physics-driven automated 4D-STEM experiments that enable probing the physics of strongly correlated systems and quantum materials and devices, as well as exploration of beam sensitive materials., Comment: The data used for analysis as well as additional materials are available through the Jupyter notebook located at: https://github.com/kevinroccapriore/AE-DKL-4DSTEM

Electronic conduction pathways in dielectric thin films are explored using automated experiments in scanning probe microscopy (SPM). Here, we use large field of view scanning to identify the position of localized conductive spots and develop a SPM workflow to probe their dynamic behavior at higher spatial resolution as a function of time, voltage, and scanning process in an automated fashion. Using this approach, we observe the variable behaviors of the conductive spots in a 20 nm thick ferroelectric Hf0.54Zr0.48O2 film, where conductive spots disappear and reappear during continuous scanning. There are also new conductive spots that appear during scanning. The automated workflow is universal and can be integrated into a wide range of microscopy techniques, including SPM, electron microscopy, optical microscopy, and chemical imaging., Comment: 12 pages, 6 figures

Rapid emergence of the multimodal imaging in scanning probe, electron, and optical microscopies have brought forth the challenge of understanding the information contained in these complex data sets, targeting both the intrinsic correlations between different channels and further exploring the underpinning causal physical mechanisms. Here, we develop such analysis framework for the Piezoresponse Force Microscopy. We argue that under certain conditions, we can bootstrap experimental observations with the prior knowledge of materials structure to get information on certain non-observed properties, and demonstrate linear causal analysis for PFM observables. We further demonstrate that this approach can be extended to complex descriptors using the deep kernel learning (DKL) model. In this DKL analysis, we use the prior information on domain structure within the image to predict the physical properties. This analysis demonstrates the correlative relationships between morphology, piezoresponse, elastic property, etc. at nanoscale. The prediction of morphology and other physical parameters illustrates a mutual interaction between surface condition and physical properties in ferroelectric materials. This analysis is universal and can be extended to explore the correlative relationships of other multi-channel datasets., Comment: 19 pages, 7 figures

Ordered mesoscale structures in 2D materials induced by small misorientations have opened pathways for a wide variety of novel electronic, ferroelectric, and quantum phenomena. Until now, the only mechanism to induce this periodic ordering was via mechanical rotations between the layers, with the periodicity of the resulting moir\'e pattern being directly related to twist angle. Here we report a fundamentally new mechanism for emergence of mesoscopic periodic patterns in multilayer sulfur-containing metal phosphorous trichalcogenide, MnPS$_{3}$, induced by the electron beam. The formation under the beam of periodic hexagonal patterns with several characteristic length scales, nucleation and transitions between the phases, and local dynamics are demonstrated. The associated mechanisms are attributed to the relative contraction of the layers caused by beam-induced sulphur vacancy formation with subsequent ordering and lattice parameter change. As a result, the plasmonic response of the system is locally altered, suggesting an element of control over plasmon resonances by electron beam patterning. We pose that harnessing this phenomenon provides both insight into fundamental physics of quantum materials and opens a pathway towards device applications by enabling controlled periodic potentials on the atomic scale., Comment: Electron microscopy data and analysis codes are freely available here: https://github.com/kevinroccapriore/MnPS3

The development of high-resolution imaging methods such as electron and scanning probe microscopy and atomic probe tomography have provided a wealth of information on structure and functionalities of solids. The availability of this data in turn necessitates development of approaches to derive quantitative physical information, much like the development of scattering methods in the early XX century which have given one of the most powerful tools in condensed matter physics arsenal. Here, we argue that this transition requires adapting classical macroscopic definitions, that can in turn enable fundamentally new opportunities in understanding physics and chemistry. For example, many macroscopic definitions such as symmetry can be introduced locally only in a Bayesian sense, balancing the prior knowledge of materials' physics and experimental data to yield posterior probability distributions. At the same time, a wealth of local data allows fundamentally new approaches for the description of solids based on construction of statistical and physical generative models, akin to Ginzburg-Landau thermodynamic models. Finally, we note that availability of observational data opens pathways towards exploring causal mechanisms underpinning solid structure and functionality.

The emergent behaviors in thin films of a multiaxial ferroelectric due to an electrochemical coupling between the rotating polarization and surface ions are explored within the framework of the 2-4 Landau-Ginzburg-Devonshire (LGD) thermodynamic potential combined with the Stephenson-Highland (SH) approach. The combined LGD-SH approach allows to describe the electrochemical switching and rotation of polarization vector in the multiaxial ferroelectric film covered by surface ions with a charge density dependent to the relative partial oxygen pressure. We calculate the phase diagrams and analyze the dependence of polarization components on the applied voltage, and discuss the peculiarities of quasi-static ferroelectric, dielectric and piezoelectric hysteresis loops in thin strained multiaxial ferroelectric films. The nonlinear surface screening by oxygen ions makes the diagrams very different from the known diagrams of e.g., strained BaTiO3 films. Quite unexpectedly we predict the appearance of the ferroelectric reentrant phases. Obtained results point on the possibility to control the appearance and features of ferroelectric, dielectric and piezoelectric hysteresis in multiaxial FE films covered by surface ions by varying their concentration via the partial oxygen pressure. The LGD-SH description of a multiaxial FE film can be further implemented within the Bayesian optimization framework, opening the pathway towards predictive materials optimization., Comment: 51 pages, 7 figures in the main text, supplementary materials with 12 figures

Recent advances in (scanning) transmission electron microscopy have enabled routine generation of large volumes of high-veracity structural data on 2D and 3D materials, naturally offering the challenge of using these as starting inputs for atomistic simulations. In this fashion, theory will address experimentally emerging structures, as opposed to the full range of theoretically possible atomic configurations. However, this challenge is highly non-trivial due to the extreme disparity between intrinsic time scales accessible to modern simulations and microscopy, as well as latencies of microscopy and simulations per se. Addressing this issue requires as a first step bridging the instrumental data flow and physics-based simulation environment, to enable the selection of regions of interest and exploring them using physical simulations. Here we report the development of the machine learning workflow that directly bridges the instrument data stream into Python-based molecular dynamics and density functional theory environments using pre-trained neural networks to convert imaging data to physical descriptors. The pathways to ensure the structural stability and compensate for the observational biases universally present in the data are identified in the workflow. This approach is used for a graphene system to reconstruct optimized geometry and simulate temperature-dependent dynamics including adsorption of Cr as an ad-atom and graphene healing effects. However, it is universal and can be used for other material systems.

Optimization of materials performance for specific applications often requires balancing multiple aspects of materials functionality. Even for the cases where generative physical model of material behavior is known and reliable, this often requires search over multidimensional parameter space to identify low-dimensional manifold corresponding to required Pareto front. Here we introduce the multi-objective Bayesian Optimization (MOBO) workflow for the ferroelectric/anti-ferroelectric performance optimization for memory and energy storage applications based on the numerical solution of the Ginzburg-Landau equation with electrochemical or semiconducting boundary conditions. MOBO is a low computational cost optimization tool for expensive multi-objective functions, where we update posterior surrogate Gaussian process models from prior evaluations, and then select future evaluations from maximizing an acquisition function. Using the parameters for a prototype bulk antiferroelectric (PbZrO3), we first develop a physics-driven decision tree of target functions from the loop structures. We further develop a physics-driven MOBO architecture to explore multidimensional parameter space and build Pareto-frontiers by maximizing two target functions jointly: energy storage and loss. This approach allows for rapid initial materials and device parameter selection for a given application and can be further expanded towards the active experiment setting. The associated notebooks provide both the tutorial on MOBO and allow to reproduce the reported analyses and apply them to other systems (https://github.com/arpanbiswas52/MOBO_AFI_Supplements)., Comment: 40 pages, including 12 figures in the main text and Appendixes with 4 figures

Both experimental and computational methods for the exploration of structure, functionality, and properties of materials often necessitate the search across broad parameter spaces to discover optimal experimental conditions and regions of interest in the image space or parameter space of computational models. The direct grid search of the parameter space tends to be extremely time-consuming, leading to the development of strategies balancing exploration of unknown parameter spaces and exploitation towards required performance metrics. However, classical Bayesian optimization strategies based on the Gaussian process (GP) do not readily allow for the incorporation of the known physical behaviors or past knowledge. Here we explore a hybrid optimization/exploration algorithm created by augmenting the standard GP with a structured probabilistic model of the expected system's behavior. This approach balances the flexibility of the non-parametric GP approach with a rigid structure of physical knowledge encoded into the parametric model. The fully Bayesian treatment of the latter allows additional control over the optimization via the selection of priors for the model parameters. The method is demonstrated for a noisy version of the classical objective function used to evaluate optimization algorithms and further extended to physical lattice models. This methodology is expected to be universally suitable for injecting prior knowledge in the form of physical models and past data in the Bayesian optimization framework, Comment: Expanded the discussion and added additional info to the supplemental materials

Emergent functionalities of structural and topological defects in ferroelectric materials underpin an extremely broad spectrum of applications ranging from domain wall electronics to high dielectric and electromechanical responses. Many of these have been discovered and quantified via local scanning probe microscopy methods. However, the search for these functionalities has until now been based by either trial and error or using auxiliary information such as topography or domain wall structure to identify potential objects of interest based on the intuition of operator or preexisting hypotheses, with subsequent manual exploration. Here, we report the development and implementation of a machine learning framework that actively discovers relationships between local domain structure and polarization switching characteristics in ferroelectric materials encoded in the hysteresis loop. The latter and descriptors such as nucleation bias, coercive bias, hysteresis loop area, or more complex functionals of hysteresis loop shape and corresponding uncertainties are used to guide the discovery via automated piezoresponse force microscopy (PFM) and spectroscopy experiments. As such, this approach combines the power of machine learning methods to learn the correlative relationships between high dimensional data, and human-based physics insights encoded in the acquisition function. For ferroelectric, this automated workflow demonstrates that the discovery path and sampling points of on-field and off-field hysteresis loops are largely different, indicating the on-field and off-field hysteresis loops are dominated by different mechanisms. The proposed approach is universal and can be applied to a broad range of modern imaging and spectroscopy methods ranging from other scanning probe microscopy modalities to electron microscopy and chemical imaging., Comment: 23 pages, 6 figures

Physics-driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here we develop and experimentally implement a deep kernel learning workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics-based selection of acquisition function, which autonomously guides the navigation of the image space. Compared to classical Bayesian optimization methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure-property relationships. In combination with the flexible scalarizer function that allows to ascribe the degree of physical interest to predicted spectra, this enables physical discovery in automated experiment. Here, this approach is illustrated for nanoplasmonic studies of nanoparticles and experimentally implemented in a truly autonomous fashion for bulk- and edge plasmon discovery in MnPS3, a lesser-known beam-sensitive layered 2D material. This approach is universal, can be directly used as-is with any specimen, and is expected to be applicable to any probe-based microscopic techniques including other STEM modalities, Scanning Probe Microscopies, chemical, and optical imaging.

Scanning transmission electron microscopy (STEM) is now the primary tool for exploring functional materials on the atomic level. Often, features of interest are highly localized in specific regions in the material, such as ferroelectric domain walls, extended defects, or second phase inclusions. Selecting regions to image for structural and chemical discovery via atomically resolved imaging has traditionally proceeded via human operators making semi-informed judgements on sampling locations and parameters. Recent efforts at automation for structural and physical discovery have pointed towards the use of "active learning" methods that utilize Bayesian optimization with surrogate models to quickly find relevant regions of interest. Yet despite the potential importance of this direction, there is a general lack of certainty in selecting relevant control algorithms and how to balance a priori knowledge of the material system with knowledge derived during experimentation. Here we address this gap by developing the automated experiment workflows with several combinations to both illustrate the effects of these choices and demonstrate the tradeoffs associated with each in terms of accuracy, robustness, and susceptibility to hyperparameters for structural discovery. We discuss possible methods to build descriptors using the raw image data and deep learning based semantic segmentation, as well as the implementation of variational autoencoder based representation. Furthermore, each workflow is applied to a range of feature sizes including NiO pillars within a La:SrMnO$_3$ matrix, ferroelectric domains in BiFeO$_3$, and topological defects in graphene. The code developed in this manuscript are open sourced and will be released at github.com/creangnc/AE_Workflows., Comment: 18 pages, 8 figures, manuscript being submitted to MLST