Search

Despite the empirical success in various domains, it has been revealed that deep neural networks are vulnerable to maliciously perturbed input data that can dramatically degrade their performance. These are known as adversarial attacks. To counter adversarial attacks, adversarial training formulated as a form of robust optimization has been demonstrated to be effective. However, conducting adversarial training brings much computational overhead compared with standard training. In order to reduce the computational cost, we propose an annealing mechanism, annealing mechanism for adversarial training acceleration (Amata), to reduce the overhead associated with adversarial training. The proposed Amata is provably convergent, well-motivated from the lens of optimal control theory, and can be combined with existing acceleration methods to further enhance performance. It is demonstrated that, on standard datasets, Amata can achieve similar or better robustness with around 1/3-1/2 the computational time compared with traditional methods. In addition, Amata can be incorporated into other adversarial training acceleration algorithms (e.g., YOPO, Free, Fast, and ATTA), which leads to a further reduction in computational time on large-scale problems.

The properties of self-healing polymers are traditionally identified through destructive testing. This means that the mechanics are explored in hindsight with either theoretical derivations and/or simulations. Here, a self-healing property evolution using energy functional dynamical (SPEED) model is proposed to predict and understand the mechanics of self-healing of polymers using images of cuts dynamically healing over time. Using machine learning, an energy functional minimization (EFM) model extracted an effective underlying dynamical system from a time series of two-dimensional cut images on a self-healing polymer of constant thickness. This model can be used to capture the physics behind the self-healing dynamics in terms of potential and interface energies. When combined with a static property prediction model, the SPEED model can predict the macroscopic evolution of material properties after training only on a small set of experimental measurements. Such temporal evolutions are usually inaccessible from pure experiments or computational modeling due to the need for destructive testing. As an example, we validate this approach on toughness measurements of an intrinsic self-healing conductive polymer by capturing over 100 000 image frames of cuts to build the machine learning (ML) model. The results show that the SPEED model can be applied to predict the temporal evolution of macroscopic properties using few measurements as training data.

As materials researchers increasingly embrace machine-learning (ML) methods, it is natural to wonder what lessons can be learned from other fields undergoing similar developments. In this Review, we comparatively assess the evolution of applied ML in materials research, gameplaying and robotics. We observe ML being integrated into each field in three phases: first into discrete hardware and software tools (toolset integration); second across different steps in a workflow (workflow integration); and third through the incorporation, generation and representation of generalizable knowledge beyond any one study (knowledge integration). We identify transferrable lessons from gameplaying and robotics to materials research, including adaptive and accessible automation, the gamification of grand challenges to focus community efforts on specific workflow integrations and motivate benchmarks and canonical datasets, and the adoption of hybrid (data-based and model-based) algorithms that combine domain expertise and current learning to economically address high-complexity tasks. We identify opportunities for researchers from different fields to collaborate, including novel ways to represent and integrate a rich but heterogeneous corpus of knowledge (such as heuristics, physical laws, literature or data) with ML algorithms to create new knowledge, and safe and equitable deployment of technologies with societally beneficial outcomes.

Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. For example, only 0.82% out of 146k total materials in the open-access Materials Project database have a negative Poisson's ratio. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence to a global optimum or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI, that builds on conventional Bayesian optimization principles to quickly and efficiently optimize Needle-in-a-Haystack problems in both less time and fewer experiments by addressing the common convergence and pigeonholing issues. ZoMBI actively extracts knowledge from the previously best-performing evaluated experiments to iteratively zoom in the sampling search bounds towards the global optimum "needle" and then prunes the memory of low-performing historical experiments to accelerate compute times by reducing the algorithm time complexity from O(n3) to O(1), as the number of experiments sampled increases. Additionally, ZoMBI implements two custom acquisition functions that use active learning to further guide the sampling of new experiments towards the global optimum. We validate the algorithm's performance on two real-world 5-dimensional Needle-in-a-Haystack material property optimization datasets: discovery of auxetic Poisson's ratio materials and discovery of high thermoelectric figure of merit materials. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering materials in under 100 experiments that are up to 3x more highly optimized than those discovered by current state-of-the-art algorithms.

Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. For example, only $0.82\%$ out of $146$k total materials in the open-access Materials Project database have a negative Poisson's ratio. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence to a global optimum or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI. ZoMBI actively extracts knowledge from the previously best-performing evaluated experiments to iteratively zoom in the sampling search bounds towards the global optimum "needle" and then prunes the memory of low-performing historical experiments to accelerate compute times by reducing the algorithm time complexity from $O(n^3)$ to $O(\phi^3)$ for $\phi$ forward experiments per activation, which trends to a constant $O(1)$ over several activations. Additionally, ZoMBI implements two custom adaptive acquisition functions to further guide the sampling of new experiments toward the global optimum. We validate the algorithm's optimization performance on three real-world datasets exhibiting Needle-in-a-Haystack and further stress-test the algorithm's performance on an additional 174 analytical datasets. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering optima in under 100 experiments that are up to 3x more highly optimized than those discovered by similar methods MiP-EGO, TuRBO, and HEBO., Comment: Paper 16 pages; SI 6 pages

Realizing general inverse design could greatly accelerate the discovery of new materials with user-defined properties. However, state-of-the-art generative models tend to be limited to a specific composition or crystal structure. Herein, we present a framework capable of general inverse design (not limited to a given set of elements or crystal structures), featuring a generalized invertible representation that encodes crystals in both real and reciprocal space, and a property-structured latent space from a variational autoencoder (VAE). In three design cases, the framework generates 142 new crystals with user-defined formation energies, bandgap, thermoelectric (TE) power factor, and combinations thereof. These generated crystals, absent in the training database, are validated by first-principles calculations. The success rates (number of first-principles-validated target-satisfying crystals/number of designed crystals) ranges between 7.1% and 38.9%. These results represent a significant step toward property-driven general inverse design using generative models, although practical challenges remain when coupled with experimental synthesis. Agency for Science, Technology and Research (A*STAR) National Research Foundation (NRF) This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) program through the Singapore Massachusetts Institute of Technology (MIT) Alliance for Research and Technology’s Low Energy Electronic Systems (LEES) research program. F.O., S.S., and Q. Liang acknowledge support from Total Energies Research grant funded through MITei. Y.J. acknowledges the support from the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (No. 2021-0-02068, Artificial Intelligence Innovation Hub). J.L. and X.W. acknowledge support from the Ministry of Education Academic Research Fund R-279-000-532-114,. Y.L. is supported by the National Key Research and Development Program of China (Grant Nos. 2017YFB0702901 and 2017YFB0701502) and the National Natural Science Foundation of China (Grant No. 91641128). S.J. and K.H. acknowledge funding from the Accelerated Materials Development for Manufacturing Program at A*STAR via the AME Programmatic Fund by the Agency for Science, Technology and Research under Grant No. A1898b0043. G.X. is grateful for the support by the Scientific Computing and Data Analysis section of the Research Support Division at Okinawa Institute of Science and Technology Graduate University (OIST). Q.Li is supported by the National Research Foundation (NRF) fellowship grant NRFF13-2021-0106. A.G.A. acknowledges support from Solar Energy Research Institute of Singapore (SERIS). SERIS is a research institute at the National University of Singapore (NUS). SERIS is supported by the National University of Singapore (NUS), the National Research Foundation Singapore (NRF), the Energy Market Authority of Singapore (EMA), and the Singapore Economic Development Board (EDB).

In this perspective, the authors challenge the status quo of polymer innovation. The authors first explore how research in polymer design is conducted today, which is both time consuming and unable to capture the multi-scale complexities of polymers. The authors discuss strategies that could be employed in bringing together machine learning, data curation, high-throughput experimentation, and simulations, to build a system that can accurately predict polymer properties from their descriptors and enable inverse design that is capable of designing polymers based on desired properties.

Theoretical models and empirical analyses have combined to suggest that, for some ranges of climate, alternative stable states in vegetation structure may be possible: positive feedbacks with fire may differentiate savanna from forest, while positive feedbacks with hydrological processes may differentiate grassland from savanna. However, existing models have largely failed to include spatially explicit fire spread, dispersal, and hydrological processes, and as such ignore key features of vegetation systems. In this paper, we incorporate spatial and stochastic effects to form a comprehensive and realistic description of the dynamics of vegetation growth. By using novel computational tools such as path-sampling, we investigate the effect of noise and spatial coupling on the stability and the dynamics of forest-savanna transitions. Spatial effects change model behaviors: We find that spatial interactions between savanna and grassland result in the eventual nucleation and propagation of one of the states to invade the other. Which state wins depends on parameter values, which in turn depend on environmental conditions. Meanwhile, transitions between savanna and forest occur much less frequently, because an event such as a period of fire suppression in savanna or a fire in forest must be larger to successfully invade a landscape; this results in transitions between savanna and forest that, although not strictly history dependent, look temporally similar to hysteresis because the time lags involved in transitions are so large.

Though machine learning algorithms are able to achieve pattern recognition from the correlation between data and labels, the presence of spurious features in the data decreases the robustness of these learned relationships with respect to varied testing environments. This is known as out-of-distribution (OoD) generalization problem. Recently, invariant risk minimization (IRM) attempts to tackle this issue by penalizing predictions based on the unstable spurious features in the data collected from different environments. However, similar to domain adaptation or domain generalization, a prevalent non-trivial limitation in these works is that the environment information is assigned by human specialists, i.e. a priori, or determined heuristically. However, an inappropriate group partitioning can dramatically deteriorate the OoD generalization and this process is expensive and time-consuming. To deal with this issue, we propose a novel theoretically principled min-max framework to iteratively construct a worst-case splitting, i.e. creating the most challenging environment splittings for the backbone learning paradigm (e.g. IRM) to learn the robust feature representation. We also design a differentiable training strategy to facilitate the feasible gradient- based computation. Numerical experiments show that our algorithmic framework has achieved superior and stable performance in various datasets, such as Colored MNIST and Punctuated Stanford sentiment treebank (SST). Furthermore, we also find our algorithm to be robust even to a strong data poisoning attack. To the best of our knowledge, this is one of the first to adopt differentiable environment splitting method to enable stable predictions across environments without environment index information, which achieves the state-of-the-art performance on datasets with strong spurious correlation, such as Colored MNIST.

In materials science, the discovery of recipes that yield nanomaterials with defined optical properties is costly and time-consuming. In this study, we present a two-step framework for a machine learning-driven high-throughput microfluidic platform to rapidly produce silver nanoparticles with the desired absorbance spectrum. Combining a Gaussian process-based Bayesian optimization (BO) with a deep neural network (DNN), the algorithmic framework is able to converge towards the target spectrum after sampling 120 conditions. Once the dataset is large enough to train the DNN with sufficient accuracy in the region of the target spectrum, the DNN is used to predict the colour palette accessible with the reaction synthesis. While remaining interpretable by humans, the proposed framework efficiently optimizes the nanomaterial synthesis and can extract fundamental knowledge of the relationship between chemical composition and optical properties, such as the role of each reactant on the shape and amplitude of the absorbance spectrum. Agency for Science, Technology and Research (A*STAR) Published version We would like to thank Swee Liang Wong, Lim Yee-Fun, Xu Yang, Jatin Kumar, Liu Xiali and Li Jiali for equipment support and helpful discussions. Support was provided by the Accelerated Materials Development for Manufacturing Program at A*STAR via the AME Programmatic Fund by the Agency for Science, Technology and Research under Grant no. A1898b0043, (F.M.B., Z.R., T.H., W.K.W., F.Z., J.X., S.J., Z.M., D.B., K.H., S.A.K., Q.L., and X.W.) and Singapore’s National Research Foundation through the Singapore MIT Alliance for Research and Technology’s Low energy electronic systems (LEES) IRG (Z.R., I.P.S.T., and T.B.).

Traditionally, the discovery of new solid-state materials with user-defined properties is driven either by human intuition, heuristic chemical rules, and/or density functional theory (DFT). However, these methods have limitations, either in accessibly (domain expertise), accuracy, and/or throughput [1]. Consequently, the material search space remains underexplored, given order 105 reported compounds compared to order 1010 theorized ternary compounds [2]. To accelerate the exploration of new solid-state materials, a framework capable of inverse design for materials with user-defined properties is needed. Herein, we present a generalized framework for inverse design of crystals with user-defined properties, which include both ground-state and excited-state properties (e.g., thermoelectric power factor) using sparsely labelled training data. The key enabler of this inverse-design framework is a general and invertible crystallographic representation that encodes the crystallographic information into the representations in both real space and reciprocal space. The trained surrogate model achieves similar property prediction accuracy and precision as DFT calculations within seconds. Using the developed inverse-design framework, we design 79 new crystals with user-targeted formation energies, 17 crystals with targeted bandgap, and 27 crystals for potential thermoelectric applications. The compositions of those designed materials are unique and cannot be found in the training or test sets. We validate our predictions using first-principle calculations. ­­Toward bridging the gap between simulation and experiment, we demonstrate a naive synthesizability metric — predicting the existence of an ICSD record — and show this methodology can, in principle, include stability and/or synthesizability as a target metric, once consensus metrics are agreed upon by the field.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Process optimization of photovoltaic devices is a time-intensive, trial-and-error endeavor, which lacks full transparency of the underlying physics and relies on user-imposed constraints that may or may not lead to a global optimum. Herein, we demonstrate that embedding physics domain knowledge into a Bayesian network enables an optimization approach for gallium arsenide (GaAs) solar cells that identifies the root cause(s) of underperformance with layer-by-layer resolution and reveals alternative optimal process windows beyond traditional black-box optimization. Our Bayesian network approach links a key GaAs process variable (growth temperature) to material descriptors (bulk and interface properties, e.g., bulk lifetime, doping, and surface recombination) and device performance parameters (e.g., cell efficiency). For this purpose, we combine a Bayesian inference framework with a neural network surrogate device-physics model that is 100× faster than numerical solvers. With the trained surrogate model and only a small number of experimental samples, our approach reduces significantly the time-consuming intervention and characterization required by the experimentalist. As a demonstration of our method, in only five metal organic chemical vapor depositions, we identify a superior growth temperature profile for the window, bulk, and back surface field layer of a GaAs solar cell, without any secondary measurements, and demonstrate a 6.5% relative AM1.5G efficiency improvement above traditional grid search methods.

Combining high-throughput experiments with machine learning allows quick optimization of parameter spaces towards achieving target properties. In this study, we demonstrate that machine learning, combined with multi-labeled datasets, can additionally be used for scientific understanding and hypothesis testing. We introduce an automated flow system with high-throughput drop-casting for thin film preparation, followed by fast characterization of optical and electrical properties, with the capability to complete one cycle of learning of fully labeled ~160 samples in a single day. We combine regio-regular poly-3-hexylthiophene with various carbon nanotubes to achieve electrical conductivities as high as 1200 S/cm. Interestingly, a non-intuitive local optimum emerges when 10% of double-walled carbon nanotubes are added with long single wall carbon nanotubes, where the conductivity is seen to be as high as 700 S/cm, which we subsequently explain with high fidelity optical characterization. Employing dataset resampling strategies and graph-based regressions allows us to account for experimental cost and uncertainty estimation of correlated multi-outputs, and supports the proving of the hypothesis linking charge delocalization to electrical conductivity. We therefore present a robust machine-learning driven high-throughput experimental scheme that can be applied to optimize and understand properties of composites, or hybrid organic-inorganic materials., Comment: 10 pages, 5 figures, includes Supplementary Information

Hypertension is an increasingly serious public-health challenge worldwide. The traditional blood pressure measurement method could easily and reliably detect blood pressure. However, the delayed symptom onset may influence the screening of essential hypertension (EH). In addition, EH is significantly associated to cardiovascular disease, stroke and kidney disease. Hence, it is urgent to define associated biomarkers with early diagnosis potential for EH. A dried blood spot method integrated with direct infusion mass spectrometry (MS) metabolomic analysis was applied for the detection of metabolites toward 87 EH patients and 91 healthy controls (HC). Multiple algorithms were run on training set (62 EH and 64 HC) for selecting differential metabolites as potential biomarkers. A test set (25 EH and 27 HC) was used to verify and evaluate selected potential biomarkers. A novel blood biomarker model based on Gly, Orn, C10, Orn/Cit, Phe/Tyr, and C5-OH/C8 exhibited potential to differentiate EH patients from HC individuals, with a sensitivity of 0.8400 and a specificity of 0.8889 in test set. The metabolomic analysis of EH is beneficial to the definition of disease-associated biomarkers and the development of new diagnostic approaches. © 2018 IUBMB Life, 70(8):777-785, 2018.

Study of diffusion in solids is of fundamental importance in understanding the materials properties such as phase transformations, segregation, and corrosion in processing and operations. Reliable diffusion data is crucial in the successful design of materials and manufacturing processes. In this work, we use gradient boosting with decision tree regression, a robust machine learning method, to develop the model for predicting interstitial diffusion activation energies of the light element X (B, C, O, and N) in 54 metals with bcc, fcc and hcp lattices. With the introduction of elastic strain energy, a dominant concept of interstitial diffusion theory, we are able to train the machine learning model achieving high accuracy of R-squared 0.9 without being over-fitted by using 94 impurity – host binary systems reported in the literature. Furthermore, our study reveals that apart from elastic strain energy, the electronic interaction between the impurity and the host (represented by two fundamental parameters in the well-known Miedema model) also plays an important role. By accurately predicting the diffusion activation energies, we report their trends across 4th, 5th and 6th period and Lanthanides series of the host atoms. More importantly, our predicted data can be used to approximate the transport rates in materials for which little or no diffusion data are available, thus accelerating the design of a wide range of new application-specific materials.

This paper establishes the global well-posedness of the nonlinear Fokker–Planck equation for a noisy version of the Hegselmann–Krause model. The equation captures the mean-field behavior of a classic multiagent system for opinion dynamics. We prove the global existence, uniqueness, nonnegativity and regularity of the weak solution. We also exhibit a global stability condition, which delineates a forbidden region for consensus formation. This is the first nonlinear stability result derived for the Hegselmann–Krause model.