Search

Emissive organic radicals are currently of great interest for their potential use in the next generation of highly efficient organic light emitting diode (OLED) devices and as molecular qubits. However, simulating their optoelectronic properties is challenging, largely due to spin-contamination and the multireference character of their excited states. Here we present a data-driven approach where, for the first time, the excited electronic states of organic radicals are learned directly from experimental excited state data, using a much smaller amount of data than required by typical Machine Learning. We adopt ExROPPP, a fast and spin-pure semiempirical method for calculation of excited states of radicals, as a surrogate physical model for which we learn the optimal set of parameters. We train the model on 81 previously published radicals and find that the trained model is a huge improvement over ExROPPP with literature parameters, giving RMS and mean absolute errors of 0.24 and 0.16 eV respectively with R$^2$ and SRCC of 0.86 and 0.88 respectively. We synthesise four new radicals and validate the model on their spectra, finding even lower errors and similar correlation as for the testing set. This model paves the way for high throughput discovery of next-generation radical based optoelectronics.

Overcoming the challenge of limited data availability within materials science is crucial for the broad-based applicability of machine learning within materials science. One pathway to overcome this limited data availability is to use the framework of transfer learning (TL), where a pre-trained (PT) machine learning model (on a larger dataset) can be fine-tuned (FT) on a target (typically smaller) dataset. Our study systematically explores the effectiveness of various PT/FT strategies to learn and predict material properties with limited data. Specifically, we leverage graph neural networks (GNNs) to PT/FT on seven diverse curated materials datasets, encompassing sizes ranging from 941 to 132,752 datapoints. We consider datasets that cover a spectrum of material properties, ranging from band gaps (electronic) to formation energies (thermodynamic) and shear moduli (mechanical). We study the influence of PT and FT dataset sizes, strategies that can be employed for FT, and other hyperparameters on pair-wise TL among the datasets considered. We find our pair-wise PT-FT models to consistently outperform models trained from scratch on the target datasets. Importantly, we develop a GNN framework that is simultaneously PT on multiple properties (MPT), enabling the construction of generalized GNN models. Our MPT models outperform pair-wise PT-FT models on several datasets considered, and more significantly, on a 2D material band gap dataset that is completely out-of-distribution from the PT datasets. Finally, we expect our PT/FT and MPT frameworks to be generalizable to other GNNs and materials properties, which can accelerate materials design and discovery for various applications.

Understanding the way cells communicate, co-locate, and interrelate is essential to furthering our understanding of how the body functions. H&E is widely available, however, cell subtyping often requires expert knowledge and the use of specialized stains. To reduce the annotation burden, AI has been proposed for the classification of cells on H&E. For example, the recent Colon Nucleus Identification and Classification (CoNIC) Challenge focused on labeling 6 cell types on H&E of the colon. However, the CoNIC Challenge was unable to classify epithelial subtypes (progenitor, enteroendocrine, goblet), lymphocyte subtypes (B, helper T, cytotoxic T), and connective subtypes (fibroblasts). We use inter-modality learning to label previously un-labelable cell types on H&E. We take advantage of multiplexed immunofluorescence (MxIF) histology to label 14 cell subclasses. We performed style transfer on the same MxIF tissues to synthesize realistic virtual H&E which we paired with the MxIF-derived cell subclassification labels. We evaluated the efficacy of using a supervised learning scheme where the input was realistic-quality virtual H&E and the labels were MxIF-derived cell subclasses. We assessed our model on private virtual H&E and public real H&E. On virtual H&E, we were able to classify helper T cells and epithelial progenitors with positive predictive values of $0.34 \pm 0.15$ (prevalence $0.03 \pm 0.01$) and $0.47 \pm 0.1$ (prevalence $0.07 \pm 0.02$) respectively, when using ground truth centroid information. On real H&E we could classify helper T cells and epithelial progenitors with upper bound positive predictive values of $0.43 \pm 0.03$ (parent class prevalence 0.21) and $0.94 \pm 0.02$ (parent class prevalence 0.49) when using ground truth centroid information. This is the first work to provide cell type classification for helper T and epithelial progenitor nuclei on H&E., Comment: arXiv admin note: text overlap with arXiv:2401.05602

Understanding the way cells communicate, co-locate, and interrelate is essential to understanding human physiology. Hematoxylin and eosin (H&E) staining is ubiquitously available both for clinical studies and research. The Colon Nucleus Identification and Classification (CoNIC) Challenge has recently innovated on robust artificial intelligence labeling of six cell types on H&E stains of the colon. However, this is a very small fraction of the number of potential cell classification types. Specifically, the CoNIC Challenge is unable to classify epithelial subtypes (progenitor, endocrine, goblet), lymphocyte subtypes (B, helper T, cytotoxic T), or connective subtypes (fibroblasts, stromal). In this paper, we propose to use inter-modality learning to label previously un-labelable cell types on virtual H&E. We leveraged multiplexed immunofluorescence (MxIF) histology imaging to identify 14 subclasses of cell types. We performed style transfer to synthesize virtual H&E from MxIF and transferred the higher density labels from MxIF to these virtual H&E images. We then evaluated the efficacy of learning in this approach. We identified helper T and progenitor nuclei with positive predictive values of $0.34 \pm 0.15$ (prevalence $0.03 \pm 0.01$) and $0.47 \pm 0.1$ (prevalence $0.07 \pm 0.02$) respectively on virtual H&E. This approach represents a promising step towards automating annotation in digital pathology.

Extended defects, including exposed surfaces and grain boundaries, are critical to the properties of polycrystalline solid electrolytes in all-solid-state batteries (ASSBs). These defects can significantly alter the mechanical and electronic properties of solid electrolytes, with direct manifestations on the performance of ASSBs. Here, by building a library of 590 surfaces and grain boundaries of 11 relevant solid electrolytes $-$including halides, oxides, and sulfides$-$ their electronic, mechanical, and thermodynamic characteristics are linked to the functional properties of polycrystalline solid electrolytes. It is found that the energy required to mechanically ``separate'' grain boundaries can be significantly lower than in the bulk region of materials, which can trigger preferential cracking of solid electrolyte particles in the grain boundary regions. The brittleness of ceramic solid electrolytes, inferred from the predicted low fracture toughnesses at the grain boundaries, contributes to their cracking under local pressure imparted by Lithium or Sodium penetration in the grain boundaries. Extended defects of solid electrolytes introduce new electronic ``interfacial'' states within bandgaps of solid electrolytes. These interfacial states alter and possibly increase locally the availability of free electrons and holes in solid electrolytes. Factoring effects arising from extended defects appear crucial to explain electrochemical and $-$mechanical observations in ASSBs.

This study proposes a novel dynamical mechanism for pattern recognition discovered by interpreting a recurrent neural network (RNN) trained on a simple task inspired by the SET card game. We interpreted the trained RNN as recognizing patterns via phase shifts in a low-dimensional limit cycle in a manner analogous to transitions in a finite state automaton (FSA). We further validated this interpretation by handcrafting a simple oscillatory model that reproduces the dynamics of the trained RNN. Our findings not only suggest of a potential dynamical mechanism capable of pattern recognition, but also suggest of a potential neural implementation of FSA. Above all, this work contributes to the growing discourse on deep learning model interpretability., Comment: 7 pages, 8 figures

Crohn's disease (CD) is a chronic and relapsing inflammatory condition that affects segments of the gastrointestinal tract. CD activity is determined by histological findings, particularly the density of neutrophils observed on Hematoxylin and Eosin stains (H&E) imaging. However, understanding the broader morphometry and local cell arrangement beyond cell counting and tissue morphology remains challenging. To address this, we characterize six distinct cell types from H&E images and develop a novel approach for the local spatial signature of each cell. Specifically, we create a 10-cell neighborhood matrix, representing neighboring cell arrangements for each individual cell. Utilizing t-SNE for non-linear spatial projection in scatter-plot and Kernel Density Estimation contour-plot formats, our study examines patterns of differences in the cellular environment associated with the odds ratio of spatial patterns between active CD and control groups. This analysis is based on data collected at the two research institutes. The findings reveal heterogeneous nearest-neighbor patterns, signifying distinct tendencies of cell clustering, with a particular focus on the rectum region. These variations underscore the impact of data heterogeneity on cell spatial arrangements in CD patients. Moreover, the spatial distribution disparities between the two research sites highlight the significance of collaborative efforts among healthcare organizations. All research analysis pipeline tools are available at https://github.com/MASILab/cellNN., Comment: Submitted to SPIE Medical Imaging. San Diego, CA. February 2024

The generation of plausible crystal structures is often the first step in predicting the structure and properties of a material from its chemical composition. Quickly generating and predicting inorganic crystal structures is important for the discovery of new materials, which can target applications such as energy or electronic devices. However, most current methods for crystal structure prediction are computationally expensive, slowing the pace of innovation. Seeding structure prediction algorithms with quality generated candidates can overcome a major bottleneck. Here, we introduce CrystaLLM, a methodology for the versatile generation of crystal structures, based on the autoregressive large language modeling (LLM) of the Crystallographic Information File (CIF) format. Trained on millions of CIF files, CrystaLLM focuses on modeling crystal structures through text. CrystaLLM can produce plausible crystal structures for a wide range of inorganic compounds unseen in training, as demonstrated by ab initio simulations. The integration with predictors of formation energy permits the use of a Monte Carlo Tree Search algorithm to improve the generation of meaningful structures. Our approach challenges conventional representations of crystals, and demonstrates the potential of LLMs for learning effective 'world models' of crystal chemistry, which will lead to accelerated discovery and innovation in materials science., Comment: Added new results and supplementary information

The traditional display of elements in the periodic table is convenient for the study of chemistry and physics. However, the atomic number alone is insufficient for training statistical machine learning models to describe and extract composition-structure-property relationships. Here, we assess the similarity and correlations contained within high-dimensional local and distributed representations of the chemical elements, as implemented in an open-source Python package ElementEmbeddings. These include element vectors of up to 200 dimensions derived from known physical properties, crystal structure analysis, natural language processing, and deep learning models. A range of distance measures are compared and a clustering of elements into familiar groups is found using dimensionality reduction techniques. The cosine similarity is used to assess the utility of these metrics for crystal structure prediction, showing that they can outperform the traditional radius ratio rules for the structural classification of AB binary solids., Comment: 7 pages, 8 figures

Many anomaly detection approaches, especially deep learning methods, have been recently developed to identify abnormal image morphology by only employing normal images during training. Unfortunately, many prior anomaly detection methods were optimized for a specific "known" abnormality (e.g., brain tumor, bone fraction, cell types). Moreover, even though only the normal images were used in the training process, the abnormal images were often employed during the validation process (e.g., epoch selection, hyper-parameter tuning), which might leak the supposed ``unknown" abnormality unintentionally. In this study, we investigated these two essential aspects regarding universal anomaly detection in medical images by (1) comparing various anomaly detection methods across four medical datasets, (2) investigating the inevitable but often neglected issues on how to unbiasedly select the optimal anomaly detection model during the validation phase using only normal images, and (3) proposing a simple decision-level ensemble method to leverage the advantage of different kinds of anomaly detection without knowing the abnormality. The results of our experiments indicate that none of the evaluated methods consistently achieved the best performance across all datasets. Our proposed method enhanced the robustness of performance in general (average AUC 0.956).

The segment anything model (SAM) was released as a foundation model for image segmentation. The promptable segmentation model was trained by over 1 billion masks on 11M licensed and privacy-respecting images. The model supports zero-shot image segmentation with various segmentation prompts (e.g., points, boxes, masks). It makes the SAM attractive for medical image analysis, especially for digital pathology where the training data are rare. In this study, we evaluate the zero-shot segmentation performance of SAM model on representative segmentation tasks on whole slide imaging (WSI), including (1) tumor segmentation, (2) non-tumor tissue segmentation, (3) cell nuclei segmentation. Core Results: The results suggest that the zero-shot SAM model achieves remarkable segmentation performance for large connected objects. However, it does not consistently achieve satisfying performance for dense instance object segmentation, even with 20 prompts (clicks/boxes) on each image. We also summarized the identified limitations for digital pathology: (1) image resolution, (2) multiple scales, (3) prompt selection, and (4) model fine-tuning. In the future, the few-shot fine-tuning with images from downstream pathological segmentation tasks might help the model to achieve better performance in dense object segmentation.

Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20x magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is three-fold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

Thermoelectric materials can be used to construct devices which recycle waste heat into electricity. However, the best known thermoelectrics are based on rare, expensive or even toxic elements, which limits their widespread adoption. To enable deployment on global scales, new classes of effective thermoelectrics are thus required. $\textit{Ab initio}$ models of transport properties can help in the design of new thermoelectrics, but they are still too computationally expensive to be solely relied upon for high-throughput screening in the vast chemical space of all possible candidates. Here, we use models constructed with modern machine learning techniques to scan very large areas of inorganic materials space for novel thermoelectrics, using composition as an input. We employ an attention-based deep learning model, trained on data derived from $\textit{ab initio}$ calculations, to predict a material's Seebeck coefficient, electrical conductivity, and power factor over a range of temperatures and $\textit{n}$- or $\textit{p}$-type doping levels, with surprisingly good performance given the simplicity of the input, and with significantly lower computational cost. The results of applying the model to a space of known and hypothetical binary and ternary selenides reveal several materials that may represent promising thermoelectrics. Our study establishes a protocol for composition-based prediction of thermoelectric behaviour that can be easily enhanced as more accurate theoretical or experimental databases become available.

Multi-instance learning (MIL) is widely used in the computer-aided interpretation of pathological Whole Slide Images (WSIs) to solve the lack of pixel-wise or patch-wise annotations. Often, this approach directly applies "natural image driven" MIL algorithms which overlook the multi-scale (i.e. pyramidal) nature of WSIs. Off-the-shelf MIL algorithms are typically deployed on a single-scale of WSIs (e.g., 20x magnification), while human pathologists usually aggregate the global and local patterns in a multi-scale manner (e.g., by zooming in and out between different magnifications). In this study, we propose a novel cross-scale attention mechanism to explicitly aggregate inter-scale interactions into a single MIL network for Crohn's Disease (CD), which is a form of inflammatory bowel disease. The contribution of this paper is two-fold: (1) a cross-scale attention mechanism is proposed to aggregate features from different resolutions with multi-scale interaction; and (2) differential multi-scale attention visualizations are generated to localize explainable lesion patterns. By training ~250,000 H&E-stained Ascending Colon (AC) patches from 20 CD patient and 30 healthy control samples at different scales, our approach achieved a superior Area under the Curve (AUC) score of 0.8924 compared with baseline models. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

Advancements in fast electron detectors have enabled the statistically significant sampling of crystal structures on the nanometre scale by means of Scanning Electron Nanobeam Diffraction (SEND). Characterisation of structural similarity across this length scale is key to bridging the gap between local atomic structure (using atomic resolution techniques such as High Resolution Scanning Transmission Electron Microscopy (HR-STEM)) and the macro-scale (using bulk techniques such as powder X-ray and neutron diffraction). The use of SEND technique allows for structural investigation of a broad range of samples, due to the techniques ability to operate with low electron dosage and its tolerance for sample thickness, relative to HR-STEM. This, coupled with the capacity for data collection over a wide areas and the automation of this collection, allows for statistically representative sampling of the microstructure. Also due to these factors, SEND generates large datasets and as a result automated/ semi-automated data processing workflows are required to aid in maximal extraction of useful information. As such, this paper outlines a versatile, data-driven approach for producing domain maps, as well as a statistical approach for assessing their applicability. The production of such domain maps for a dataset can help highlight nuance in the microstructure, as well as improve the manageability of that dataset for further investigation. The workflow outlined utilises a Variational AutoEncoder to identify and learn the sources of variance in the diffraction signal and this, in combination with clustering techniques, is used to produce domain maps for a set of varied example cases. This approach: is agnostic to domain crystallinity; requires no prior knowledge of crystal structure; and does not require the, potentially prohibitive, simulation of a library of appropriate diffraction patterns.

Ab initio prediction of the variation of properties in the configurational space of solid solutions is computationally very demanding. We present an approach to accelerate these predictions via a combination of density functional theory and machine learning, using the cubic spinel nitride GeSn$_2$N$_4$ as a case study, exploring how formation energy and electronic bandgap are affected by configurational variations. Furthermore, we demonstrate the utility of applying explainable machine learning to understand the crystal chemistry origins of the trends that we observe. Different configuration descriptors (Coulomb matrix eigenspectrum, many-body tensor representation, and cluster correlation function vectors) are combined with different models (linear regression, gradient-boosted decision tree, and multi-layer perceptron) to extrapolate the calculation of ab initio properties from a small set of configurations to the full space with thousands of configurations. We discuss the performance of different descriptors and models. SHAP (SHapley Additive exPlanations) analysis of the machine learning models highlights how values of formation energy are dominated by variations in local crystal structure (single polyhedral environments), while values of electronic bandgap are dominated by variations in more extended structural motifs. Finally, we demonstrate the usefulness of this approach by constructing structure-property maps, identifying important configurations of GeSn$_2$N$_4$ with extremal properties, as well as by calculating accurate equilibrium properties using configurational averaging.

The rapid development of diagnostic technologies in healthcare is leading to higher requirements for physicians to handle and integrate the heterogeneous, yet complementary data that are produced during routine practice. For instance, the personalized diagnosis and treatment planning for a single cancer patient relies on the various images (e.g., radiological, pathological, and camera images) and non-image data (e.g., clinical data and genomic data). However, such decision-making procedures can be subjective, qualitative, and have large inter-subject variabilities. With the recent advances in multi-modal deep learning technologies, an increasingly large number of efforts have been devoted to a key question: how do we extract and aggregate multi-modal information to ultimately provide more objective, quantitative computer-aided clinical decision making? This paper reviews the recent studies on dealing with such a question. Briefly, this review will include the (1) overview of current multi-modal learning workflows, (2) summarization of multi-modal fusion methods, (3) discussion of the performance, (4) applications in disease diagnosis and prognosis, and (5) challenges and future directions.

Several diffuse interstellar bands (DIBs) have profiles with resolved sub-peaks that resemble rotational bands of large molecules. Analysis of these profiles can constrain the sizes and geometries of the DIB carriers, especially if the profiles exhibit clear variations along lines of sight probing different physical conditions. Using the extensive data set from the EDIBLES survey we searched for systematic variations in the peak-to-peak separation of these sub-peaks for the $\lambda\lambda$5797, 6379, and 6614 DIBs in lines of sight with a single dominant interstellar cloud. We used the spectra of twelve single-cloud sight lines to measure the peak-to-peak separation in the band profile substructures for these DIBs. We adopted the rotational contour formalism to infer the rotational constant for each DIB carrier and the rotational excitation temperature in the sight lines. We compared these to rotational constants for linear and spherical molecules to estimate the DIB carrier sizes. All three DIBs have peak separations that vary systematically between lines of sight, indicating correlated changes in the rotational excitation temperatures. We derived $B_{6614}$=$(22.2\pm8.9)\times 10^{-3}$ cm$^{-1}$, consistent with previous estimates. Assuming a similar rotational temperature for the $\lambda$6614 DIB carrier and assuming a linear carrier, we found B$_{5797}^{\rm linear}=(5.1\pm2.0)\times10^{-3}~{\rm cm}^{-1}$ and B$_{6379}^{\rm linear} =(2.3\pm0.9)\times10^{-3}~{\rm cm}^{-1}$. If the carriers of those DIBs however are spherical species, their rotational constants are half that value, $B_{5797}^{\rm spherical} = (2.6\pm1.0)\times10^{-3}~{\rm cm}^{-1}$ and $B_{6379}^{\rm spherical} = (1.1\pm0.4)\times10^{-3}~{\rm cm}^{-1}$. We estimate molecule sizes that range from 7--9 carbon atoms ($\lambda$6614 carrier, linear) to 77--114 carbon atoms ($\lambda$6379, spherical)., Comment: 21 pages, 56 figures. Accepted for publication in Astronomy & Astrophysics

Bismuth ferrite, BiFeO3, is a multiferroic solid that is attracting increasing attention as a potential photocatalytic material, because the ferroelectric polarisation enhances the separation of photogenerated carriers. With the motivation of finding routes to engineer the band gap and the band alignment, while conserving or enhancing the ferroelectric properties, we have investigated the thermodynamic, electronic and ferroelectric properties of BiCoxFe1 xO3 solid solutions, with 0 < x < 0.13, using density functional theory. We show that the band gap can be reduced from 2.9 eV to 2.1 eV by cobalt substitution, while simultaneously increasing the spontaneous polarisation, which is associated with a notably larger Born effective charge of Co compared to Fe cations. We discuss the interaction between Co impurities, which is strongly attractive and would drive the aggregation of Co, as evidenced by Monte Carlo simulations. Phase separation into a Co-rich phase is therefore predicted to be thermodynamically preferred, and the homogeneous solid solution can only exist in metastable form, protected by slow cation diffusion kinetics. Finally, we discuss the band alignment of pure and Co-substituted BiFeO3 with relevant redox potentials, in the context of its applicability in photocatalysis., Comment: Biblography expanded; typos corrected; improved discussion of photocatalytic applications

Mixed-anion mixed-cation perovskites with (FAPbI$_3$)$_{1-x}$(MAPbBr$_3$)$_x$ composition have allowed record efficiencies in photovoltaic solar cells, but their atomic-scale behaviour is not well understood yet, in part because their theoretical modelling requires consideration of complex and interrelated dynamic and disordering effects. We present here an ab initio molecular dynamics investigation of the structural, thermodynamic, and electronic properties of the (FAPbI$_3$)$_{0.875}$(MAPbBr$_3$)$_{0.125}$ perovskite. A special quasi-random structure is proposed to mimic the disorder of both the molecular cations and the halide anions, in a stoichiometry that is close to that of one of today's most efficient perovskite solar cells. We show that the rotation of the organic cations is more strongly hindered in the mixed structure in comparison with the pure compounds. Our analysis suggests that this mixed perovskite is thermodynamically stable against phase separation despite the endothermic mixing enthalpy, due to the large configurational entropy. The electronic properties are investigated by hybrid density functional calculations including spin-orbit coupling in carefully selected representative configurations extracted from the molecular dynamics. Our model, that is validated here against experimental information, provides a more sophisticated understanding of the interplay between dynamic and disordering effects in this important family of photovoltaic materials., Comment: 10 pages, 7 figures

Multiplex immunofluorescence (MxIF) is an emerging imaging technique that produces the high sensitivity and specificity of single-cell mapping. With a tenet of 'seeing is believing', MxIF enables iterative staining and imaging extensive antibodies, which provides comprehensive biomarkers to segment and group different cells on a single tissue section. However, considerable depletion of the scarce tissue is inevitable from extensive rounds of staining and bleaching ('missing tissue'). Moreover, the immunofluorescence (IF) imaging can globally fail for particular rounds ('missing stain''). In this work, we focus on the 'missing stain' issue. It would be appealing to develop digital image synthesis approaches to restore missing stain images without losing more tissue physically. Herein, we aim to develop image synthesis approaches for eleven MxIF structural molecular markers (i.e., epithelial and stromal) on real samples. We propose a novel multi-channel high-resolution image synthesis approach, called pixN2N-HD, to tackle possible missing stain scenarios via a high-resolution generative adversarial network (GAN). Our contribution is three-fold: (1) a single deep network framework is proposed to tackle missing stain in MxIF; (2) the proposed 'N-to-N' strategy reduces theoretical four years of computational time to 20 hours when covering all possible missing stains scenarios, with up to five missing stains (e.g., '(N-1)-to-1', '(N-2)-to-2'); and (3) this work is the first comprehensive experimental study of investigating cross-stain synthesis in MxIF. Our results elucidate a promising direction of advancing MxIF imaging with deep image synthesis., Comment: Accepted at the third MICCAI workshop on Computational Pathology (COMPAY 2021)