Search

Materials discovery and development are critical for addressing global challenges. Yet, the exponential growth in materials science literature comprising vast amounts of textual data has created significant bottlenecks in knowledge extraction, synthesis, and scientific reasoning. Large Language Models (LLMs) offer unprecedented opportunities to accelerate materials research through automated analysis and prediction. Still, their effective deployment requires domain-specific adaptation for understanding and solving domain-relevant tasks. Here, we present LLaMat, a family of foundational models for materials science developed through continued pretraining of LLaMA models on an extensive corpus of materials literature and crystallographic data. Through systematic evaluation, we demonstrate that LLaMat excels in materials-specific NLP and structured information extraction while maintaining general linguistic capabilities. The specialized LLaMat-CIF variant demonstrates unprecedented capabilities in crystal structure generation, predicting stable crystals with high coverage across the periodic table. Intriguingly, despite LLaMA-3's superior performance in comparison to LLaMA-2, we observe that LLaMat-2 demonstrates unexpectedly enhanced domain-specific performance across diverse materials science tasks, including structured information extraction from text and tables, more particularly in crystal structure generation, a potential adaptation rigidity in overtrained LLMs. Altogether, the present work demonstrates the effectiveness of domain adaptation towards developing practically deployable LLM copilots for materials research. Beyond materials science, our findings reveal important considerations for domain adaptation of LLMs, such as model selection, training methodology, and domain-specific performance, which may influence the development of specialized scientific AI systems.

In recent years, the diagnosis of gastrointestinal (GI) diseases has advanced greatly with the advent of high-tech video capsule endoscopy (VCE) technology, which allows for non-invasive observation of the digestive system. The MisaHub Capsule Vision Challenge encourages the development of vendor-independent artificial intelligence models that can autonomously classify GI anomalies from VCE images. This paper presents CNN architecture designed specifically for multiclass classification of ten gut pathologies, including angioectasia, bleeding, erosion, erythema, foreign bodies, lymphangiectasia, polyps, ulcers, and worms as well as their normal state., Comment: 11 pages, 7 figuers

Across the technology industry, many companies have expressed their commitments to AI ethics and created dedicated roles responsible for translating high-level ethics principles into product. Yet it is unclear how effective this has been in leading to meaningful product changes. Through semi-structured interviews with 26 professionals working on AI ethics in industry, we uncover challenges and strategies of institutionalizing ethics work along with translation into product impact. We ultimately find that AI ethics professionals are highly agile and opportunistic, as they attempt to create standardized and reusable processes and tools in a corporate environment in which they have little traditional power. In negotiations with product teams, they face challenges rooted in their lack of authority and ownership over product, but can push forward ethics work by leveraging narratives of regulatory response and ethics as product quality assurance. However, this strategy leaves us with a minimum viable ethics, a narrowly scoped industry AI ethics that is limited in its capacity to address normative issues separate from compliance or product quality. Potential future regulation may help bridge this gap.

Advances in machine learning have led to the development of foundation models for atomistic materials chemistry, enabling quantum-accurate descriptions of interatomic forces across diverse compounds at reduced computational cost. Hitherto, these models have been benchmarked relying on descriptors based on atoms' interaction energies or harmonic vibrations; their accuracy and efficiency in predicting observable and technologically relevant heat-conduction properties remains unknown. Here, we introduce a framework that leverages foundation models and the Wigner formulation of heat transport to overcome the major bottlenecks of current methods for designing heat-management materials: high cost, limited transferability, or lack of physics awareness. We present the standards needed to achieve first-principles accuracy in conductivity predictions through model's fine-tuning, discussing benchmark metrics and precision/cost trade-offs. We apply our framework to a database of solids with diverse compositions and structures, demonstrating its potential to discover materials for next-gen technologies ranging from thermal insulation to neuromorphic computing., Comment: 15 pages, 10 figures

Metal halide perovskite-based semi-conducting hetero-structures have emerged as promising electronics for solar cells, light-emitting diodes, detectors, and photo-catalysts. Perovskites' efficiency, electronic properties and their long-term stability directly depend on their morphology [1-24]. Therefore, to manufacture stable and higher efficiency perovskite solar cells and electronics, it is now crucial to understand their micro-structure evolution. In this study, we perform molecular dynamics simulations to investigate the formation of cesium lead bromide perovskite on interfaces. Our simulations reveal that perovskite crystallizes in a heteroepitaxial manner on widely employed oxide interfaces. This could introduce the formation of dislocations, voids and defects in the buried interface, and grain boundaries in the bulk crystal. From simulations, we find that lattice-matched interfaces could enable epitaxial ordered growth of perovskites and may prevent defect formation in the buried interface.

The efficiency and stability of halide perovskite-based solar cells and light-emitting diodes directly depend on the intricate dynamics of solid-solid crystallization[1-23]. In this study, we employ a multi-scale approach using random phase approximation, density functional theory, machine learning potentials, reduced charge force fields, and both enhanced sampling biased and brute-force unbiased molecular dynamics simulations to understand the solid-solid phase transitions in cesium lead iodide perovskite. Our simulations uncover that the direct phase transition from the non-perovskite to the perovskite involves the formation of stacked-faulted and low-dimensional intermediate structures. Through extensive large-scale all-atom simulations encompassing up to 650,000 atoms, we observe that solid-solid crystallization may require the formation of a sufficiently large critical nucleus to grow into a faceted perovskite crystal. Based on simulations, we determine that utilizing (100)-faceted seeded crystallization could offer a promising path for manufacturing high-performance and stable perovskite solar cells.

The rise of Machine Learning as a Service (MLaaS) has led to the widespread deployment of machine learning models trained on diverse datasets. These models are employed for predictive services through APIs, raising concerns about the security and confidentiality of the models due to emerging vulnerabilities in prediction APIs. Of particular concern are model cloning attacks, where individuals with limited data and no knowledge of the training dataset manage to replicate a victim model's functionality through black-box query access. This commonly entails generating adversarial queries to query the victim model, thereby creating a labeled dataset. This paper proposes "MisGUIDE", a two-step defense framework for Deep Learning models that disrupts the adversarial sample generation process by providing a probabilistic response when the query is deemed OOD. The first step employs a Vision Transformer-based framework to identify OOD queries, while the second step perturbs the response for such queries, introducing a probabilistic loss function to MisGUIDE the attackers. The aim of the proposed defense method is to reduce the accuracy of the cloned model while maintaining accuracy on authentic queries. Extensive experiments conducted on two benchmark datasets demonstrate that the proposed framework significantly enhances the resistance against state-of-the-art data-free model extraction in black-box settings., Comment: Under Review

Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. Fairness aims to reduce model bias for social/demographic sub-groups. Privacy via differential privacy (DP) mechanisms, on the other hand, limits the impact of any individual's training data on the resulting model. The trade-offs between privacy and fairness goals of trustworthy ML pose a challenge to those wishing to address both. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models (LLMs), producing models more biased than ones fine-tuned without DP. We find the cause of the amplification to be a disparity in convergence of gradients across sub-groups. Through the case of binary gender bias, we demonstrate that Counterfactual Data Augmentation (CDA), a known method for addressing bias, also mitigates bias amplification by DP. As a consequence, DP and CDA together can be used to fine-tune models while maintaining both fairness and privacy.