Search

Soil spectroscopy is a widely used method for estimating soil properties that are important to environmental and agricultural monitoring. However, a bottleneck to its more widespread adoption is the need for establishing large reference datasets for training machine learning (ML) models, which are called soil spectral libraries (SSLs). Similarly, the prediction capacity of new samples is also subject to the number and diversity of soil types and conditions represented in the SSLs. To help bridge this gap and enable hundreds of stakeholders to collect more affordable soil data by leveraging a centralized open resource, the Soil Spectroscopy for Global Good initiative has created the Open Soil Spectral Library (OSSL). In this paper, we describe the procedures for collecting and harmonizing several SSLs that are incorporated into the OSSL, followed by exploratory analysis and predictive modeling. The results of 10-fold cross-validation with refitting show that, in general, mid-infrared (MIR)-based models are significantly more accurate than visible and near-infrared (VisNIR) or near-infrared (NIR) models. From independent model evaluation, we found that Cubist comes out as the best-performing ML algorithm for the calibration and delivery of reliable outputs (prediction uncertainty and representation flag). Although many soil properties are well predicted, total sulfur, extractable sodium, and electrical conductivity performed poorly in all spectral regions, with some other extractable nutrients and physical soil properties also performing poorly in one or two spectral regions (VisNIR or NIR). Hence, the use of predictive models based solely on spectral variations has limitations. This study also presents and discusses several other open resources that were developed from the OSSL, aspects of opening data, current limitations, and future development. With this genuinely open science project, we hope that OSSL becomes a driver of the soil spectroscopy community to accelerate the pace of scientific discovery and innovation.

Diffuse reflectance spectroscopy has been extensively employed to deliver timely and cost-effective predictions of a number of soil properties. However, although several soil spectral laboratories have been established worldwide, the distinct characteristics of instruments and operations still hamper further integration and interoperability across mid-infrared (MIR) soil spectral libraries. In this study, we conducted a large-scale ring trial experiment to understand the lab-to-lab variability of multiple MIR instruments. By developing a systematic evaluation of different mathematical treatments with modeling algorithms, including regular preprocessing and spectral standardization, we quantified and evaluated instruments' dissimilarity and how this impacts internal and shared model performance. We found that all instruments delivered good predictions when calibrated internally using the same instruments' characteristics and standard operating procedures by solely relying on regular spectral preprocessing that accounts for light scattering and multiplicative/additive effects, e.g., using standard normal variate (SNV). When performing model transfer from a large public library (the USDA NSSC-KSSL MIR library) to secondary instruments, good performance was also achieved by regular preprocessing (e.g., SNV) if both instruments shared the same manufacturer. However, significant differences between the KSSL MIR library and contrasting ring trial instruments responses were evident and confirmed by a semi-unsupervised spectral clustering. For heavily contrasting setups, spectral standardization was necessary before transferring prediction models. Non-linear model types like Cubist and memory-based learning delivered more precise estimates because they seemed to be less sensitive to spectral variations than global partial least square regression. In summary, the results from this study can assist new laboratories in building spectroscopy capacity utilizing existing MIR spectral libraries and support the recent global efforts to make soil spectroscopy universally accessible with centralized or shared operating procedures.