Evaluating the Use of Mid Infrared Spectroscopy in Predicting Soil Carbon, Nitrogen Contents, and Soil Texture in Varying South African Soil Samples

Abstract

There is a growing need for soil information to help address problems such as soil nutrient depletion and incorrect fertilizer inputs which affect yields and threaten food security and the environment. Low soil fertility is a significant issue in main grain-producing regions in South Africa. Routine soil analysis addresses these issues as farmers are provided with valuable information to enhance agricultural productivity and sustainability. However, lack of efficient and cost effective soil testing infrastructure is a limitation in South Africa. Soil spectroscopy has proven to be an efficient soil analysis method that complements traditional methods of soil analysis. It is more cost-effective than analytical laboratory procedures since measurements may be completed faster, and multiple soil parameters can be deduced from a single spectral measurement (Nocita et al., 2015). This research aimed at developing a spectral library of South African soils and to test the ability of mid-infrared (MIR; 4000 – 500cm-1) diffuse reflectance spectroscopy coupled with machine learning algorithms to predict total carbon, total nitrogen, and soil texture. Soil samples for this study were selected from the ARC-SCW archive using the Kennard-Stone algorithm to ensure diversity. Chemical analysis of total carbon (%) and nitrogen (%) was conducted through total combustion with a CHNS-O analyser, while physical properties (sand (%), silt (%), and clay (%) content) were determined through the hydrometer analysis technique. This data was then used with corresponding spectra to develop PLSR (partial least squares regression) and MBL (memory-based learner) predictive models. During model assessment, the criteria used to determine a good predictive model was that an R2 value had to exceed 0.7, RMSEp < 10% of the range of the independent validation dataset and RPIQ > 1.7. Predictive regression models were developed successfully, the results for clay, silt, and sand contents showed that soil texture can be predicted well with diffuse reflectance MIR spectroscopy. The highest R2 values were 0.80, 0.78, and 0.81 for clay, silt, and sand contents respectively. Based on these results, these models can be used in real world situations for accurate predictions of sand, silt, and clay content. The best sand and clay content models were developed with UVS- elected variables, selected from a raw spectra dataset. On the other hand, during model assessment, the results were poor for predicting TC (total carbon) and TN (total nitrogen). The highest R2 values were 0.51 and 0.61 for TC and TN respectively. This meant that the predictive models developed in this study for predicting TC and TN were unreliable and may not be applicable for predictions. This poor result was due to highly variable and limited data used during model training. Compared to results obtained with full-spectrum, Boruta, and CARS-selected variables datasets, the raw spectra UVS-selected variables datasets produced better results. Generally, MBL produced better results than PLSR. Pre-processing the spectra with either Savitzky-Golay first derivative filter (SG) or standard normal variate (SNV) was proved to be ineffective in improving model performance. Soil spectroscopy is a data hungry analysis method, therefore it is recommended to develop predictive models with many samples such that the number of samples is greater or equal to the number of predictor variables especially when the data is diverse.