Our main goal is to develop a UAS-based artificial intelligence (AI) toolkit to detect soybean cyst nematode (SCN) related stresses of soybean. We have been developing a UAS-based remote sensing predictive modeling tool to determine SCN population density and its impact on soybean health.

Proposal description: The soybean cyst nematode (SCN) is the most important soybean pathogen in the United States, with 3 billion dollars in annual losses, exceeding the total losses of the next seven soybean yield-robbing pathogens combined. While there are management options available for this pathogen, the ever-adapting nature of SCN populations in production fields puts extensive pressure on available management tools, such as using soybean varieties resistant to SCN. Thus, the proper management of SCN requires adopting an integrated approach using a multitude of control measures for efficient and sustainable management of this pest. A critical need for sustainable and effective management of soilborne pathogens is to have a precise assessment of the inoculum potential in the soil. However, the methods that are currently used for nematode quantification are crude with many limitations, such as the lack of accuracy and efficiency at the field or farm level. Also, crop damage thresholds are difficult to establish for identifying SCN parasitism due to confounding factors such as heterogeneous soil chemical and physical properties, the influence of other pathogens, and the effects of environmental factors.

Recent advances in multispectral imaging and spatial modeling make remote sensing a promising and attractive alternative to traditional crop disease detection and monitoring methods owing to its flexibility, low costs, and effectiveness. Remote sensing technologies have been widely used to detect crop diseases (Calderón et al., 2013; Di Gennaro et al., 2016; Hatton et al. 2017), when the leaf canopy exhibits unique spectral signatures of crop stresses in response to different diseases and infection levels. These technologies have been applied to a wide range of sensors and platforms, including handheld spectroradiometers, Unmanned Aerial Systems (UAS), aircraft, and satellites, which allow the diseases to be detected in a wide range of scales. However, there is still a general paucity of research on the combined use of such tools to monitor SCN and pathogens of soybean.

Spatial technologies such as geographic information systems (GIS) and remote sensing are crucial to optimize and streamline the decision-making process in crop production and are being increasingly integrated into advanced pathogen management systems. For example, Nutter et al. (2002) evaluated GIS and remote sensing technologies that can be used to detect SCN-related damage and identify the extent and distribution of the pathogen across fields. Ground-based data was collected using a multispectral radiometer. Aerial images were collected by an RGB and infrared camera attached to a plane. Aerial and ground-based data were collected every ten to fourteen days during the growing season. Also, the data was supplemented by Landsat satellite imagery. The results show that the data collected by integrating three imaging platforms explained more variation in yield than the traditional testing methods for SCN populations in the targeted soybean-production fields. A more recent study targeting the sugar beet cyst nematode (Heterodera schachtii), revealed that multispectral imagery collected with UAVs could differentiate the areas of high nematode infestation across a field (Joalland et al., 2018). The study also found that the canopy temperatures of susceptible sugar beet cultivars tend to be higher compared to those recorded with tolerant cultivars. Differences in cultivars were also observed for biomass, chlorophyll content, and general stress. These differences were identified using Normalized Difference Vegetation Index (NDVI), chlorophyll content (CHLG), and healthy-index (HI) indices developed from UAV images and field spectroradiometer measurements. Note that the sugar beet cyst nematode has a similar biology and parasitism strategy to the SCN. We propose to expand on these studies by developing and optimizing plant health monitoring algorithms and associated predictive modeling tools to assess SCN levels and their effect on plant health. These tools would be made available to the public to allow broad adoption for the benefit of Illinois soybean producers.

Proposed methods/tactics: We will develop field and greenhouse trials and drone image data analytics. Figure 1 shows the general flowchart for developing the SCN detection toolkit. We will focus on (1) establishing greenhouse trials to better understand the interplay of SCN infestation and soybean phenology, and (2) improving the image analyses with both supervised and unsupervised classification methods.

For this project, field trials will be established at two locations in southern Illinois. These locations will serve as data generators for the predictive models.

First experimental setup – The experiment will be conducted at two locations. Soybean will be planted in SCN infested fields. These fields will represent traditional fields in Illinois with varying levels of SCN.

Ground Truth Data Collection – Fields with a range of low (0 - 500 eggs per 100 cc soil) to high (<15,000 eggs per 100 cc of soil) SCN egg counts will be selected. The field sizes range from 70 to 150 acres. The fields will be mapped into 1/3-acre grids, where soil samples will be collected. Soil samples will be collected at planting, midseason, and harvest to determine the population densities of SCN and conduct a soil fertility analysis. By determining the SCN population densities, we will be able to validate the areas in the field with high populations and relate those populations to the imagery. At the end of the season, soybeans will be harvested, and the yield data will be collected.

Collection of hyperspectral and multispectral data – First, a field spectroradiometer (ASD Hand-Held 2: VNIR Spectroradiometer, Malvern Panalytical, United Kingdom) will be used to collect close-range hyperspectral data from controlled greenhouse trials. The data will be used to examine the spectral bands and indices that are most sensitive to SCN parasitism indicated by egg counts over different phenological stages. At the production field level, multispectral UAV images will be collected using an X5S RGB camera, and an Altum multispectral camera mounted to a UAV. In the strips where there are different treatments, 20 spectral readings will be collected using a spectroradiometer, and the data will then be averaged to represent each strip. Imagery and spectra will be collected 30, 45, 60, and 75 days after planting (DAP). All flight missions will be completed under the local drone-related legislative and campus regulations. Each flight mission will cover the entirety of the trial area, and the time of flight will be dependent on the size of the area. Ground control points (GCPs) to ensure that the imagery will be correctly georeferenced. The GCPs will be placed throughout the study area, and the GPS coordinates will be collected. RTK correction will be used to give sub-centimeter accuracy.

Development of AI algorithms – As shown in previous research, parasitism by the nematodes (e.g., Heterodera schactii) presents unique spectral and textual signatures on the phenological representation of fields. We propose to develop agricultural artificial intelligence (Agro-AI) models to identify the areas with high nematode densities for better management of SCN. The Agro-AI models are especially suitable for identifying these fuzzy-look features that are difficult to be examined with regular field or image inspection. This study will collect UAV image samples to develop both statistical models and deep learning models.

First, hyperspectral data collected from SCN affected areas and healthy areas using a field spectroradiometer will be compared to identify the most sensitive bands and spectral indices that can indicate the plant stress associated with SCN parasitism. Although most of the existing research has focused on chlorophyll-related indicators (e.g., NDVI and leaf health), less knowledge has been gained in terms of how to detect the early stress before chlorophyll damage is present (e.g., evapotranspiration stress and early nitrogen deficit). This work is expected to advance the knowledge of how crop stress may progress in response to SCN parasitism at different phenological stages (e.g., nth trifoliolate and flowering). Then, both linear and non-linear multivariate regression models will be employed to understand if those sensitive spectra indicators can explain the variation in SCN parasitism. We will use the coefficients of determination (R2) to evaluate if the spectral indicators can sufficiently represent the variability in SCN parasitism.

Second, we will experiment with both an unsupervised machine learning and a supervised deep learning approach to identify SCN-infested areas (Figure 2). For the unsupervised machine learning, we will use the multi-band segmentation algorithm to divide a time series of drone images into spatial objects, which were then clustered based on the vegetation indices most sensitive to SCN infestation. The spatial object clusters will be associated with SCN infestation based on a comparison with SCN egg counts. For the supervised deep learning approach, the manually labeled UAV images containing both nematode-affected and healthy areas will be randomly divided 2/3 and 1/3 as the training and testing samples, respectively. Data augmentation techniques such as rotation, reflection, and Gaussian smoothing kernel filtering will be applied to the original training samples to increase the size of the training dataset and prevent the deep neural network from overfitting. The training samples will then be used to feed a deep learning convolutional neural network (CNN) algorithm. The Faster R-CNN detector can be trained by the layers with ten epochs. The likelihood of each feature reflecting nematodes infection is predicted by the detector. The non-maximum suppression (NMS) will be used to suppress redundant detections surrounding the same feature. Every training image is sliced into 9 equal size small images to enhance training efficiency. Final training samples used to train the model are enlarged to 5 times the original data. A valid detection is defined by an overlap ratio of the predicted bounding box and ground truth box beyond 0.4. Accuracy on every single image and corresponding detection method will be recorded as the response and method factor, respectively. As the response is bounded from 0 to 1, a beta Generalized Linear Model (GLM) is fitted to interpret the treatment effect of the detection methods and environmental types on detection accuracy.

Proposed outcomes: This project will address two of the most challenging aspects of SCN management: how much SCN is in the producer’s field and the limitations of trying to interpret what the SCN populations are doing (increasing or decreasing) from just a few soil samples that are collected one to several times each year. Addressing these questions with the proposed tools will allow Illinois producers to better manage SCN populations in their fields. This project has the potential to transform the practices of field assessment for SCN infestation and density. In the last 50 years, there have been very few advancements made on how nematodes are monitored. In addition, we are still dealing with the limitations with just a few samples being collected per field, and yet the SCN population density is being used to interpolate the potential density across fields. By collecting drone images and ground truth SCN data, this project will develop a set of algorithms and tools to assess the impact of SCN and potentially other pathogens on plant health.

Communications plan: We will disseminate our research findings through conference presentations, extension publications, journal articles, and media releases to the precision farming industry and crop consultants. We will work with the Soybean Research Information Initiative (SRII) and the Crop Protection Network to publish updates and release information on the availability of the toolkit to a broader farming community for potential adoption. In addition, we will use social media like Facebook to share project information and progress. The proposed toolkit will be freely released via GitHub, a web-based computer algorithm sharing system, so that the feedback can make improvements from precision agricultural research communities.

Updated September 30, 2022:
Appendix – Project Update

We established multiple experiments and collected over 20 gigabyte drone images that cover the growing and harvest seasons of soybean production fields. We developed methods to integrate established vegetation indices with plant-pathogen population densities in both controlled green house and outdoor environments. These methods are being refined for future AI integration. We also determined critical stages for imagery acquisition related to increased stress from parasitism by SCN populations. Initial results were reported in the Systems and Technologies for Remote Sensing Applications Through Unmanned Aerial Systems (STRATUS) annual meeting and the American Association of Geographers (AAG) annual meeting. STRATUS is a prestigious meeting for interdisciplinary topics ranging from the latest challenges in UAV technology, sensor design, information gathering and processing, and modeling to support decision-making. AAG is among the world’s largest geography meeting that features over 20 UAV and big data related sessions.

A Matrice 210 UAV was used to collect imagery with an X5S RGB camera and an Altum Multispectral/Thermal camera. In addition to the yield data, 224 soil samples were collected from the field that was divided into 1/3-acre grids. Drone images captured with Altum Multispectral/Thermal camera were first radiometrically calibrated and georeferenced in Pix4Dmapper (Pix4D, S.A., Lausanne, Switzerland). The generated ortho-mosaic images were then transformed to spectral reflectance values in Pix4Dmapper. Reflectance maps for each multispectral band were used to calculate UAV- based Vegetation Indices (VI) using ArcGIS v10.8 software (ESRI Inc., Redlands, CA, USA). A total of 16 common VIs was selected (Table A1) and examined for their relationships with SCN infestation (SCN egg counts) as well as yields (in bushels). To compare these vegetation indices with ground-truth data in the sampling areas, we average those VIs based on each field plot using the zonal statistics tool in the ArcGIS software. Then, the Pearson correlations between SCN and yield variables and those developed vegetation indices collected on July 20, 2021 are shown in Figure A1. Most of VIs are highly correlated with each other. Enhanced Vegetation Index (EVI) and Difference Vegetation Index (DVI) appear to have the highest correlations with both SCN egg counts and crop yields. Note that the better performance of EVI and DVI than the popular NDVI may be due to the fact that NDVI is subject to saturation at high canopy levels.

Then, we used EVI, DVI, and NDVI to develop regression models for predicting SCN egg counts and crop yields. Figure A2 shows that EVI exhibits itself as a strong predictor of SCN egg counts (R2=0.81) and crop yields (R2=0.84). The more SCN egg counts, the less vegetation canopy. The promising outcome suggests that SCN infestation and related yield reduction may be virtually represented at the canopy level using drone-based vegetation indices.

In order to understand spectral responses from soybeans to SCN infestation, we collected hyperspectral radiometer data under varying levels of SCN infestations (1,000, 5,000, 10,000 eggs treatment) in a controlled greenhouse environment. Figure A3 shows that Red and Near Infrared bands are most sensitive while Red Edge is the least sensitive band. This result is consistent with the correlation coefficients presented in Figure A1, in which those VIs that involve Red Edge have lower coefficients. Thus, future monitoring efforts should be based on these VIs that involve Red and Near Infrared.

In summary, upon the successful implementation of the project, we found that drone-based remote sensing is a cost-effective method for monitoring soybean damage caused by SCN. In the last year’s effort, we were able to (1) identify Red and Near Infrared as the most sensitive and Red Edge the least sensitive bands to SCN infestation, and (2) understand the sensitivity of different vegetation indices to SCN infestation and crop yields. However, our multi-image data analysis results indicate that spectral responses to SCN infestation vary over different time periods (or phenological stages). It is imperative to understand how phenology may be in play with the observable symptoms associated with different levels of SCN infestation. Therefore, we propose to further our research in a third-year project that will focus on (1) extending the greenhouse trials to better understand the interplay of SCN infestation and soybean phenology, and (2) expanding the image analyses with both supervised and unsupervised classification methods.

We propose to expand on these studies by developing and optimizing plant health monitoring algorithms and associated predictive modeling tools to assess SCN levels and their effect on plant health. These tools would be made available to the public to allow broad adoption for the benefit of Illinois soybean producers.