timothylimyl / spaceapps_hackathon

Explanation and validation of hackathon idea.

NASA SpaceApps Hackathon (Team AgriCarer | Better Together)

Introduction

Our team (AgriCarer) is seeking to tackle the knowledge inequality problem currently faced by many small and family-owned farms. From our research and personal conversations with a local farmer in Malaysia, we found that most farmers in developing countries are smallholder farmers that generally lack access to the best agriculture practices, information, and technology. Smallholders farmers do not have the necessary funding to procure state-of-the-art technologies and seek expert advice.

Thus, local farmers are very vulnerable to the occurrences of diseases and pests in ruining their entire crop and reducing their crop productivity. In some cases, the whole farm may have no yield as none of the proper actions were taken due to the lack of knowledge. This can lead to problems such as food insecurity, environmental issues, and harm to the wellbeing of the farmers.

This issue of knowledge inequality needs to be addressed immediately to level the playing field of food production and most importantly ensuring food security.

Idea Proposal

We are proposing a user-friendly smartphone app (AgriCare) to support farmers with state-of-the-art technologies and expert research advice to solve the problem of the knowledge gap and the problem of yield losses in crop plants. Most local farmers that we talked to do not have the necessary extra funds to invest in technology and have an aversion to new technology.. Therefore, a smartphone app was suggested by the team as it will lower the cost of entry for farmers as most farmers already own a smartphone, making the technology much easier to adopt for smallholder farmers that need it the most.

The smartphone app proposed will have features such as remote sensing technology, plant disease diagnosis, and timely reminders to support farmers in taking care of their farms. 

Remote Sensing Technology (Validation of concept done on a Malaysia Farm)

The remote sensing technology will be done by collecting free open-source satellite data from space agencies around the world such as NASA, CSA, CNES, JAXA, and ESA. These data will be integrated into our smartphone app to allow smallholder farmers access to such the technology to improve the crop production of their farms as shown below:

The remote sensing functionality utilizes the Multispectral Scanner System (MSS) sensors equipped on satellites. MSS sensors have the ability to capture data at an exceptional spatial resolution as well as determine reflectance in near-infrared. Studies have shown that the electromagnetic data collected by the MSS sensors can be utilised for crop monitoring using vegetation indices (computed by using a combination of electromagnetic data captured).

Vegetation indices can be defined as the indicator that quantifies vegetation biomass and plant vigor for each pixel in a remote sensing image which allows reliable spatial and temporal inter- comparisons of terrestrial photosynthetic activity and canopy structural variations. A few examples of vegetation indices that can be calculated from the data obtained from MSS sensor on satellites are NDVI, MCARI, TCARI and PVI, all of which are capable of relaying some type of useful information about the farm while accurately indicating the affected area. (Reference: 1 2)

Our validation step involves obtaining open data multispectral imagery captured by satellite MSS sensors and processing these images to derive the NDVI index for an area of farmland. The image data extraction and processing are aided by functionalities and software readily available by the European Space Agency (ESA). We source our data directly from the Copernicus Open Access Hub which provides free and open access to imagery captured by the Sentinel satellites launched during the Copernicus program from 2014 to 2015. The Open Access hub allows us to specify to exact regions of interest, satellite sensing periods, and selection of Sentinel imagery required. We then utilise ESA’s Sentinel Application Platform (SNAP) program, which allows the combination of several multispectral data to display satellite images.

The common types of images are RGB (Red, Green, Blue) and Infrared as shown below (we extracted images of a farm in Bukit Rotan, Kuala Selangor, Malaysia) to show a proof of concept:

RGB:

Infrared:

As a proof of concept, our team derived one common vegetation index, Normalized Difference Vegetation Index (NDVI) using the software SNAP to customise the combination of light bands to derive useful mapping presentation. In this case, we derive the NDVI mapping by considering the formula above, with NIR and Red represented by bands B8 and B4.

Example of NDVI images that was preprocessed:

NDVI, which measures the amount of green vegetation in an area. NDVI is based on the principle that actively growing green plants strongly absorb radiation in the visible region of the spectrum (the “PAR,” or “photosynthetically active radiation”), while strongly reflecting radiation in the near-infrared (NIR) region. A healthy plant with lots of chlorophyll. NDVI is a standardized way to measure healthy vegetation. When you have high NDVI values, you have healthier vegetation. When you have low NDVI, you have less or no vegetation and good cell structure will actively absorb red light and reflect near-infrared. Reference

In our instance above, we defined our spectrum of low NDVI (represented by yellow) and high NDVI (green). Greener regions, i.e towards the end of the spectrum suggests healthier vegetation and yellowish regions suggest the contrary.

Plant Disease Diagnosis (Validation on open-source dataset)

This repository consists of a simple prototype/proof of an AI system that is able to classify different plant diseases. We used the FastAI API to train on our dataset. The dataset was obtained from PlantVillage. PlantVillage is an open-source dataset consisting of over 50,000 expertly curated images of healthy and infected leaves of crop plants.

In this case of our proof, we decided to train on a subset of the PlantVillage dataset and took only three categories namely Tomato Healthy, Tomato Early Blight, and Tomato Late Blight. Data used can be found HERE, there is a total of 4500 raw images in it. Utilising a ResNet50 with data augmentation, we were already able to achieve  98.89% accuracy on the validation set consisting of 900 images (20% of 4500 raw images was used for validation) as seen in the confusion matrix below:

However, it is worth noting that the model trained on PlantVillage data does not transfer well to the real-domain which is of expectation as the AI system is trained using supervised learning. Supervised learning depends heavily on the quantity, quality, and diversity of the datasets to perform well. The PlantVillage database is taken under controlled lab conditions with a single leaf in the photo as shown below.

 

Thus, when a plant is taken as a whole, the AI will most probably fail to classify correctly as the AI is overfitting the images taken under controlled lab conditions.  An example of misclassification, a healthy tomato plant is classified as late blight:

 

Our team is well aware of the problem of misclassification which is why we believe that this making an app that is widely available and accessible is a great idea for mass data collection in order to train a more robust AI model. We believe that by pre-processing images taken by the farmer and allowing farmers to pick out leaves of interest for diagnosis (classification) will ensure great AI classification results that farmers can confidently trust in. The most important principle for success is a sound standardized imaging procedure that yields repeatable results. Our team also believes that the classifier can be extended to classify pests to support farmers.

An illustration of our app prototype:

 

We will approach plant pathologists and researchers to seek out expert advice on what kind of immediate actions are required, general advice, and preventative measures for the specific disease detected. These pieces of information can then be immediately disseminated to the farmers right after the plant is classified through the app. Thus, the app can greatly support farmers in identifying diseases and recommend the best course of action. We believe that this will increase the crop yield of local farmers and bridge the gap between farmers and researchers.