lrsb / an2dl-chall-2020

Artificial neural networks and deep learning course challenges

Artificial Neural Network and Deep Learning Challenges

Image Classification • Image Segmentation • VQA

Image Classification

The goal is to classify images depicting groups of people based on the number of masked people. In the specific, the solution must discriminate between images depending on the following cases:

1. All the people in the image are wearing a mask
2. No person in the image is wearing a mask
3. Someone in the image is not wearing a mask

The following images are taken from the dataset and each one is of a different class (Up-Left (3), Up-Right (1), Bottom (2)).

Dataset Details:

• Image size: variable
• File Format: JPG
• Number of classes: 3
• Training: 5614 images
• Test: 450 images

Classes:

• 0: "NO PERSON in the image is wearing a mask", 1900 images
• 1: "ALL THE PEOPLE in the image are wearing a mask", 1897 images
• 2: "SOMEONE in the image is not wearing a mask", 1817 images

Result: 92.2% accuracy on testset.

Image Segmentation

1st ACRE Cascade Competition!

ACRE is the Agri-food Competition for Robot Evaluation, part of the METRICS project funded by the European Union’s Horizon 2020 research and innovation program under grant agreement No 871252. Autonomous robots compete to demonstrate their ability to perform agricultural tasks (such as removing weeds or surveying crops down to individual-plant resolution). At field campaigns, participants collect data that are then made available for online competitions (Cascade Campaigns) like the one you are seeing. For more information about ACRE and METRICS visit the official website.

After years of decline, the number of undernourished people began to slowly increase again in 2015. Food Security requires that everyone can have enough food produced in a sustainable manner. The topic is increasingly gaining attention as food scarcity is worsened by a continuously growing population. Also, food production is threatened by climate change. The topic is so relevant that is part of one of the 17 Sustainable Development Goals of the UN 2030 Agenda. In particular, Food Security is a pillar of SDG number 2, Zero Hunger.

In this context, the agricultural sector is going under a process of revolution by the introduction of digital technologies. The Digital Agricultural Revolution can help to reduce the use of resources (water, fertilizers, and pesticides), thus diminishing the environmental contamination and the costs for the farmers. Also, it could increase the climate resilience of crops and their productivity.

Automatic crop and weed segmentation can be a driver of innovations to optimize the agricultural processes. Indeed, automatic weed detection can be exploited by a ground robot for mechanical weeding. Thus, pesticides could even be completely avoided.

Submissions are evaluated on the mean Intersection over Union (IoU) obtained on the two classes, crop and weed. IoU is typically used in segmentation tasks and it essentially quantifies the percentage of overlap between predicted and target segmentations.

Dataset Details:

• Color space: RGB
• Number of Training images (per team per crop): 90
• Number of Test_Dev images (per team per crop): 15
• Number of Test images (per team per crop): 20

Classes:

• Crop
• Weed
• Other vegetation
• Soil

Result: 0.6443 IoU on testset.

Visual Question Answering

This competition is a visual question answering (VQA) problem on the proposed dataset. The dataset is composed by synthetic scenes, in which people and objects interact, and by corresponding questions, which are about the content of the images. Given an image and a question, the goal is to provide the correct answer. Answers belong to 3 possible categories: 'yes/no', 'counting' (from 0 to 5) and 'other' (e.g. colors, location, ecc.) answers.

Q: Is the man's shirt blue?

A: yes

Q: How many bikes?!

A: 1

Dataset Details:

• Image size: 400x700 pixels
• Color space: RGB
• File Format: png
• Total number of images: 29333

Questions:

• Number of training questions: 58832
• Number of test questions: 6372

Answers (targets):

58 possible answers belonging to 3 possible categories: 'yes/no' answers, 'counting' answers (from 0 to 5) and 'other' (e.g., colors, objects, ecc.). In the following the labels associated to each answer:

labels_dict = {
  '0': 0,
  '1': 1,
  '2': 2,
  '3': 3,
  '4': 4,
  '5': 5,
  'apple': 6,
  'baseball': 7,
  'bench': 8,
  'bike': 9,
  'bird': 10,
  'black': 11,
  'blanket': 12,
  'blue': 13,
  'bone': 14,
  'book': 15,
  'boy': 16,
  'brown': 17,
  'cat': 18,
  'chair': 19,
  'couch': 20,
  'dog': 21,
  'floor': 22,
  'food': 23,
  'football': 24,
  'girl': 25,
  'grass': 26,
  'gray': 27,
  'green': 28,
  'left': 29,
  'log': 30,
  'man': 31,
  'monkey bars': 32,
  'no': 33,
  'nothing': 34,
  'orange': 35,
  'pie': 36,
  'plant': 37,
  'playing': 38,
  'red': 39,
  'right': 40,
  'rug': 41,
  'sandbox': 42,
  'sitting': 43,
  'sleeping': 44,
  'soccer': 45,
  'squirrel': 46,
  'standing': 47,
  'stool': 48,
  'sunny': 49,
  'table': 50,
  'tree': 51,
  'watermelon': 52,
  'white': 53,
  'wine': 54,
  'woman': 55,
  'yellow': 56,
  'yes': 57
}

Best result: 63.496% accuracy on testset.