navaneethanrajasekaran / thesis

Thesis

Image annotation has been an ongoing research topic due to its prominent significance in image retrieval and computer vision. Crowd-sourcing is widely used in industry and research for manual image annotation. Crowd-sourced tasks are prone to errors, however. Direct Assessment (DA) is a methodology for crowd-sourcing human assessments of translation quality primarily developed to access the performance of machine trans- lation systems. In DA, Some of the poorly translated sentences and perfectly translated sentences are presented to workers intentionally, with Machine translated system outputs and asked them to evaluate it to identify their reliability. In this work, we develop a new assessment strategy based on DA to quality control on crowd-sourcing image annotation. The strategy will allow researchers to source quality data cheaply and on a large scale.