captain-whu / asp

Aerial Scene Parsing: From Tile-level Scene Classification to Pixel-wise Semantic Labeling

Aerial image recognition has become an active topic due to its crucial role in a wide range of applications. The interpretation methods for aerial image recognition have been developing with the improvement of image quality, of which the interpretation performance has been significantly promoted by transferring natural image knowledge with data-driven approaches. In this context, this paper addresses the aerial image recognition from tile-level scene classification to pixel-level semantic parsing after reviewing the aerial image interpretation research. Specifically, we first conduct the review by revisiting the development of aerial image interpretation prototypes and depict their connections with aerial image characters. We then present a large-scale aerial image recognition dataset which consists of more than a million scene instances, termed Million-AID. To provide reliable benchmark for future research, we also report multi-class and multi-label scene classification experiments on Million-AID using the widely employed convolutional neural networks (CNNs). Finally, we explore the transferability of semantic scene knowledge of Million-AID to advance aerial image interpretation from tile-level scene classification to pixel-level semantic parsing. Intensive experiments show that scene recognition on Million-AID is of great challenge and thus able to serve as evaluation benchmark for aerial scene classification algorithms. For scene knowledge transfer, CNN models pre-trained on Million-AID show considerable superiority than those on ImageNet for tile-level semantic interpretation, which demonstrate the strong generalization ability of the proposed Million-AID. Moreover, our designed hierarchical multi-task learning methods achieves the state-of-the-art performance for pixel-level semantic parsing on the challenging GID, which is a profitable attempt to bridge the tile-level scene classification toward pixel-level semantic parsing for aerial image interpretation. We hope our work could serve as a baseline for aerial scene recognition and inspire rethinking the semantic classification of high resolution aerial images.

A website is available at: https://captain-whu.github.io/ASP/