The impact of using multi-source remote sensing images on building segmentation with U-Net model

Abstract

The building extraction from remote sensing (RS) images has been a significant area of research in the photogrammetric and remote sensing communities, especially with the development of deep learning for over a decade. With the availability of multi-source data from RS images, accurately identifying buildings with different spatial image resolutions has become a challenging task. In this study, we assessed how the unalignment of image resolution between the training and testing data sets affects the ability to extract buildings. Image resolution plays a crucial role in the performance of building extraction. Our experiments found that as the image resolution decreased from 10 cm to 50 cm, the efficiency of building segmentation reduced from 0.759 to 0.585 according to the IoU metric. Besides, the ability and accuracy of building segmentation significantly decreased when the difference in image resolution between the training and testing data sets increased. In the case study, we use the model trained on a 10 cm resolution dataset to predict for 50 cm resolution data, the IoU drops significantly to 0.299. This research offers important insights into building segmentation tasks using multi-source data from satellite, airborne, and UAV images.