Comparison of convolutional neural networks for cloudy optical images reconstruction from single or multitemporal joint sar and optical images

Cresson Romain, Narçon N., Gaetano Raffaele, Dupuis A., Tanguy Y., May S., Commandré B.. 2022. Comparison of convolutional neural networks for cloudy optical images reconstruction from single or multitemporal joint sar and optical images. In : XXIV ISPRS Congress “Imaging today, foreseeing tomorrow”. Jiang J. (ed.), Shaker A. (ed.), H. Zhang H. (ed.). Hanovre : ISPRS, 1317-1326. (ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B3-2022) ISPRS Congress Commission III. 24, Nice, France, 6 Juin 2022/11 Juin 2022.

https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-1317-2022

Communication avec actes

Prévisualisation

Version publiée - Anglais
Sous licence

.
ID606411.pdf
Télécharger (29MB) | Prévisualisation

Résumé : With the increasing availability of optical and synthetic aperture radar (SAR) images thanks to the Sentinel constellation, and the explosion of deep learning, new methods have emerged in recent years to tackle the reconstruction of optical images that are impacted by clouds. In this paper, we focus on the evaluation of convolutional neural networks that use jointly SAR and optical images to retrieve the missing contents in one single polluted optical image. We propose a simple framework that ease the creation of datasets for the training of deep nets targeting optical image reconstruction, and for the validation of machine learning based or deterministic approaches. These methods are quite different in terms of input images constraints, and comparing them is a problematic task not addressed in the literature. We show how space partitioning data structures help to query samples in terms of cloud coverage, relative acquisition date, pixel validity and relative proximity between SAR and optical images. We generate several datasets to compare the reconstructed images from networks that use a single pair of SAR and optical image, versus networks that use multiple pairs, and a traditional deterministic approach performing interpolation in temporal domain.

Mots-clés libres : Image reconstruction, Clouds, Atmospheric perturbation, SAR, Optical, Convolutional neural networks, Gap-filling

Auteurs et affiliations