Machine and deep learning based identification of organs within LiDAR scans of tree canopies: Application to the estimation of apple production

Artzet Simon, Pallas Benoît, Costes Evelyne, Boudon Frédéric. 2020. Machine and deep learning based identification of organs within LiDAR scans of tree canopies: Application to the estimation of apple production. In : Book of abstracts of the 9th International Conference on Functional-Structural Plant Models: FSPM2020, 5 - 9 October 2020. Kahlen Katrin (ed.), Chen Tsu-Wei (ed.), Fricke Andreas (ed.), Stützel Hartmut (ed.). Hochschule Geisenheim University, University of Hannover. Hanovre : Institute of Horticultural Production Systems, Résumé, 47-48. International Conference on Functional-Structural Plant Models (FSPM 2020). 9, Allemagne, 5 Octobre 2020/9 Octobre 2020.

Matériel d'accompagnement : 1 diaporama (32 vues)

Note générale : FSPM2020 s'est déroulé virtuellement du 5-9 oct 2020

Résumé : Introduction - Theoretically, 3D acquisition systems such as terrestrial LiDAR technology could allow capturing tree shapes at high throughput with a high precision. However, in practice, the quality of the canopy reconstruction from data acquired in the field largely depends on the weather conditions, shape of the trees and on the position and number of scans collected. In a previous study, a HT protocol using T-LIDAR technology was developed for characterising simple architectural traits at the tree scale (volume, light interception efficiency) on a large population of apple trees (Coupel-Ledru et al., 2019). Nevertheless, LIDAR point clouds generated with this HT protocol were highly noisy, limiting thus the ability to identify all individual organs within the canopy. Machine and deep learning methods seems an interesting solution as they are capable to adapt to various types of noise by learning directly from training data. As a first case of study, we aimed to automatically detect apples within apple tree point clouds. For this, we developed two automatic pipelines based on machine and deep learning methods that were applied to tree point clouds acquired from LiDAR technology or simulated from synthetic data. Materials and Methods - Our study was carried out on 281, 3 and 4-years-old, apple trees scanned in 2018 and 2019 with terrestrial LiDAR using two specific acquisition protocols. The first one, called LowRes (described in Coupel-Ledru et al., 2019) consisted in taking a scan in the middle of the row every 5 trees, in the different rows of the orchard. With this protocol applied during 1 week, 250 trees with apples were scanned. A second protocol, called HiRes, consisted in scanning more precisely 31 trees: each being scanned from both sides. For the validation, mean and total weight of apple of each tree were measured allowing to estimate the number of apples. Additionally, synthetic data were generated by simulating LiDAR scans on 239 virtual apple trees from their 4 cardinal sides with the MAppleT model (Costes et al., 2008) and the PlantGL software (Pradal et al., 2009). Our training dataset was composed of 10 point clouds of apple trees, in which points were manually labeled into two classes (unknown or apple). It was complemented by a training dataset of 100 synthetic point clouds of simulated apple trees automatically labeled. The first pipeline based on machine learning includes three steps. First, points are characterized with Fast Point Feature Histograms (FPFH) (Rusu et al., 2009). Second, based on the computed FPFH features and reflectance information from the LiDAR, a random forest model was trained to predict the class of each point. Third, points are clustered into individual apples using the DBSCAN algorithm. The second pipeline merge the two first steps by classifying points directly from their position and reflectance information using a deep learning model based on PointNet (Qi et al., 2017). Finally pipeline performance was assessed by comparing the number of apple point clusters detected in each tree to the number of apples estimated during harvest. Results and Discussion - The random forest model currently showed the best performance, with an accuracy of 0.76 at test stage. Using the same configuration, the PointNet based model performed worse with an accuracy of 0.60, certainly due to the limited size of the training data. Using the random forest model, the pipeline predicted the number of apples per tree on synthetic data with a high accuracy (linear regression coefficient c=1.03 and r²=0.79). The accuracy was lower when applied on apple trees scanned with the HiRes protocol (c=2.18 and r²=0.51) and even lower with the LowRes protocol (c=2.16 and r²=0.23). A strong effect of the LIDAR position relative to the tree and consequently of the point cloud resolution was observed. Indeed, for the trees closest to the scan positions with LowRes protocol (i.e .for a distance tree-LIDAR nearly equivalent to HiRes protocol), the prediction was more reliable (c=2.16 and r²=0.45) than for the farest trees from the scan positions (c=2.08 and r²=0.07). Conclusion - We compared two phenotyping pipelines based on machine and deep learning approaches and applied them to evaluate and virtually experience LiDAR acquisition protocols in order to improve the quality of canopy reconstruction and organ detection. Our results suggest that a tradeoff between scan resolution and accuracy of organ detection has to be considered in future protocols, depending on the objectives. This tradeoff may depend on the tree age and training systems. With limited ground truth data, our experience shows better results with machine learning approach. However, results of the deep learning model can certainly be improved with more realistic geometric models and scanning noise for the synthetic data and/or with a larger amount of annotated data from real scans.

Auteurs et affiliations

• Artzet Simon, INRAE (FRA)
• Pallas Benoît, INRAE (FRA)
• Costes Evelyne, INRAE (FRA)
• Boudon Frédéric, CIRAD-BIOS-UMR AGAP (FRA) ORCID: 0000-0001-9636-3102

Source : Cirad-Agritrop (https://agritrop.cirad.fr/597184/)

Métriques

[ Page générée et mise en cache le 2024-10-30 ]