Tree Detection and Crown Delineation using UAV-SfM Data: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Rita Xu and Version 1 by Steffen Dietenberger.

Accurate detection and delineation of individual trees and their crowns in dense forest environments are essential for forest management and ecological applications. This research explores the potential of combining leaf-off and leaf-on structure from motion (SfM) data products from unoccupied aerial vehicles (UAVs) equipped with RGB cameras. 

Accurate detection and delineation of individual trees and their crowns in dense forest environments are essential for forest management and ecological applications. This research explores the potential of combining leaf-off and leaf-on structure from motion (SfM) data products from unoccupied aerial vehicles (UAVs) equipped with RGB cameras.

  • unoccupied aerial vehicle (UAV)
  • structure from motion (SfM)
  • leaf-off
  • leaf-on
  • deciduous forest
  • individual tree crown delineation (ITCD)
  • tree detection

1. UAV Imagery Processing Using Structure from Motion

One of the great advantages of UAV data is their ability to generate 3D information, such as 3D point clouds, from 2D drone imagery applying photogrammetric processing steps, commonly known as structure from motion (SfM) [16][1]. During data acquisition, highly overlapping images are captured, providing different perspectives on the same ground spots. Prominent feature points are extracted from each image, and features corresponding to the same 3D point are matched in the overlapping regions of different images. Aerial triangulation, such as bundle adjustment, is then applied to define camera positions and orientations, as well as to obtain 3D geometry, creating tracks from a set of matched features. Based on these estimated camera and image positions, densification algorithms (dense stereo matching) can be used to generate dense 3D point clouds. From the point cloud, a digital surface model (DSM) can be derived and, by projecting the single images using the DSM, an orthomosaic can be generated [16,17][1][2]. These data products can be further used to extract specific forest parameters. The UAV-SfM approach has the potential to provide both geometric and spectral datasets, serving as input data for various forest parameter extraction algorithms. While methods integrating geometrical point and spectral image data are increasingly used in the field of forestry, most studies rely on LiDAR data rather than on UAV-SfM point clouds [18,19][3][4].

2. UAV-Data-Based Products for Tree Crown Delineation

In order to analyze tree parameters on an individual tree level within a forest stand, it is necessary to first segment single trees. This involves delineating the projected tree crown, which can be identified in the orthomosaic or height models, as separate objects. This individual tree crown delineation (ITCD) serves as the foundation for various subsequent analysis steps, including tree species classification, environmental and forest monitoring at the tree level, and the extraction of individual tree parameters directly from remote sensing data [20,21,22,23][5][6][7][8]. Over the past few decades, numerous ITCD methods have developed, based on generalized characteristics of trees or forests. Most of these methods can be applied to either 2D raster such as orthomosaic, 2.5D raster such as canopy height models (CHMs), or 3D data products such as point clouds. However, there are certain methods that specifically rely on 3D data and cannot be used when only 2D data are available. These methods often originate from the field of laser scanning and are increasingly being tested for point clouds derived from SfM as well [24][9].
Most methods typically assume a similar—hemispherical to conical—tree shape, with one tree top located at the center of the crown. Due to its exposed position, the top of the tree receives the highest solar radiation, resulting in the highest intensity and brightness values [20,22][5][7]. Algorithms are expected to yield better results for forests with a sparse canopy, lower species diversity, and similar age structure. These characteristics are more commonly found in managed forests and in coniferous, savannas, or tundra forest systems [22][7].
The analysis is often divided into two parts: tree detection, which involves identifying the position of a tree trunk, and delineating the (entire) tree crown [20][5]. In the following, some of the more commonly used methods for tree detection and delineation will be presented.
Local maxima and region growing: This method builds upon the previously mentioned canopy characteristics. Initially, individual trees are detected by identifying local maxima, which ideally represent tree midpoints. These maxima can be based on both CHM values and brightness values [20,22][5][7]. Difficulties may arise when defining the search radius for local maxima, which is derived from pixel size and average crown diameter and due to the fact that, in reality, tree crowns are not symmetrically aligned around a central point. Smoothing filters can reduce unwanted noise within the maxima [23,25,26,27][8][10][11][12]. Wulder et al. [28][13] propose using varying local maxima search radii, each based on the semivariance of the pixel.
Starting from these initial seed points, neighboring pixels or objects that exhibit similarity are added to the crown objects until a termination criterion is met, indicating that the crown has been delineated. This process is known as region growing [20][5].
Valley-following approach: The valley-following approach, initially introduced by Gougeon [29][14], consists of two parts. The first part involves classifying the areas between the individual tree canopies, while the second part utilizes a rule-based method to refine these classified areas [29][14]. These intermediate areas are referred to as valleys. They are characterized by higher shading, resulting in lower intensity and brightness values compared to the surrounding areas or represent local minima values [25,30][10][15]. According to Ke and Quackenbush [20][5] and Workie [23][8], relying solely on this shade-following approach often leads to incomplete separation of individual trees because, depending on tree density, not all crowns are adequately separated from each other by “valleys”.
Watershed segmentation: The watershed segmentation, first described by Beucher and Lantuéjoul [31][16], is a form of image processing segmentation. It also draws upon the topological analogy of the canopy described earlier. In this method, the values of a gray-scale layer are inverted and “flooded”, starting from local minima (tree tops). The resulting individual watersheds are separated from each other by dams, creating distinct segments [21,22,32][6][7][17]. Markers can be used in this process, representing the local minima from which the “flooding” originates, ideally representing tree tops. The resulting segments are supposed to delimit the individual tree tops [23][8]. According to Derivaux et al. [32][17], a common challenge in this method is over-segmentation, which can be mitigated through targeted marker placement, application of smoothing filters, or through the combined use with region-merging techniques and other methods.
Template matching: Template matching is a method that can be employed to detect individual tree crowns when the tree crowns exhibit similar shapes and comparable spectral values. Templates are created based on a gray value layer, representing patterns of typical tree shapes and values, mostly averaged. Both radiometric and geometric properties of the crown are utilized, and different viewing angles can be considered. The template is compared to all possible tree points, with high correlation values above a defined threshold representing individual trees [21,22,25,33][6][7][10][18].
Deep learning methods: Deep learning methods, such as instance segmentation, are increasingly being utilized for the detection and delineation of single tree crowns [34][19]. Commonly used model architectures are Mask R-CNN [35][20], artificial neural networks [15][21], or U-nets [36][22]. These methods offer advantages such as the ability to use multi-band images as input, instead of relying solely on a single band, and a focus on textural features, which is beneficial as adjacent trees might have very similar spectral properties. However, a disadvantage is the need for training data, which are typically obtained through time-consuming manual crown delineation and/or field work.
Point cloud-based methods: Terrestrial/airborne laser scanning and UAV-SfM provide 3D point cloud data that can be directly used for the single crown delineation. Most studies focusing on 3D methods utilize LiDAR data, as UAV-SfM does not penetrate dense crown structures well, especially during leaf-on season. As a result, UAV-SfM leaf-on point clouds tend to be less dense for lower canopy and forest floor areas. Various point-cloud-based approaches have been applied for ITCD, including K-means clustering [37][23], template matching [38][24], voxel space approaches [39][25], and mean-shift segmentation [40][26]. Deep learning models are trained using reference point clouds and have been used to segment single trees [39][25], employing frameworks such as PointNet [41][27].
Several recent studies applied one or multiple of the above-mentioned methods to detect tree positions and perform crown delineation using UAV-based imagery data products. A selection of relevant studies is presented in Table 1.
Table 1. Selection of recent studies on tree detection and crown delineation in forest ecosystems using UAV optical data.
Paper Forest

Characteristics		 UAV Imagery Type Camera

Angle		 Tree Detection

Algorithm		 Crown

Delineation Algorithm		 Description
[42][28] Dry conifer forest, low tree density DJI Phantom 4 Pro (RGB) Nadir & oblique Local maxima on CHM - Various flight altitudes, patterns, and camera angles were tested, and better accuracy values were achieved by combining crosshatch flight patterns with nadir camera angles. The maximum F-score values ranged between 0.429 and 0.771.
[43][29] Broadleaf forest, test sites with different stand densities DJI Phantom 4 Pro (RGB) Nadir Local maxima on CHM Region growing +

inverse watershed segmentation		 The highest overall accuracy (F-score = 0.79) was obtained for the low-density stand by applying a region growing algorithm on the CHM. Accuracy also varied among different tree species, with the best results obtained for Caspian poplar and the lowest for Persian ironwood. In high-density stands, the crown delineation results could be improved by applying weak gaussian filtering to the CHM.
[44][30] Mixed conifer forest, open canopy DJI Phantom 3 (RGB) Nadir Local maxima on CHM - Fixed window sizes were used in the local maxima filter, and it was observed that accuracy decreased as the filter sizes exceeded 1 × 1 m. Challenges in tree detection were specifically noted in steep areas and regions with high canopy closure. DTMs obtained through SfM tended to overestimate height in dense vegetation in comparison to DTMs derived from airborne laser scanning.
[45][31] Mixed-conifer forest, moderate density DJI Phantom 4 (RGB) Nadir, oblique & composite Variable window filter (VWF),

3D point cloud-based algorithms		 - Different flight parameters (altitude, camera angle, and image overlap), SfM processing settings (depth filtering, alignment, and dense cloud quality), and tree detection algorithms (CHM smoothing, VWF parameters, and point-cloud-based methods) were investigated. Higher accuracies were achieved at high flight altitudes (120 m) and with high image overlap (90%). The combination of nadir and oblique imagery resulted in detection rates worse than using only nadir data. CHM-based VWF methods produced the most accurate results, with F-scores up to 0.664 (trees > 10 m) and 0.826 (trees > 20 m).
[46][32] Pine tree plantations DJI Mavic Pro (RGB) Oblique Local maxima on CHM - Prior to tree detection using local maxima, the CHM was mean-filtered with a user-defined filter size. Accuracy of up to 0.78 (F-score) was achieved.
[47][33] Spruce-pine forest DJI Phantom 4 Pro, Parrot Disco-Pro Ag & DJI Matrice 210 (RGB & multispectral) Nadir Local maxima on CHM Watershed segmentation Using consumer-grade cameras yielded higher tree detection rates and more accurate crown diameters compared to multispectral cameras. Cameras with higher spatial resolution performed better at higher flight altitudes, whereas the opposite was observed for cameras with lower resolution. The best results were achieved with the DJI Phantom 4 RGB drone, detecting 84% of the trees correctly. The mean absolute error of crown diameters derived was 0.79–0.99 m (Phantom 4, RGB) and 0.88–1.16 m (Zenmuse X5S).
[48][34] Mixed conifer forest, open canopy DJI Phantom 3 (RGB) Nadir Local maxima on CHM - Different window sizes for local maxima detection were tested, and the performance of smoothed and non-smoothed CHM was compared. Lower window sizes for local maxima and smoothing proved to be more successful in detecting trees. The overall F-score value was 0.86.
[49][35] Forest plantations, high canopy density DJI M600 Pro (5-lens oblique) Oblique Adaptive-/fixed-kernel bandwidth mean-shift (AMS/FMS), region growing on CHM AMS, FMS, region growing on CHM Kernel bandwidth was defined based on canopy properties and applied to the mean-shift tree detection and delineation algorithm. The AMS method outperformed FMS and seed-based region growing methods, achieving an overall accuracy of ≥0.72 for tree detection and a relative RMSE of ≤0.13 for crown width.
[50][36] Orchard yard/naturally wooded pasture/urban trees, low tree density DJI Phantom 4 (RGB)   Single shot detector (SSD) deep learning model on raster data (returning bounding boxes around tree position) Several additional datasets were derived from the RGB and DSM and were used for training purpose. SSD was used for tree detection and species classification. Ensembled models with different input datasets generally demonstrated higher performance compared to models based on only one type of input data.
[51][37] Conifer and mixed regenerating forest stands, low tree density DJI Phantom 4 Pro (RGB) Nadir - Mask R-CNN on raster data Mask R-CNN were trained using manually delineated crowns; pretrained networks were also incorporated. An average F1-score of 0.91 was achieved. ITCD remained more challenging for heterogeneous and denser forest stands, as well as for smaller crowns.
[35][20] Conifer plantation, moderate tree densities DJI Phantom 4 (multispectral)   Local maxima on RGB and CHM Marker-controlled watershed segmentation,

Mask R-CNN on raster data		 The performance of local maxima, marker-controlled watershed segmentation and Mask R-CNN were compared. Local maxima and watershed algorithms scored the best results when applied to the CHM. Overall, Mask R-CNN outperformed the classic algorithms.
[52][38] Subtropical broadleaf forest, moderate tree density DJI Matrice 600 (RGB & hyperspectral)   - Watershed-spectral-texture-controlled normalized cut An extension was made to the CHM-based watershed segmentation to reduce over-segmentation. Objects segmented by the watershed algorithm were clustered based on the normalized cut criterion, considering both spectral and textural information.
As emphasized by several of these studies, detection and delineation of tree crowns in complex forest structures with dense canopy closure and overlapping tree crowns remains challenging [35,51][20][37]. Some studies focus on ITCD in forest with low to moderate canopy closure or open forest stands [42,48,53][28][34][39], as well as on forest plantations characterized by a more regular tree spacing and structure [44,46][30][32]. These types of forest tend to facilitate the detection of crown boundaries and the number of trees, as most algorithms perform better in homogeneous forest stands with lower canopy closure [19][4].

References

  1. Jiang, S.; Jiang, C.; Jiang, W. Efficient structure from motion for large-scale UAV images: A review and a comparison of SfM tools. ISPRS J. Photogramm. Remote Sens. 2020, 167, 230–251.
  2. Schonberger, J.L.; Frahm, J.-M. Structure-from-Motion Revisited. In Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 4104–4113, ISBN 978-1-4673-8851-1.
  3. Deluzet, M.; Erudel, T.; Briottet, X.; Sheeren, D.; Fabre, S. Individual Tree Crown Delineation Method Based on Multi-Criteria Graph Using Geometric and Spectral Information: Application to Several Temperate Forest Sites. Remote Sens. 2022, 14, 1083.
  4. Zhen, Z.; Quackenbush, L.; Zhang, L. Trends in Automatic Individual Tree Crown Detection and Delineation—Evolution of LiDAR Data. Remote Sens. 2016, 8, 333.
  5. Ke, Y.; Quackenbush, L.J. A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing. Int. J. Remote Sens. 2011, 32, 4725–4747.
  6. Wang, L.; Gong, P.; Biging, G.S. Individual Tree-Crown Delineation and Treetop Detection in High-Spatial-Resolution Aerial Imagery. Photogramm. Eng. Remote Sens. 2004, 70, 351–357.
  7. Tochon, G.; Féret, J.B.; Valero, S.; Martin, R.E.; Knapp, D.E.; Salembier, P.; Chanussot, J.; Asner, G.P. On the use of binary partition trees for the tree crown segmentation of tropical rainforest hyperspectral images. Remote Sens. Environ. 2015, 159, 318–331.
  8. Workie, T.G. Estimating forest above-ground carbon using object-based analysis of very high spatial resolution satellite images. Afr. J. Environ. Sci. Technol. 2017, 11, 587–600.
  9. Lindberg, E.; Holmgren, J. Individual Tree Crown Methods for 3D Data from Remote Sensing. Curr. Forestry Rep. 2017, 3, 19–31.
  10. Erikson, M.; Olofsson, K. Comparison of three individual tree crown detection methods. Mach. Vis. Appl. 2005, 16, 258–265.
  11. Hirschmugl, M.; Ofner, M.; Raggam, J.; Schardt, M. Single tree detection in very high resolution remote sensing data. Remote Sens. Environ. 2007, 110, 533–544.
  12. Ottoy, S.; Tziolas, N.; van Meerbeek, K.; Aravidis, I.; Tilkin, S.; Sismanis, M.; Stavrakoudis, D.; Gitas, I.Z.; Zalidis, G.; de Vocht, A. Effects of Flight and Smoothing Parameters on the Detection of Taxus and Olive Trees with UAV-Borne Imagery. Drones 2022, 6, 197.
  13. Wulder, M.; Niemann, K.; Goodenough, D.G. Local Maximum Filtering for the Extraction of Tree Locations and Basal Area from High Spatial Resolution Imagery. Remote Sens. Environ. 2000, 73, 103–114.
  14. Gougeon, F.A. A Crown-Following Approach to the Automatic Delineation of Individual Tree Crowns in High Spatial Resolution Aerial Images. Can. J. Remote Sens. 1995, 21, 274–284.
  15. Gougeon, F.A.; Leckie, D.G. The Individual Tree Crown Approach Applied to Ikonos Images of a Coniferous Plantation Area. Photogramm. Eng. Remote Sens. 2006, 72, 1287–1297.
  16. Beucher, S.; Lantuéjoul, C. Use of Watersheds in Contour Detection. In Proceedings of the International Workshop on Image Processing: Real-time Edge and Motion Detection/Estimation, Rennes, France, 17–21 September 1979.
  17. Derivaux, S.; Forestier, G.; Wemmert, C.; Lefèvre, S. Supervised image segmentation using watershed transform, fuzzy classification and evolutionary computation. Pattern Recognit. Lett. 2010, 31, 2364–2374.
  18. Olofsson, K.; Wallerman, J.; Holmgren, J.; Olsson, H. Tree species discrimination using Z/I DMC imagery and template matching of single trees. Scand. J. For. Res. 2006, 21, 106–110.
  19. Zhao, H.; Morgenroth, J.; Pearse, G.; Schindler, J. A Systematic Review of Individual Tree Crown Detection and Delineation with Convolutional Neural Networks (CNN). Curr. Forestry Rep. 2023, 9, 149–170.
  20. Yu, K.; Hao, Z.; Post, C.J.; Mikhailova, E.A.; Lin, L.; Zhao, G.; Tian, S.; Liu, J. Comparison of Classical Methods and Mask R-CNN for Automatic Tree Detection and Mapping Using UAV Imagery. Remote Sens. 2022, 14, 295.
  21. Ahmadi, P.; Mansor, S.; Farjad, B.; Ghaderpour, E. Unmanned Aerial Vehicle (UAV)-Based Remote Sensing for Early-Stage Detection of Ganoderma. Remote Sens. 2022, 14, 1239.
  22. Freudenberg, M.; Magdon, P.; Nölke, N. Individual tree crown delineation in high-resolution remote sensing images based on U-Net. Neural Comput. Applic. 2022, 34, 22197–22207.
  23. Dalponte, M.; Frizzera, L.; Gianelle, D. Individual tree crown delineation and tree species classification with hyperspectral and LiDAR data. PeerJ 2019, 6, e6227.
  24. Weiser, H.; Schäfer, J.; Winiwarter, L.; Krašovec, N.; Fassnacht, F.E.; Höfle, B. Individual tree point clouds and tree measurements from multi-platform laser scanning in German forests. Earth Syst. Sci. Data 2022, 14, 2989–3012.
  25. Chen, X.; Jiang, K.; Zhu, Y.; Wang, X.; Yun, T. Individual Tree Crown Segmentation Directly from UAV-Borne LiDAR Data Using the PointNet of Deep Learning. Forests 2021, 12, 131.
  26. Xiao, W.; Zaforemska, A.; Smigaj, M.; Wang, Y.; Gaulton, R. Mean Shift Segmentation Assessment for Individual Forest Tree Delineation from Airborne Lidar Data. Remote Sens. 2019, 11, 1263.
  27. Qi, C.R.; Su, H.; Mo, K.; Guibas, L.J. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. arXiv 2016, arXiv:1612.00593.
  28. Swayze, N.C.; Tinkham, W.T.; Vogeler, J.C.; Hudak, A.T. Influence of flight parameters on UAS-based monitoring of tree height, diameter, and density. Remote Sens. Environ. 2021, 263, 112540.
  29. Miraki, M.; Sohrabi, H.; Fatehi, P.; Kneubuehler, M. Individual tree crown delineation from high-resolution UAV images in broadleaf forest. Ecol. Inform. 2021, 61, 101207.
  30. Guerra-Hernández, J.; Cosenza, D.N.; Rodriguez, L.C.E.; Silva, M.; Tomé, M.; Díaz-Varela, R.A.; González-Ferreiro, E. Comparison of ALS- and UAV(SfM)-derived high-density point clouds for individual tree detection in Eucalyptus plantations. Int. J. Remote Sens. 2018, 39, 5211–5235.
  31. Young, D.J.N.; Koontz, M.J.; Weeks, J. Optimizing aerial imagery collection and processing parameters for drone-based individual tree mapping in structurally complex conifer forests. Methods Ecol. Evol. 2022, 13, 1447–1463.
  32. Mohan, M.; Leite, R.V.; Broadbent, E.N.; Wan Mohd Jaafar, W.S.; Srinivasan, S.; Bajaj, S.; Dalla Corte, A.P.; do Amaral, C.H.; Gopan, G.; Saad, S.N.M.; et al. Individual tree detection using UAV-lidar and UAV-SfM data: A tutorial for beginners. Open Geosci. 2021, 13, 1028–1039.
  33. Komárek, J.; Klápště, P.; Hrach, K.; Klouček, T. The Potential of Widespread UAV Cameras in the Identification of Conifers and the Delineation of Their Crowns. Forests 2022, 13, 710.
  34. Mohan, M.; Silva, C.; Klauberg, C.; Jat, P.; Catts, G.; Cardil, A.; Hudak, A.; Dia, M. Individual Tree Detection from Unmanned Aerial Vehicle (UAV) Derived Canopy Height Model in an Open Canopy Mixed Conifer Forest. Forests 2017, 8, 340.
  35. Lei, L.; Yin, T.; Chai, G.; Li, Y.; Wang, Y.; Jia, X.; Zhang, X. A novel algorithm of individual tree crowns segmentation considering three-dimensional canopy attributes using UAV oblique photos. Int. J. Appl. Earth Obs. Geoinf. 2022, 112, 102893.
  36. Pleșoianu, A.-I.; Stupariu, M.-S.; Șandric, I.; Pătru-Stupariu, I.; Drăguț, L. Individual Tree-Crown Detection and Species Classification in Very High-Resolution Remote Sensing Imagery Using a Deep Learning Ensemble Model. Remote Sens. 2020, 12, 2426.
  37. Chadwick, A.J.; Goodbody, T.R.H.; Coops, N.C.; Hervieux, A.; Bater, C.W.; Martens, L.A.; White, B.; Röeser, D. Automatic Delineation and Height Measurement of Regenerating Conifer Crowns under Leaf-Off Conditions Using UAV Imagery. Remote Sens. 2020, 12, 4104.
  38. Qin, H.; Zhou, W.; Yao, Y.; Wang, W. Individual tree segmentation and tree species classification in subtropical broadleaf forests using UAV-based LiDAR, hyperspectral, and ultrahigh-resolution RGB data. Remote Sens. Environ. 2022, 280, 113143.
  39. Wang, Y.; Yang, X.; Zhang, L.; Fan, X.; Ye, Q.; Fu, L. Individual tree segmentation and tree-counting using supervised clustering. Comput. Electron. Agric. 2023, 205, 107629.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
ScholarVision Creations
Feedback