You're using an outdated browser. Please upgrade to a modern browser for the best experience.
Detection of Weeds Growing in Turf
Edit

Precision spraying can significantly reduce herbicide input for turf weed management. A major challenge for autonomous precision herbicide spraying is to accurately and reliably detect weeds growing in turf. Deep convolutional neural networks (DCNNs), an important artificial intelligent tool, demonstrated extraordinary capability to learn complex features from images. The feasibility of using DCNNs, including various image classification or object detection neural networks, has been investigated to detect weeds growing in turf.

computer vision deep learning turf precision herbicide application

1. Introduction

Turf is the predominant vegetation cover in urban landscapes, golf courses, residential lawns, and sports fields. In the United States, it was estimated that the total turf area covers 163,812 km2 with a lower and upper 95% confidence interval bounds of ±35,850 km2 [1]. According to the National Golf Foundation, there are over 15,000 golf courses, with an average of 50 to 73 ha per golf course, in the United States [2]. Turf offers many benefits, such as providing evaporative cooling in an urban area, remediating contaminated soil, absorbing atmospheric pollutants, and increasing the aesthetic value of residential and non-residential areas [3]. Nevertheless, weeds are a challenging issue for turf management. Weeds compete with turf for sunlight, nutrients, and water resources and may significantly reduce turf aesthetics and functionalities [4][5][6].
Turf managers predominately rely on synthetic herbicides for controlling weeds [7][8][9]. Unfortunately, for controlling certain weeds growing in turf, the present control programs relying on synthetic herbicides are not cost-efficient [6][10]. For example, repeat applications of sulfonylurea herbicides thiencarbazone + foramsulfuron + halosulfuron in combination with amicarbazone at 0.25 kg ai ha−1 adequately controlled tropical signalgrass (Urochloa distachya (L.) T.Q. Nguyen] in bermudagrass (Cynodon dactylon (L.) Pers.) [11]. However, repeated application of this herbicide program is expensive since a single application of amicarbazone at 0.25 kg ai ha−1 would cost approximately 1500 U.S. dollars.
Moreover, some synthetic herbicides used in turf are suspected of polluting environments [12]. Possible adverse impacts include, but are not limited to, the damaging effect on non-target organisms, water pollution, and harmful impact on humans [13][14][15]. In the United States, it was reported that nearly 80% of stream samples in urban/suburban contained at least five pesticides [16]. Atrazine is one of the most commonly used herbicides in warm-season turfgrasses [17]; however, it is frequently detected in underground water [16][18]. Monosodium methylarsenate (MSMA) is a highly effective broad-spectrum herbicide against a number of difficult-to-control weed species, including dallisgrass (Paspalum dilatatum Poir.), but pollutes underground water [19]. Following application, MSMA is converted to a more toxic form of inorganic arsenic that may contaminate water through soil runoff [20]. In the United States, only spot-treatment of MSMA is permitted to be sprayed on established golf courses [21].
Deep learning, a subset of machine learning technology, has emerged as successful applications in various scientific domains, including computer vision [22][23][24]. Deep convolutional neural networks (DCNNs) demonstrated extraordinary capability to extract complex features from images [25] and are utilized as a tool to detect weeds and perform precision herbicide spraying [26][27][28][29][30][31]. For example, See & Spray®, an autonomous smart sprayer utilizing DCNNs for weed detection, has been developed for precision herbicide application in agronomic crops [32]. Detection of weeds growing in turf needs to consider weeds (e.g., weed growth stage, weed species, and biotypes) and turf factors (e.g., turf quality, mowing height, dormant vs. non-dormant stages). DCNNs recognize weeds based on plant morphological features, leaf texture, and color [33][34][35][36]. Therefore, it is logical to assume that the detection of weeds growing in dormant turfgrass is easier than in actively growing turfgrass; the detection of large-leaved weeds is easier than small-leaved weeds; and the detection of broadleaf weeds is easier than grasses or grass-like weeds growing in turfgrass (Figure 1) [37][38].
Figure 1. Presumed difficulty of using DCNNs for detecting weeds growing in turfgrass.

2. Detection of Weeds Growing in Turf

2.1. Image Classification versus Object Detection

As shown in Figure 2, weeds grow either scatteringly or in relatively large patches in turf. The preparation of training datasets for object detection neural networks involves drawing bounding boxes on the training images. For this reason, object detectors are used to detect scattered weeds growing in turf [37][39][40]. However, to detect inconspicuous weeds, such as common lespedeza (Kummerowia striata L.) and spotted spurge (Euphorbia maculata L.), labeling the ground-truth locations within images for individual weeds is rather painstaking and laborious. Moreover, when detecting weeds in relatively large patches, a large number of weeds per image need to be labeled prior to training the object detectors.
Figure 2. Dandelion (Taraxacum officinale F.H. Wigg.) scatteringly grows in perennial ryegrass (Lolium perenne L.) turf (A). Smooth crabgrass (Digitaria ischaemum (Schreb.) Muhl) grows in a relatively large patch in bermudagrass turf (B).
Compared to object detectors, the training of image classification neural networks takes less time because it does not need to draw the bounding boxes. The grid cells (sub-images) could be created on the input images. Subsequently, the developed image classification neural networks could be used to detect if the grid cells contain weeds [41]. The image classification neural networks could be employed to detect either scattered or relatively large-patched weeds in turf. When using the image classification neural networks as the machine vision decision system, the spray outputs of the smart sprayers need to be the same or slightly larger than the size of the sub-images in order to fully cover the sub-images containing the target weeds [42].

2.2. Detection of Weeds in Dormant Turfgrass

Yu et al. [37] evaluated DetectNet, GoogLeNet, and VGGNet for detecting annual bluegrass (Poa annua L.) or annual bluegrass growing in proximity to various broadleaf weeds, such as common chickweed (Stellaria media (L.) Vill.), dandelion, and white clover (Trifolium repens L.). The authors reported that DetectNet was the most effective, while GoogLeNet was the least effective among the neural networks evaluated for detecting annual bluegrass in dormant bermudagrass. DetectNet achieved high precision and recall values with the highest F1 score (≥0.99) at detecting annual bluegrass growing in dormant bermudagrass. In another study, Yu et al. [38] reported that VGGNet achieved high F1 scores with high recall values (1.00) for detecting various broadleaf weeds, including common chickweed [Stellaria media (L.) Vill.], dandelion, henbit (Lamium amplexicaule L.), purple deadnettle (Lamium purpureum L.), and white clover (Trifolium repens L.) in dormant bermudagrass turf.

2.3. Detection of Broadleaf Weeds in Actively Growing Turfgrass

Yu et al. [43] compared DetectNet, GoogLeNet, and VGGNet to detect dandelion, ground ivy (Glechoma hederacea L.), and spotted spurge growing in actively growing perennial ryegrass and reported that VGGNet was more effective than AlexNet and GoogLeNet in detecting these weeds. When the neural networks were trained with 15,486 negative (images without weeds) and 17,600 positive images (6500 images contain spotted spurge, 4600 images contain ground ivy, and 6500 images contain dandelion), VGGNet achieved high F1 scores (≥0.9345) with high recall values (≥0.9952) to detect these weeds; the F1 scores of AlexNet and GoogLeNet did not exceed 0.9103, while DetectNet was highly effective and achieved high F1 scores (≥0.9843) to detect dandelion growing in perennial ryegrass.

2.4. Detection of Grass or Grass-Like Weeds in Actively Growing Turfgrasss

It was assumed that machine vision-based detection of grass or grass-like weeds in turfgrass is especially challenging due to the similarity in plant morphology [37][38][44]. Yu et al. [44] evaluated the use of image classification neural networks, including AlexNet, GoogLeNet, and VGGNet, for the detection of smooth crabgrass (Digitaria ischaemum L.), dallisgrass, doveweed [Murdannia nudiflora (L.) Brenan], and tropical signalgrass [Urochloa distachya (L.) T.Q. Nguyen] growing in bermudagrass with erratic turf surface conditions (i.e., varying mowing heights and surface qualities). The authors found that VGGNet achieved excellent performances for detecting these weed species with high F1 scores (≥0.93) and recall values (1.00). Although AlexNet and GoogLeNet achieved high recall, they exhibited low precision [44]. The low precision indicates that the neural networks are more likely to misclassify turfgrass as weeds, leading to herbicide applications in turf where weeds do not occur.

2.5. Weed Localization

Object detectors, such as Faster R-CNN [45], YOLO (You Only Look Once) [46], and SSD (Single Shot Detector) [47], generate bounding box outputs but do not determine the exact location of weeds on the images. Mask R-CNN, a segmentation network, can address this issue because it can achieve finer image segmentation for object detection [48]. Nevertheless, this neural network requires pixel-wise precise ground truth labeling, which is time-consuming. Xie et al. [49] developed an algorithm to generate synthetic data and constructed a nutsedge (Cyperus spp.) skeleton-based probabilistic map as the neural network input to reduce the dependence on pixel-wise precise labeling. This approach effectively overcame the effect of insufficient training images and reduced the labeling time by 95%, and meanwhile, it outperformed the original Mask R-CNN approach for weed detection.
Despite all the successes described in previous paragraphs, detecting weeds growing in turf with image classification neural networks faces challenges [37][50]. While previous researchers reported that image classification neural networks could detect and discriminate the sub-images containing weeds, they did not attempt to identify the location of weeds on the images [37][38][50]. When using the image classification neural networks for weed detection, the exact location of the sub-images containing weeds on the input images needs to be determined to realize precision herbicide application with the smart sprayers. To address this issue, Yu and Jin [41] developed a software that can integrate image classification neural networks and OpenCV-Python to create the grid cells on the input images. This software can crop the testing image (1920 × 1080 pixels) into a total of 40 equal size grid cells. The software marks the grid cells as “spray” if the inference of the developed neural networks indicates that they contained weeds and marks as “non-spray” if the inference indicates that they did not contain weeds. The x, y coordinates of the grid cells containing weeds are located with the developed software when used in conjunction with the image classification neural networks. Using this software, Jin et al. [41] found that EfficientNetV2 was reliably inferred if the grid cells contained the target weeds with high F1 scores (≥0.980) and noted that DenseNet, EfficientNetV2, ResNet, RegNet, and VGGNet reliably detected and discriminated the grid cells contained in dandelion, dallisgrass, purple nutsedge, and white clover.

2.6. Detection of Weeds Growing in Various Turfgrass Surface Conditions

Image classification and object detection neural networks can detect weeds growing in various turf surface conditions (Table 1) [37][50]. When the neural networks were trained with images taken at athletic fields, institutional lawns, and various golf course management zones (i.e., fairways, tees, putting greens, and rough), VGGNet demonstrated high F1 score values (≥0.95) and effectively detected dollar weeds (Hydrocotyle spp.), old world diamond-flower (Hedyotis cormybosa L. Lam.), and Florida pusley (Richardia scabra L.) in actively growing bermudagrass turf [37].
Table 1. A summary of published reports on the use of DCNNs for detecting weeds growing in turf.

References

  1. Milesi, C.; Elvidge, C.; Dietz, J.; Tuttle, B.; Nemani, R.; Running, S. A strategy for mapping and modeling the ecological effects of US lawns. J. Turfgrass Manag. 2005, 1, 83–97.
  2. Beditz, J. The development and growth of the US golf market. In Science and Golf II; Taylor & Francis: Abingdon, UK, 2002; pp. 678–686.
  3. Stier, J.C.; Steinke, K.; Ervin, E.H.; Higginson, F.R.; McMaugh, P.E. Turfgrass benefits and issues. Turfgrass Biol. Use Manag. 2013, 56, 105–145.
  4. Busey, P. Cultural management of weeds in turfgrass: A review. Crop Sci. 2003, 43, 1899–1911.
  5. Hao, Z.; Bagavathiannan, M.; Li, Y.; Qu, M.; Wang, Z.; Yu, J. Wood vinegar for control of broadleaf weeds in dormant turfgrass. Weed Technol. 2021, 35, 901–907.
  6. McElroy, J.; Martins, D. Use of herbicides on turfgrass. Planta Daninha 2013, 31, 455–467.
  7. Gómez de Barreda, D.; Yu, J.; McCullough, P.E. Seedling tolerance of cool-season turfgrasses to metamifop. HortScience 2013, 48, 1313–1316.
  8. McCullough, P.E.; Yu, J.; de Barreda, D.G. Seashore paspalum (Paspalum vaginatum) tolerance to pronamide applications for annual bluegrass control. Weed Technol. 2012, 26, 289–293.
  9. Tate, T.M.; Meyer, W.A.; McCullough, P.E.; Yu, J. Evaluation of mesotrione tolerance levels and mesotrione absorption and translocation in three fine fescue species. Weed Sci. 2019, 67, 497–503.
  10. McCullough, P.E.; Yu, J.; Raymer, P.L.; Chen, Z. First report of ACCase-resistant goosegrass (Eleusine indica) in the United States. Weed Sci. 2016, 64, 399–408.
  11. Pearsaul, D.G.; Leon, R.G.; Sellers, B.A.; Silveira, M.L.; Odero, D.C. Evaluation of verticutting and herbicides for tropical signalgrass (Urochloa subquadripara) control in turf. Weed Technol. 2018, 32, 392–397.
  12. Balogh, J.C.; Anderson, J.L. Environmental Impacts of Turfgrass Pesticides; Golf Course Management & Construction: Boca Raton, FL, USA, 2020; pp. 221–353.
  13. Tappe, W.; Groeneweg, J.; Jantsch, B. Diffuse atrazine pollution in German aquifers. Biodegradation 2002, 13, 3–10.
  14. Nitschke, L.; Schüssler, W. Surface water pollution by herbicides from effluents of waste water treatment plants. Chemosphere 1998, 36, 35–41.
  15. Starrett, S.; Christians, N.; Al Austin, T. Movement of herbicides under two irrigation regimes applied to turfgrass. Adv. Environ. Res. 2000, 4, 169–176.
  16. Petrovic, A.M.; Easton, Z.M. The role of turfgrass management in the water quality of urban environments. Int. Turfgrass Soc. Res. J. 2005, 10, 55–69.
  17. Yu, J.; McCullough, P.E. Triclopyr reduces foliar bleaching from mesotrione and enhances efficacy for smooth crabgrass control by altering uptake and translocation. Weed Technol. 2016, 30, 516–523.
  18. Pimentel, D.; Zuniga, R.; Morrison, D. Update on the environmental and economic costs associated with alien-invasive species in the United States. Ecol. Econ. 2005, 52, 273–288.
  19. Mahoney, D.J.; Gannon, T.W.; Jeffries, M.D.; Matteson, A.R.; Polizzotto, M.L. Management considerations to minimize environmental impacts of arsenic following monosodium methylarsenate (MSMA) applications to turfgrass. J. Environ. Manag. 2015, 150, 444–450.
  20. Mulligan, C.; Yong, R.; Gibbs, B. Remediation technologies for metal-contaminated soils and groundwater: An evaluation. Eng. Geol. 2001, 60, 193–207.
  21. Busey, P. Managing goosegrass II. Removal. Golf Course Manag. 2004, 72, 132–136.
  22. Shi, J.; Li, Z.; Zhu, T.; Wang, D.; Ni, C. Defect detection of industry wood veneer based on NAS and multi-channel mask R-CNN. Sensors 2020, 20, 4398.
  23. He, T.; Liu, Y.; Yu, Y.; Zhao, Q.; Hu, Z. Application of deep convolutional neural network on feature extraction and detection of wood defects. Measurement 2020, 152, 107357.
  24. Zhou, H.; Zhuang, Z.; Liu, Y.; Liu, Y.; Zhang, X. Defect Classification of Green Plums Based on Deep Learning. Sensors 2020, 20, 6993.
  25. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444.
  26. Le, V.N.T.; Ahderom, S.; Alameh, K. Performances of the lbp based algorithm over cnn models for detecting crops and weeds with similar morphologies. Sensors 2020, 20, 2193.
  27. Liu, B.; Bruch, R. Weed Detection for Selective Spraying: A Review. Curr. Robot. Rep. 2020, 1, 19–26.
  28. Sharpe, S.M.; Schumann, A.W.; Boyd, N.S. Detection of Carolina geranium (Geranium carolinianum) growing in competition with strawberry using convolutional neural networks. Weed Sci. 2019, 67, 239–245.
  29. Jin, X.; Che, J.; Chen, Y. Weed identification using deep learning and image processing in vegetable plantation. IEEE Access 2021, 9, 10940–10950.
  30. Zhuang, J.; Li, X.; Bagavathiannan, M.; Jin, X.; Yang, J.; Meng, W.; Li, T.; Li, L.; Wang, Y.; Chen, Y. Evaluation of different deep convolutional neural networks for detection of broadleaf weed seedlings in wheat. Pest Manag. Sci. 2022, 78, 521–529.
  31. Jin, X.; Sun, Y.; Che, J.; Bagavathiannan, M.; Yu, J.; Chen, Y. A novel deep learning-based method for detection of weeds in vegetables. Pest Manag. Sci. 2022, 78, 1861–1869.
  32. Chostner, B. See & Spray: The next generation of weed control. Resour. Mag. 2017, 24, 4–5.
  33. Dyrmann, M.; Karstoft, H.; Midtiby, H.S. Plant species classification using deep convolutional neural network. Biosyst. Eng. 2016, 151, 72–80.
  34. Ghosal, S.; Blystone, D.; Singh, A.K.; Ganapathysubramanian, B.; Singh, A.; Sarkar, S. An explainable deep machine vision framework for plant stress phenotyping. Proc. Natl. Acad. Sci. USA 2018, 115, 4613–4618.
  35. Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw. 2015, 61, 85–117.
  36. Wang, A.; Zhang, W.; Wei, X. A review on weed detection using ground-based machine vision and image processing techniques. Comput. Electron. Agric. 2019, 158, 226–240.
  37. Yu, J.; Sharpe, S.M.; Schumann, A.W.; Boyd, N.S. Deep learning for image-based weed detection in turfgrass. Eur. J. Agron. 2019, 104, 78–84.
  38. Yu, J.; Sharpe, S.M.; Schumann, A.W.; Boyd, N.S. Detection of broadleaf weeds growing in turfgrass with convolutional neural networks. Pest Manag. Sci. 2019, 75, 2211–2218.
  39. Medrano, R. Feasibility of Real-Time Weed Detection in Turfgrass on an Edge Device. Master’s Thesis, The California State Univeristy, Camarillo, CA, USA, 2021.
  40. Sharpe, S.M.; Schumann, A.W.; Boyd, N.S. Goosegrass detection in strawberry and tomato using a convolutional neural network. Sci. Rep. 2020, 10, 9548.
  41. Jin, X.; Bagavathiannan, M.; McCullough, P.E.; Chen, Y.; Yu, J. A deep learning-based method for classification, detection, and localization of weeds in turfgrass. Pest Manag. Sci. 2022, 78, 4809–4821.
  42. Jin, X.; Bagavathiannan, M.; Maity, A.; Chen, Y.; Yu, J. Deep learning for detecting herbicide weed control spectrum in turfgrass. Plant Methods 2022, 18, 94.
  43. Yu, J.; Schumann, A.W.; Cao, Z.; Sharpe, S.M.; Boyd, N.S. Weed Detection in Perennial Ryegrass With Deep Learning Convolutional Neural Network. Front. Plant Sci. 2019, 10, 1422.
  44. Yu, J.; Schumann, A.W.; Sharpe, S.M.; Li, X.; Boyd, N.S. Detection of grassy weeds in bermudagrass with deep convolutional neural networks. Weed Sci. 2020, 68, 545–552.
  45. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv 2015, arXiv:1506.01497.
  46. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788.
  47. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. Ssd: Single shot multibox detector. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 11–14 October 2016; pp. 21–37.
  48. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969.
  49. Xie, S.; Hu, C.; Bagavathiannan, M.; Song, D. Toward Robotic Weed Control: Detection of Nutsedge Weed in Bermudagrass Turf Using Inaccurate and Insufficient Training Data. arXiv 2021, arXiv:2106.08897.
  50. Zhuang, J.; Jin, X.; Chen, Y.; Meng, W.; Wang, Y.; Yu, J.; Muthukumar, B. Drought stress impact on the performance of deep convolutional neural networks for weed detection in Bahiagrass. Grass Forage Sci. 2022. early view.
More
Upload a video for this entry
Information
Subjects: Agronomy
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register : , , ,
View Times: 685
Revisions: 2 times (View History)
Update Date: 19 Dec 2022
Academic Video Service