Submitted Successfully!
To reward your contribution, here is a gift for you: A free trial for our video production service.
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Version Summary Created by Modification Content Size Created at Operation
1
JOUNGMIN PARK
 -- 1891 2024-02-05 13:23:17 |
2 format correct
Wendy Huang
Meta information modification 1891 2024-02-07 07:24:30 |

Video Upload Options

Do you have a full video?
Send video materials Upload full video

Confirm

Are you sure to Delete?
Yes No
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Park, J.; Shin, J.; Kim, R.; An, S.; Lee, S.; Kim, J.; Oh, J.; Jeong, Y.; Kim, S.; Jeong, Y.R.; et al. Strawberry Ripeness Classification. Encyclopedia. Available online: https://encyclopedia.pub/entry/54766 (accessed on 02 July 2024).
Park J, Shin J, Kim R, An S, Lee S, Kim J, et al. Strawberry Ripeness Classification. Encyclopedia. Available at: https://encyclopedia.pub/entry/54766. Accessed July 02, 2024.
Park, Joungmin, Jinyoung Shin, Raehyeong Kim, Seongmo An, Sangho Lee, Jinyeol Kim, Jongwon Oh, Youngwoo Jeong, Soohee Kim, Yue Ri Jeong, et al. "Strawberry Ripeness Classification" Encyclopedia, https://encyclopedia.pub/entry/54766 (accessed July 02, 2024).
Park, J., Shin, J., Kim, R., An, S., Lee, S., Kim, J., Oh, J., Jeong, Y., Kim, S., Jeong, Y.R., & Lee, S.E. (2024, February 05). Strawberry Ripeness Classification. In Encyclopedia. https://encyclopedia.pub/entry/54766
Park, Joungmin, et al. "Strawberry Ripeness Classification." Encyclopedia. Web. 05 February, 2024.
Strawberry Ripeness Classification
Edit
This entry is adapted from the peer-reviewed paper 10.3390/electronics13020344

Image analysis-based artificial intelligence (AI) models leveraging convolutional neural networks (CNN) take a significant role in evaluating the ripeness of strawberry, contributing to the maximization of productivity. However, the convolution, which constitutes the majority of the CNN models, imposes significant computational burdens. Additionally, the dense operations in the fully connected (FC) layer necessitate a vast number of parameters and entail extensive external memory access. Therefore, reducing the computational burden of convolution operations and alleviating memory overhead is essential in embedded environment.

artificial intelligence (AI) convolutional neural network (CNN) strawberry ripeness sensor computer vision technology

1. Introduction

Recently, AI technologies have undergone rapid development, bringing significant changes to the field of agriculture. Research and applications utilizing AI to enhance agricultural productivity and promote sustainable farming practices have emerged diversely [1][2][3]. Within the domain of AI applications, image analysis models assume a critical role in shaping agricultural production and management. Image-based AI models contribute significantly to discerning the state of crops and evaluating the ripeness of fruits, thereby predicting the optimal harvesting time to maximize productivity [4][5][6]. Particularly, accurately measuring the ripeness of fruits in this field is emphasized as a key task in the modern agricultural paradigm to optimize yield and resource utilization. Strawberries, being a representative crop, enable the evaluation of ripeness levels based on color. Strawberries are economically valuable fruits due to their sweet taste and rich nutritional content [7][8]. The flavor and market impact of strawberries are significantly influenced by their degree of ripeness. A more advanced level of ripening results in distinct taste variations and has considerable implications for distribution. For these reasons, evaluating the appropriate ripeness of strawberries is essential as it enhances the marketability and economic viability of the fruit through timely harvesting [9].
To harvest strawberries at the right time, the characteristics of strawberries must be accurately extracted and classified according to their ripeness. Specifically, finding unique patterns or structures within images is crucial for extracting appropriate features from the images. Traditional methods for feature extraction include algorithms such as scale-invariant feature transform (SIFT), speeded-up robust features (SURF), and features from accelerated segment test (FAST). SIFT is an algorithm that detects features in an image that are invariant to scale and rotation [10]. SURF improves upon SIFT, providing faster processing speeds [11]. FAST is a computer vision algorithm that rapidly detects corners in an image, identifying keypoints based on intensity differences between central and neighboring pixels [12]. However, deep learning (DL), which automatically learns features from large amounts of data and is applicable to various complex tasks, has demonstrated higher performance and versatility compared to traditional feature extraction methods.
Among DL models for image analysis, CNN models especially demonstrate excellent performance in extracting features and learning from images, making them effective in analyzing images in complex agricultural environments [13]. The CNN model consists of a combination of a convolution layer, responsible for extracting feature maps, and an FC layer, which performs the final classification based on the extracted feature maps. In the convolution layer, kernels are applied using small windows to perform convolution operations, extracting various feature maps from the input image. During this process, each kernel effectively captures features related to the relationships between pixels within the window, revealing information about local patterns, edges, textures, and more [14]. In the FC layer, a flattened feature vector is typically received as input. In this process, the locally extracted multi-channel feature map data from convolution layer is flattened into a single-channel feature vector. This flattened vector is connected to the neurons in the FC layer, where each neuron corresponds to a specific feature. Based on this connectivity, the model learns global patterns of the input image, leading to the final classification [15].
In CNN models, convolution operations constitute a significant portion of the overall computation. Equation (1) represents the feature map, expressed as the cumulative sum of the product between weights and input vectors. Here, i and j denote the positions of the output feature map, M and N represent the kernel size, and C indicates the number of channels in the input image. The convolution operations, especially in AI processors that demand high performance, require significant computational resources. Therefore, the developments of convolution operation accelerators have emerged as crucial topics in the field [16]. In particular, the CNN accelerator utilizing stochastic computing (SC-CNN) has achieved energy efficiency and performance improvement by introducing probabilistic bit representation and highly parallelized methods [17][18]. Additionally, binary neural networks (BNN) are known for limiting weights and activations to binary values, making them suitable for edge devices with low power consumption and resource requirements. BNN contributes to improving latency and throughput performance of specific hardware models by binarizing the network [19][20].
 
F e a t u r e m a p ( i , j ) = ∑ m = 0 M − 1 ∑ n = 0 N − 1 ∑ c = 0 C − 1 I n p u t ( m , n , c ) ∗ K e r n e l ( i − m , j − n , c )
However, the CNN accelerators primarily emphasize convolution operations, necessitating consideration for operations in the FC layer. As the depth of deep neural networks (DNN) increases, in the FC layer, there are numerous parameters that need to be accessed from external memory to perform extensive weight multiplication operations for millions of parameters [21][22]. Furthermore, the FC layer is to be challenging compared to accelerating convolution layer, especially in achieving high data reuse performance. These requirements lead to issues such as speed degradation due to external memory access and memory overhead arising from the substantial volume of parameters stored in external memory [23]. Especially in embedded systems, limited memory resources, and computational performance become obstacles to performing FC layer operations. Additionally, the massive computational volume of the FC layer hinders efficient power consumption and management in embedded systems. To address these challenges, various methods have been proposed, including techniques based on the utilization of statistics from extracted feature maps to reduce power consumption and approaches employing the k-NN algorithm [24]. Especially, the k-NN algorithm demonstrates robust performance, particularly in small-scale datasets, and is efficient in terms of both memory and computational resources due to its concise and intuitive structure. Furthermore, the k-NN algorithm performs relatively fewer computations compared to the FC layer, which involves multiplication operations with weights for numerous nodes. Therefore, the algorithm notably enhances performance and efficiency, especially on edge devices, where optimizing limited resources is paramount [25].

2. Strawberry Ripeness Classification Systems

With the development of AI, sensor technology, and computer vision technology, research has been conducted to apply these technologies to the agricultural field, such as smart farms, to create high added-value [26][27][28][29][30][31][32][33]. In particular, for the system that determines the ripeness of fruits, many studies have been carried out utilizing AI and computer vision technology, as it is often necessary to judge the ripeness by external factors such as color, shape, etc. [26][27][28].
In [26], a system was proposed to estimate the ripeness of Hass avocados with hyperspectral imaging (HSI) and DL. A camera with a spectral resolution of 1.3 nm was employed to obtain hyperspectral images, and a total of 551 avocado images were obtained. The 44,096 sub-images obtained from the avocado images were used for training and testing the CNN to determine the number of days left until the avocado ripens, and an average of 1.17 days of root mean squared error (RMSE) was obtained. In [27], a system capable of object detection of Cavendish bananas and Carabao mangoes with the you only look once (YOLO) algorithm and determining the ripeness of the fruits with a support vector machine (SVM) is shown. With 33 test images, the system’s overall accuracy regarding its classification of ripeness to the fruit subjects is 90.9%. Meanwhile, in addition to research on systems employing AI and computer vision technology, there are also studies using new sensors. In [28], a tactile-based sensor was used to determine the ripeness of fruit. The ripeness was estimated by developing a low-cost, lithography-free, highly sensitive and flexible capacitive tactile sensor to measure the firmness and stiffness of the fruit, and it was confirmed that the capacitance of the sensor varies depending on the stiffness of tomatoes.
Strawberries have been widely employed in many studies as a fruit for ripeness detection research based on computer vision technology due to their distinct color changes depending on the degree of ripeness [29][30][31][32][33]. The authors in [29] utilized the YOLOv7 model for object detection and ripeness measurement of strawberries. They also visualized the location and ripeness with augmented reality (AR) technology. The ripeness of strawberries was divided into ‘Unripe’, ‘Partially Ripe’, and ‘Ripe’, and the performance of the model was improved by preventing overfitting through data augmentation. A total of 2900 images were set for training and validation, and the accuracy of ripeness prediction had an F1 score of 0.92. In [30], a two-path model that learns the coordinates of the stalk while learning the ripeness utilizing semantic segmentation was proposed. The model was designed based on VGG16, and the highest accuracy was 90.33%. In [31], a method of measuring strawberry ripeness by extracting RGB features and utilizing the k-NN algorithm was proposed. The authors measured the ripeness using the k-NN algorithm by inputting representative values calculated from features such as component values, area features, roundness, slenderness, etc., extracted from the RGB data of the image. As a result of training with a total of 30 images and testing with 20 images, it showed a ripeness prediction accuracy of 85%. In [32], a multiclass SVM with a radial basis function (RBF) kernel was utilized. The proposed system consisted of an image resizing process to reduce execution time and memory usage, RGB to HSV converting, feature extraction represented by the average value of RGB, and classification with the SVM. The maximum accuracy of the system was 85.64%. In [33], a ripeness detection system that combines semantic graphics for data annotation with a fully convolutional neural network (FCN) made up of an encoder and an expanding decoder was proposed, showing an average ripeness prediction accuracy of 88.57%. Table 1 displays the comparison of related works regarding strawberry ripeness estimation mentioned above.
Table 1. The comparison of related works in strawberry ripeness prediction.
Source Proposed Approach Accuracy Pros Cons
[29] (1) Combination of YOLOv7 and AR to detect and visualize the ripeness of strawberries. (2) Multiscale training of YOLO to overcome challenges in detecting small objects. F1 score of 0.92 It has proven superior performance compared to other state-of-the-art methodologies. It is not suitable for embedded environments because there are many computing elements.
[30] (1) The ripeness of strawberries is detected with a semantic segmentation model. (2) To reduce parameters of the model, a two-path convolution method was proposed. 90.33% The two-path convolution method effectively reduced the number of parameters. The amount of strawberry image data is small, and the diversity is insufficient.
[31] (1) RGB feature extraction: to obtain the color information of strawberry images, the paper calculates the mean and standard deviation of each color component. (2) The k-NN algorithm was used to classify the strawberry images into ripeness categories. 85% The amount of computation is low compared to other methods. RGB features alone cannot completely distinguish the degree of ripening of strawberries.
[32] (1) SVM with RBF kernel function is used to classify the ripeness. (2) A prototype of the strawberry classification system was constructed with real-time video. 85.64% SVM is a simple and lightweight classification method. The number of categories that can be classified with RBF kernel is limited.
[33] (1) Semantic graphics for data annotation: use simple graphic elements to display the quantity and ripening of strawberries. (2) FCN is utilized to recognize and learn semitangible graphics. 88.57% Desired targets can be efficiently tagged with semantic graphics. The external characteristics of strawberries are not fully represented.

References

  1. Bhat, S.A.; Huang, N.F. Big Data and AI Revolution in Precision Agriculture: Survey and Challenges. IEEE Access 2021, 9, 110209–110222.
  2. Ragavi, B.; Pavithra, L.; Sandhiyadevi, P.; Mohanapriya, G.; Harikirubha, S. Smart Agriculture with AI Sensor by Using Agrobot. In Proceedings of the 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 11–13 March 2020; pp. 1–4.
  3. Kamilaris, A.; Prenafeta-Boldú, F.X. Deep learning in agriculture: A survey. Comput. Electron. Agric. 2018, 147, 70–90.
  4. Dey, B.; Masum Ul Haque, M.; Khatun, R.; Ahmed, R. Comparative performance of four CNN-based deep learning variants in detecting Hispa pest, two fungal diseases, and NPK deficiency symptoms of rice (Oryza sativa). Comput. Electron. Agric. 2022, 202, 107340.
  5. Wang, H.; Mou, Q.; Yue, Y.; Zhao, H. Research on Detection Technology of Various Fruit Disease Spots Based on Mask R-CNN. In Proceedings of the 2020 IEEE International Conference on Mechatronics and Automation (ICMA), Beijing, China, 13–16 October 2020; pp. 1083–1087.
  6. Kuo, H.H.; Barik, D.S.; Zhou, J.Y.; Hong, Y.K.; Yan, J.J.; Yen, M.H. Design and Implementation of AI aided Fruit Grading Using Image Recognition. In Proceedings of the 2022 IEEE/ACIS 23rd International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), Taichung, Taiwan, 7–9 December 2022; pp. 194–199.
  7. Devi, M.S.; Shanthana, S.; Hemasri, B. Inception Adaptive Gradient L2 Regularized Learning Rate CNN for Strawberry Leaf disease Detection. In Proceedings of the 2023 Fifth International Conference on Electrical, Computer and Communication Technologies (ICECCT), Erode, India, 22–24 February 2023; pp. 1–4.
  8. Huang, Z.; Sklar, E.; Parsons, S. Design of Automatic Strawberry Harvest Robot Suitable in Complex Environments. In Proceedings of the Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, HRI ’20, Cambridge, UK, 23–26 March 2020; pp. 567–569.
  9. Cho, W.; Na, M.; Kim, S.; Jeon, W. Automatic prediction of brix and acidity in stages of ripeness of strawberries using image processing techniques. In Proceedings of the 2019 34th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), JeJu, Republic of Korea, 23–26 June 2019; pp. 1–4.
  10. Agarwal, A.; Samaiya, D.; Gupta, K.K. A Comparative Study of SIFT and SURF Algorithms under Different Object and Background Conditions. In Proceedings of the 2017 International Conference on Information Technology (ICIT), Bhubaneswar, India, 21–23 December 2017; pp. 42–45.
  11. Micheal, A.A.; Vani, K. Comparative analysis of SIFT and SURF on KLT tracker for UAV applications. In Proceedings of the 2017 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, 6–8 April 2017; pp. 1000–1003.
  12. Rosten, E.; Porter, R.; Drummond, T. Faster and Better: A Machine Learning Approach to Corner Detection. IEEE Trans. Pattern Anal. Mach. Intell. 2010, 32, 105–119.
  13. Apostolopoulos, I.D.; Tzani, M.; Aznaouridis, S.I. A General Machine Learning Model for Assessing Fruit Quality Using Deep Image Features. AI 2023, 4, 812–830.
  14. Nayak, J.; Kaje, S.B. Fast Image Convolution and Pattern Recognition using Vedic Mathematics on Field Programmable Gate Arrays (FPGAs). In Proceedings of the 2022 OITS International Conference on Information Technology (OCIT), Bhubaneswar, India, 14–16 December 2022; pp. 569–573.
  15. Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998, 86, 2278–2324.
  16. Kim, Y.H.; An, G.J.; Sunwoo, M.H. CASA: A Convolution Accelerator using Skip Algorithm for Deep Neural Network. In Proceedings of the 2019 IEEE International Symposium on Circuits and Systems (ISCAS), Sapporo, Japan, 26–29 May 2019; pp. 1–5.
  17. Hu, A.; Li, W.; Lv, D.; He, G. An Efficient Stochastic Convolution Accelerator Based on Pseudo-Sobol Sequences. In Proceedings of the 17th ACM International Symposium on Nanoscale Architectures, NANOARCH ’22, Virtual, OR, USA, 7–9 December 2022.
  18. Yoon, H.; Kim, C.; Um, S.; Yoon, H.W.; Kang, H.G. SC-CNN: Effective Speaker Conditioning Method for Zero-Shot Multi-Speaker Text-to-Speech Systems. IEEE Signal Process. Lett. 2023, 30, 593–597.
  19. Andri, R.; Karunaratne, G.; Cavigelli, L.; Benini, L. ChewBaccaNN: A Flexible 223 TOPS/W BNN Accelerator. In Proceedings of the 2021 IEEE International Symposium on Circuits and Systems (ISCAS), Daegu, Republic of Korea, 22–28 May 2021; pp. 1–5.
  20. Guo, P.; Ma, H.; Chen, R.; Li, P.; Xie, S.; Wang, D. FBNA: A Fully Binarized Neural Network Accelerator. In Proceedings of the 2018 28th International Conference on Field Programmable Logic and Applications (FPL), Dublin, Ireland, 27–31 August 2018; pp. 51–513.
  21. Tsai, T.H.; Lin, D.B. An On-Chip Fully Connected Neural Network Training Hardware Accelerator Based on Brain Float Point and Sparsity Awareness. IEEE Open J. Circuits Syst. 2023, 4, 85–98.
  22. Chen, S.; Sun, W.; Huang, L.; Yang, X.; Huang, J. Compressing Fully Connected Layers using Kronecker Tensor Decomposition. In Proceedings of the 2019 IEEE 7th International Conference on Computer Science and Network Technology (ICCSNT), Dalian, China, 19–20 October 2019; pp. 308–312.
  23. Naronglerdrit, P. Facial Expression Recognition: A Comparison of Bottleneck Feature Extraction. In Proceedings of the 2019 Twelfth International Conference on Ubi-Media Computing (Ubi-Media), Bali, Indonesia, 5–8 August 2019; pp. 164–167.
  24. Montgomerie-Corcoran, A.; Savvas-Bouganis, C. DEF: Differential Encoding of Featuremaps for Low Power Convolutional Neural Network Accelerators. In Proceedings of the 2021 26th Asia and South Pacific Design Automation Conference (ASP-DAC), Tokyo, Japan, 18–21 January 2021; pp. 703–708.
  25. Jeong, Y.; Oh, H.W.; Kim, S.; Lee, S.E. An Edge AI Device based Intelligent Transportation System. J. Inf. Commun. Converg. Eng. 2022, 20, 166–173.
  26. Davur, Y.J.; Kämper, W.; Khoshelham, K.; Trueman, S.J.; Bai, S.H. Estimating the Ripeness of Hass Avocado Fruit Using Deep Learning with Hyperspectral Imaging. Horticulturae 2023, 9, 599.
  27. Yumang, A.N.; Rubia, D.C.; Yu, K.P.G. Determining the Ripeness of Edible Fruits using YOLO and the OVA Heuristic Model. In Proceedings of the 2022 IEEE 14th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Boracay Island, Philippines, 1–4 December 2022; pp. 1–6.
  28. Maharshi, V.; Sharma, S.; Prajesh, R.; Das, S.; Agarwal, A.; Mitra, B. A Novel Sensor for Fruit Ripeness Estimation Using Lithography Free Approach. IEEE Sens. J. 2022, 22, 22192–22199.
  29. Chai, J.J.K.; Xu, J.L.; O’Sullivan, C. Real-Time Detection of Strawberry Ripeness Using Augmented Reality and Deep Learning. Sensors 2023, 23, 7639.
  30. Kim, S.J.; Jeong, S.; Kim, H.; Jeong, S.; Yun, G.Y.; Park, K. Detecting Ripeness of Strawberry and Coordinates of Strawberry Stalk using Deep Learning. In Proceedings of the 2022 Thirteenth International Conference on Ubiquitous and Future Networks (ICUFN), Barcelona, Spain, 5–8 July 2022; pp. 454–458.
  31. Anraeni, S.; Indra, D.; Adirahmadi, D.; Pomalingo, S.; Sugiarti; Mansyur, S.H. Strawberry Ripeness Identification Using Feature Extraction of RGB and K-Nearest Neighbor. In Proceedings of the 2021 3rd East Indonesia Conference on Computer and Information Technology (EIConCIT), Surabaya, Indonesia, 9–11 April 2021; pp. 395–398.
  32. Indrabayu, I.; Arifin, N.; Areni, I.S. Strawberry Ripeness Classification System Based on Skin Tone Color using Multi-Class Support Vector Machine. In Proceedings of the 2019 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia, 24–25 July 2019; pp. 191–195.
  33. Ilyas, T.; Kim, H. A Deep Learning Based Approach for Strawberry Yield Prediction via Semantic Graphics. In Proceedings of the 2021 21st International Conference on Control, Automation and Systems (ICCAS), Jeju, Republic of Korea, 12–15 October 2021; pp. 1835–1841.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
Upload a video for this entry
Information
Subjects: Computer Science, Artificial Intelligence
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :
Joungmin Park
,
Jinyoung Shin
,
Raehyeong Kim
,
Seongmo An
,
Sangho Lee
,
Jinyeol Kim
,
Jongwon Oh
,
Youngwoo Jeong
,
Soohee Kim
,
Yue Ri Jeong
,
Seung Eun Lee
View Times: 156
Revisions: 2 times (View History)
Update Date: 07 Feb 2024
Table of Contents
    1000/1000
    Hot Most Recent
    Video Production Service
    Feedback