Artificial Intelligence in the Diagnosis of Dairy Cows: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Jessie Wu and Version 1 by Danail Brezov.

TWithe paper proposes an approach for estimating the rectal temperature of dairy cows based on the non-invasive real-time monitoring of their respiration rates and the temperature-humidity index (THI) of the environment, combined with the analysis of i the rapid growth of computational power and data transfer capabilities, machine learning (ML) and artificial intelligence (AI) are also making inroads into animal husbandry and veterinarian research. In particular, Infrared images. We use multimodal machine learning for the joint processing (fusion) of these different typethermography (IRT) is being increasingly used for health monitoring and the diagnosis of data. The implementation is  performed using a new open source AutoML Python module named AutoGluon. After training and optimizing three different regression models (a neural network and two powerful boosting algorithms), it reduces the variance of the result using level 2 stacking. The evaluation metrics we work with are the mean absolute error, MAE, and the coefficient of determination, R2. Firy cows, especially in studies related to heat stress, which causes severe losses, helping us analyze its effects on nutrition, milk production, reproduction, etc. There is plenty of evidence for the potential benefits of using IRT for monitoring udder health status in dairy cows and for the early detection orf a sample of 295 studied animals, a weighted ensemble provides quite decent results: R2 = 0.73 mastitis. Its role in detecting hoof lesions and MAE = 0.1oC. For lamenother sample of 118 cows, we additionally use the pulse rate as a predictor and we achieve R2ess has also been reported. The growth of the population and the increase of =quality 0.65, MAE = 0.2oC. The mstaximndal error is almost 1oCrds has set a dreque to outliers, but the median absolute error in both cases is significantly lower: MedAE < 0.1oC,irement for the production of more and better quality food. witTh the standard deviations respectively being  0.118oCe capabilities and potential benefits of and  0.137oC. IRThese encouraging results give us confidence that tabular and visual data fusion in ML models has great potential for the advancement of non-invasive real-time monitoring and early diagnostics methodsmake systems for the automatic collection and processing of thermographic information and decision-making particularly important.

  • infrared thermography
  • multimodal machine learning
  • heat stress
  • smart farming

1. Introduction

When the health condition changes, the amount of heat produced in the body usually responds. The blood flow carries some of this heat to the surface layers of the tissues, where it is released into the surrounding space through the processes of heat exchange. Its realization, however, depends strongly on the temperature and humidity characteristics of the surrounding environment. Therefore, one complex indicator is introduced to characterize the environment, the so-called temperature-humidity index (THI). Nowadays, there are many devices that measure this indicator. In order for normal heat exchange to take place, THI must be within some permissible limits. Beyond these limits, the normal heat exchange with the surrounding environment is disturbed and the body temperature cannot be considered an objective indicator for the health condition anymore. In addition, researchers consider two more basic parameters which provide information about the physiological state of dairy cows; namely, the pulse and the respiration rate. The problem that wresearchers focus on in the present research is in estimating the unknown rectal temperature of the animals based solely on these data and infrared images, without any intrusive procedures. The paper proposes the use of machine learning for the joint processing of multimodal data. Three different models are used to estimate rectal temperatures. Each one of them receives a plausibility score, determining its weight in the final prediction. Infrared thermography (IRT) allows for monitoring of the surface temperature distribution. The animals are periodically photographed without disturbing their daily routine. These IR images provide uresearchers with additional input data for ourresearcher's regression models. Its non-invasiveness makes it both harmless to the animal and cost-effective to the farmer. In this way, it allows for the continuous monitoring of physiological parameters and using the gathered data for scientific research. It also enables working from a sufficient distance, without introducing additional stress to the animal. These qualities make it a promising tool in veterinary medicine research [1,2,3,4,5,6,7][1][2][3][4][5][6][7].

2. Infrared Thermography and Machine Learning in Animal Farms

With the rapid growth of computational power and data transfer capabilities, machine learning (ML) and artificial intelligence (AI) are also making inroads into animal husbandry and veterinarian research. In particular, IRT is being increasingly used for health monitoring and the diagnosis of dairy cows, especially in studies related to heat stress, which causes severe losses, helping uresearchers analyze its effects on nutrition, milk production, reproduction, etc. There is plenty of evidence for the potential benefits of using IRT for monitoring udder health status in dairy cows and for the early detection of mastitis [8,9,10,11,12,13,14,15,16][8][9][10][11][12][13][14][15][16]. Its role in detecting hoof lesions and lameness has also been reported [17,18,19][17][18][19]. The growth of the population and the increase of quality standards has set a requirement for the production of more and better quality food. The capabilities and potential benefits of IRT make systems for the automatic collection and processing of thermographic information and decision-making particularly important [20]. Some studies use computer vision technology and IRT to detect hoof diseases in dairy cows (see [21]). Others resort on deep learning methods, and more precisely, convolutional neural networks (CNNs), for detecting mastitis in dairy cows based on infrared images [22,23,24][22][23][24]. The possibility of using a combination of visible and infrared imaging has also been explored; for instance, to automate the measurement of pig skin surface temperature, emphasizing once more on the benefits of non-invasive methods [25]. The expansion of ML and AI into modern farming practices is facilitated by the advancement of increasingly more precise and affordable sensors reacting to subtle changes in the environment and the animals’ physiological parameters. This provides new opportunities for data analysis and ML-based optimization, which is the leading idea of ’smart farming’. There are many directions for development of this modern research field, but researchers shall focus on non-invasive diagnostics, allowing for precise real-time monitoring of the animals’ health condition without disturbing their comfort and exposing them to unnecessary stress. Body temperature is one of the most important physiological parameters to keep track of, and there are abundant data for it. Hence, multi-regression models predicting the rectal temperature are quite promising for the improvement of health conditions in animal farms and their overall optimization. Although the research area is relatively new, such studies have been conducted before, using different methods [26,27,28,29,30,31,32][26][27][28][29][30][31][32] and technologies [33[33][34],34], leading to different conclusions. Researchers rely a lot on thermal images, like in [35[35][36],36], but researchers feed them into a multimodal algorithm along with tabular features, which improves the accuracy significantly. These ML models have become popular only in recent years due to their complexity, but with modern AutoML tools, they are already a standard task for data scientists [37,38][37][38]. The development of algorithms for processing information from multiple sensors began in the late 1980s, when the field of data fusion emerged [39]. Its aims to increase the accuracy of the assessment and reliability of the information system in case one of its sensors fails, to obtain a more accurate picture of the observed event or process, and to expand the area of observation. Multi-model approaches have become necessary in cases where the systems under evaluation may have different modes of operation (changing dynamics), each corresponding to a different model. The main advantage of this approach is its extreme robustness, which naturally comes at the cost of a higher computing load due to processing several models simultaneously. In the great variety of multi-model approaches [40], the algorithm of interacting multiple models [41] has emerged as one of the most successful. Stacking methods emerging in the 1990s; however, they became more popular in machine learning algorithms for various reasons (see [42,43][42][43]). They rely on weighted ensembles of models, rather than alternative choices. In this way, the imperfections of each individual ML algorithm tend to cancel out to some extend, and the overall variance of the result is reduced. There is plenty of research on the applications of artificial intelligence in improving health conditions and quality of life in animal farms [44], thus optimizing the food production and minimizing the cost. Some authors study the behavioral response, while others focus on the early detection of diseases or environmental monitoring [45]. The data are collected with interconnected sensors, and advanced algorithms are used for processing and analysis. Remote monitoring systems, combined with recognition and identification techniques powered by AI algorithms, are aimed at obtaining better assessments of crucial physiological parameters and characteristics of the environment, in particular, discovering deviations in the health status [46]. Food production is increasingly demanding and leads to new fields of applied science, such as precision agriculture and intelligent animal husbandry [47]. Global changes in the climate and extreme weather conditions also contribute to the stress in farmed animals. This has negative impacts on the quality and quantity of milk produced in farms, as recent studies reveal [48]. The digitalization of agriculture and the advent of modern technology facilitate the expansion of AI into animal husbandry. Cattle identification methods no longer rely just on radio frequencies, but uses image recognition techniques as well. Behavioral research applications are being created as an aid for disease prevention and the early detection of health issues, as well as may other diverse farming applications [49].

References

  1. Lahiri, B.B.; Bagavathiappan, S.; Jayakumar, T.; Philip, J. Medical Applications of Infrared Thermography: A Review. Infrared Phys. Technol. 2012, 55, 221–235.
  2. Vainer, B.G.; Morozov, V.V. Infrared Thermography-Based Biophotonics: Integrated Diagnostic Technique for Systemic Reaction Monitoring. Phys. Procedia 2017, 86, 81–85.
  3. Schlessinger, M. Infrared Technology Fundamentals; Routledge: London, UK, 2019; ISBN 9780203750834.
  4. Ring, F.J.; Ng, E.Y.K.; Diakides, M.; Bronzino, J.; Peterson, D. Infrared Thermal Imaging Standards for Human Fever Detection. In Medical Infrared Imaging: Principles and Practices; CRC Press: Boca Raton, FL, USA, 2012; pp. 1–5.
  5. Miller, J.L. Principles of Infrared Technology; Springer: New York, NY, USA, 2012; ISBN 9781461576662.
  6. Soroko, M.; Howell, K. Infrared Thermography: Current Applications in Equine Medicine. J. Equine Vet. Sci. 2018, 60, 90–96.e2.
  7. Bansi, D. Utilization of Infrared Thermography in Cattle Production and Its Application Potency in Indonesia. WARTAZOA. Indones. Bull. Anim. Vet. Sci. 2018, 28, 99–106.
  8. Colak, A.; Polat, B.; Okumus, Z.; Kaya, M.; Yanmaz, L.E.; Hayirli, A. Short Communication: Early Detection of Mastitis Using Infrared Thermography in Dairy Cows. J. Dairy Sci. 2008, 91, 4244–4248.
  9. Porcionato, M.A.F.; Canata, T.F.; Oliveira, C.E.L.D.E.; Santos, M.V.D.O.S. Udder thermography of gyr cows for subclinical mastitis detection. Rev. Bras. Eng. Biossistemas 2009, 3, 251–257.
  10. Polat, B.; Colak, A.; Cengiz, M.; Yanmaz, L.E.; Oral, H.; Bastan, A.; Kaya, S.; Hayirli, A. Sensitivity and Specificity of Infrared Thermography in Detection of Subclinical Mastitis in Dairy Cows. J. Dairy Sci. 2010, 93, 3525–3532.
  11. Sathiyabarathi, M.; Jeyakumar, S.; Manimaran, A.; Jayaprakash, G.; Pushpadass, H.A.; Sivaram, M.; Ramesha, K.P.; Das, D.N.; Kataktalware, M.A.; Prakash, M.A.; et al. Infrared Thermography: A Potential Noninvasive Tool to Monitor Udder Health Status in Dairy Cows. Vet. World 2016, 9, 1075–1081.
  12. Zaninelli, M.; Redaelli, V.; Luzi, F.; Bronzo, V.; Mitchell, M.; Dell’Orto, V.; Bontempo, V.; Cattaneo, D.; Savoini, G. First Evaluation of Infrared Thermography as a Tool for the Monitoring of Udder Health Status in Farms of Dairy Cows. Sensors 2018, 18, 862.
  13. Berry, R.J.; Kennedy, A.D.; Scott, S.L.; Kyle, B.L.; Schaefer, A.L. Daily Variation in the Udder Surface Temperature of Dairy Cows Measured by Infrared Thermography: Potential for Mastitis Detection. Can. J. Anim. Sci. 2003, 83, 687–693.
  14. Sinha, R.; Bhakat, M.; Mohanty, T.K.; Ranjan, A.; Kumar, R.; Lone, S.A.; Rahim, A.; Paray, A.R.; Khosla, K.; Danish, Z. Infrared Thermography as Non-Invasive Technique for Early Detection of Mastitis in Dairy Animals—A Review. Asian J. Dairy Food Res. 2018, 37, 1–6.
  15. Pamparienė, I.; Veikutis, V.; Oberauskas, V.; Žymantienė, J.; Želvytė, R.; Stankevičius, A.; Marčiulionytė, D.; Palevičius, P. Thermography Based Inflammation Monitoring of Udder State in Dairy Cows: Sensitivity and Diagnostic Priorities Comparing with Routine California Mastitis Test. J. Vibroeng. 2016, 18, 511–521.
  16. Hovinen, M.; Siivonen, J.; Taponen, S.; Hänninen, L.; Pastell, M.; Aisla, A.-M.; Pyörälä, S. Detection of Clinical Mastitis with the Help of a Thermal Camera. J. Dairy Sci. 2008, 91, 4592–4598.
  17. Alsaaod, M.; Büscher, W. Detection of Hoof Lesions Using Digital Infrared Thermography in Dairy Cows. J. Dairy Sci. 2012, 95, 735–742.
  18. Amezcua, R.; Walsh, S.; Luimes, P.H.; Friendship, R.M. Infrared Thermography to Evaluate Lameness in Pregnant Sows. Can. Vet. J. 2014, 55, 268–272.
  19. Novotna, I.; Langova, L.; Havlicek, Z. Risk Factors and Detection of Lameness Using Infrared Thermography in Dairy Cows—A Review. Ann. Anim. Sci. 2019, 19, 563–578.
  20. Koltes, J.E.; Koltes, D.A.; Mote, B.E.; Tucker, J.; Hubbell, D.S., 3rd. Automated Collection of Heat Stress Data in Livestock: New Technologies and Opportunities. Transl. Anim. Sci. 2018, 2, 319–323.
  21. Kříž, P.; Horčičková, M.; Bumbálek, R.; Bartoš, P.; Smutný, L.; Stehlík, R.; Zoubek, T.; Černý, P.; Vochozka, V.; Kuneš, R. Application of the Machine Vision Technology and Infrared Thermography to the Detection of Hoof Diseases in Dairy Cows: A Review. Appl. Sci. 2021, 11, 11045.
  22. Wang, Y.; Kang, X.; He, Z.; Feng, Y.; Liu, G. Accurate Detection of Dairy Cow Mastitis with Deep Learning Technology: A New and Comprehensive Detection Method Based on Infrared Thermal Images. Animal 2022, 16, 100646.
  23. Çevik, K.K.; Boğa, M. Body Condition Score (BCS) Segmentation and Classification in Dairy Cows Using R-CNN Deep Learning Architecture. Eur. J. Sci. Technol. 2019, 1248–1255.
  24. Bochtis, D.D.; Moshou, D.E.; Vasileiadis, G.; Balafoutis, A.; Pardalos, P.M. Information and Communication Technologies for Agriculture—Theme II: Data; Springer Nature: Berlin/Heidelberg, Germany, 2022; ISBN 9783030841485.
  25. Xie, Q.; Wu, M.; Yang, M.; Bao, J.; Zheng, P. A Deep Learning-Based Fusion Method of Infrared Thermography and Visible Image for Pig Body Temperature Detection. In Proceedings of the International Symposium on Animal Environment and Welfare, Chongqing, China, 20–23 October 2021; pp. 326–333.
  26. Manullang, M.C.T.; Lin, Y.-H.; Lai, S.-J.; Chou, N.-K. Implementation of Thermal Camera for Non-Contact Physiological Measurement: A Systematic Review. Sensors 2021, 21, 7777.
  27. Oliveira, R.F.d.; Ferreira, R.A.; Abreu, L.H.P.; Yanagi Júnior, T.; Lourençoni, D. Estimation of respiratory frequency and rectal temperature on pigs in heat stress by fuzzy logic. Eng. Agrícola 2018, 38, 457–470.
  28. Li, G.; Chen, S.; Chen, J.; Peng, D.; Gu, X. Predicting Rectal Temperature and Respiration Rate Responses in Lactating Dairy Cows Exposed to Heat Stress. J. Dairy Sci. 2020, 103, 5466–5484.
  29. Neves, S.F.; Silva, M.C.F.; Miranda, J.M.; Stilwell, G.; Cortez, P.P. Predictive Models of Dairy Cow Thermal State: A Review from a Technological Perspective. Vet. Sci. 2022, 9, 416.
  30. Theusme, C.; Avendaño-Reyes, L.; Macías-Cruz, U.; Castañeda-Bustos, V.; García-Cueto, R.; Vicente-Pérez, R.; Mellado, M.; Meza-Herrera, C.; Vargas-Villamil, L. Prediction of Rectal Temperature in Holstein Heifers Using Infrared Thermography, Respiration Frequency, and Climatic Variables. Int. J. Biometeorol. 2022, 66, 2489–2500.
  31. Kulaz, E.; Ser, G. A Meta-Analysis of Heat Stress in Dairy Cattle: The Increase in Temperature Humidity Index Affects Both Milk Yield and Some Physiological Parameters. Czech J. Anim. Sci. 2022, 67, 209–217.
  32. Jorquera-Chavez, M.; Fuentes, S.; Dunshea, F.R.; Warner, R.D.; Poblete, T.; Jongman, E.C. Modelling and Validation of Computer Vision Techniques to Assess Heart Rate, Eye Temperature, Ear-Base Temperature and Respiration Rate in Cattle. Animals 2019, 9, 1089.
  33. Bleul, U.; Hässig, M.; Kluser, F. Screening of febrile cows using a small handheld infrared thermography device. Tierärztliche Prax. Ausg. G Großtiere Nutztiere 2021, 49, 12–20.
  34. Lewis Baida, B.E.; Swinbourne, A.M.; Barwick, J.; Leu, S.T.; van Wettere, W.H.E.J. Technologies for the Automated Collection of Heat Stress Data in Sheep. Anim. Biotelemetry 2021, 9, 4.
  35. Hennessey, E.; DiFazio, M.; Hennessey, R.; Cassel, N. Artificial Intelligence in Veterinary Diagnostic Imaging: A Literature Review. Vet. Radiol. Ultrasound 2022, 63 (Suppl. 1), 851–870.
  36. Lazri, Z.M.; Zhu, Q.; Chen, M.; Wu, M.; Wang, Q. Detecting Essential Landmarks Directly in Thermal Images for Remote Body Temperature and Respiratory Rate Measurement with a Two-Phase System. IEEE Access 2022, 10, 39080–39094.
  37. Erickson, N.; Shi, X.; Sharpnack, J.; Smola, A. Multimodal AutoML for Image, Text and Tabular Data. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, 14–18 August 2022; Association for Computing Machinery: New York, NY, USA, 2022; pp. 4786–4787.
  38. Erickson, N.; Mueller, J.W.; Shirkov, A.; Zhang, H.; Larroy, P.; Li, M.; Smola, A. AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data. arXiv 2020, arXiv:2003.06505.
  39. Aluja-Banet, T.; Daunis-I-Estadella, J.; Brunsó, N.; Mompart-Penina, A. Improving Prevalence Estimation through Data Fusion: Methods and Validation. BMC Med. Inform. Decis. Mak. 2015, 15, 49.
  40. Mazor, E.; Averbuch, A.; Bar-Shalom, Y.; Dayan, J. Interacting multiple model methods in target tracking: A survey. IEEE Trans. Aerosp. Electron. Syst. 1998, 34, 103–123.
  41. Li, X.R. Engineers’ Guide to Variable-Structure Multiple-Model Estimation and Tracking. Multitarg.-Multisens. Track. Appl. Adv. 2000, 3, 499–567.
  42. Wolpert, D.H. Stacked Generalization. Neural Netw. 1992, 5, 241–259.
  43. Caruana, R.; Niculescu-Mizil, A.; Crew, G.; Ksikes, A. Ensemble Selection from Libraries of Models. In Proceedings of the Twenty-First International Conference on Machine Learning, Banff, AB, Canada, 4–8 July 2004; Association for Computing Machinery: New York, NY, USA, 2004; p. 18.
  44. Bao, J.; Xie, Q. Artificial Intelligence in Animal Farming: A Systematic Literature Review. J. Clean. Prod. 2022, 331, 129956.
  45. Selvaraju, V.; Spicher, N.; Wang, J.; Ganapathy, N.; Warnecke, J.M.; Leonhardt, S.; Swaminathan, R.; Deserno, T.M. Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review. Sensors 2022, 22, 4097.
  46. Fuentes, S.; Gonzalez Viejo, C.; Tongson, E.; Dunshea, F.R. The Livestock Farming Digital Transformation: Implementation of New and Emerging Technologies Using Artificial Intelligence. Anim. Health Res. Rev. 2022, 23, 59–71.
  47. Wang, S.; Jiang, H.; Qiao, Y.; Jiang, S.; Lin, H.; Sun, Q. The Research Progress of Vision-Based Artificial Intelligence in Smart Pig Farming. Sensors 2022, 22, 6541.
  48. Fuentes, S.; Gonzalez Viejo, C.; Cullen, B.; Tongson, E.; Chauhan, S.S.; Dunshea, F.R. Artificial Intelligence Applied to a Robotic Dairy Farm to Model Milk Productivity and Quality based on Cow Data and Daily Environmental Parameters. Sensors 2020, 20, 2975.
  49. De Vries, A.; Bliznyuk, N.; Pinedo, P. Invited Review: Examples and Opportunities for Artificial Intelligence (AI) in Dairy Farms. Appl. Anim. Sci. 2023, 39, 14–22.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback