Computer Vision Technology for Physical Exercise Monitoring

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Salik Ram Khanal	--	4424	2023-02-03 19:22:21	\|
2	format	Jason Zhu	Meta information modification	4424	2023-02-06 06:42:59	\|

This entry is adapted from the peer-reviewed paper 10.3390/a15120444

Physical activity is movement of the body or part of the body to make the muscles more active and to lose the energy from the body. Regular physical activity in the daily routine is very important to maintain good physical and mental health. It can be performed at home, a rehabilitation center, gym, etc., with a regular monitoring system. How long and which physical activity is essential for specific people is very important to know because it depends on age, sex, time, people that have specific diseases, etc. Therefore, it is essential to monitor physical activity either at a physical activity center or even at home. Physiological parameter monitoring using contact sensor technology has been practiced for a long time.

exercise monitoring physical activity vital sign monitoring computer vision

1. Introduction

Physical exercise is an important part of the daily routine that helps to overcome the risk of diseases as well as improve quality of life. Physical exercise is essential to prevent and reduce the risk of many diseases and improve physical and mental health ^[1]. People are quite busy in modern times and this trend is increasing day by day, therefore, this is a time to think about the appropriate technology for monitoring their activities and caring for their elderly ^[2]. An intelligent interface between the human body and the monitoring mechanism could solve this type of problem. In the recent years, many technologies have been implemented for extracting and collecting physiological data, such as heart rate, blood pressure, temperature, etc., using various types of sensors before, during, and after physical exercise ^[3]^[4]^[5], however, these techniques are not adequate for regular monitoring. An intelligent model using image processing techniques is required to measure physical exercise intensity during physical exercise.

In recent years, many studies have proposed various methodologies to extract various physiological data and monitor physical exercise. Mainly, two approaches (contact sensor technology and contactless technology) are widely used in recent times to extract physiological features whether during physical exercise or resting. Both the approaches have pros/cons, however, the proposed implementation is parallelly used. In contact sensor techniques, the sensors are attached on the body and the result is interpreted using machine learning or statistical approaches. For non-contact/contact sensor techniques, various types of cameras are used to capture video/images and processed with various image processing techniques including Infrared camera and thermal camera. Whatever the image extraction technique, they must be followed by computer vision or machine learning and deep learning techniques to interpret the result. The facial expression in different states of tiredness may depend on the age of people or may be different for male and female users. The intelligence interface should be more useful for elderly people who cannot control a physical activity machine themselves during physical activities. They do not even want to use a wearable device on the body; in this situation, this model gives a better way of monitoring.

2. Heart Rate Extraction

Heart rate (HR) is the rate at which the heart muscle veins perform contraction and expansion to pump and absorb the blood from various parts of the body. It is an important indicator that reflects the level of exertion during exercise ^[6]^[7]. There are several non-contact techniques to measure HR at rest or during physical exercise while avoiding the motion artifact.

2.1. Doppler-Based System

This is one of the most famous signal extraction techniques from an object using radar signal. This technology has been used to extract the signals from the human body to monitor physiological parameters. Knobloch ^[8] evaluated the relationship between breath-by-breath oxygen uptake (VO2), cardia index (CI), stock volume (SU) and HR determined by a non-invasive doppler-based system in athletes during exercise. Authors co-related VO2 with CI, SU, and HR in various stages of exercise such as peak, 1 min recovery, and 3 min recovery. A similar work was presented by Ebrahim ^[9] (Ebrahim et al., 2017) to monitor breathing rate and heart rate using the RADAR based Doppler system during standing and walking positions. For this purpose, five tests had been performed with different distances of the human body from antenna, different angles or positions of antenna and body, and standing and walking positions. The maximum distance covered was 150 cm. The signal acquired from the antenna is followed by various signal processing steps including low pass filter, band pass filter, etc., and finally that signal was input to oscilloscope.

2.2. Near-Infrared Spectroscopy

Near-infrared spectroscopy (NIR) is a widespread non-invasive technology to evaluate muscle oxidative metabolism including peripheral and cardiorespiratory measurement. It is an optical device that transmits optical signals in a tissue and receives the reflection of light ^[10]. The reflection of light intensity might be affected by the amount of hemoglobin in the blood. Xu ^[11] employed a near-infrared spectroscopy (NIRS) muscle oxygen monitor system to measure the relative change in muscle oxygenation, with the heart rate, oxygen uptake, production of carbon dioxide (VCO2), and respiratory exchange ratio. The results showed a high correlation between relative change in oxy-hemoglobin concentration and heart rate. The experiments were performed on a treadmill with an incremental protocol. Astaras ^[12] proposed an image sensing device using NIR, which acquires ECG signals and after a series of signal processing, the signal can be classified into four classes. Resting, walking, running, and lying were the four classes that can classify according to the ECG signal. They extracted HR, breathing rate, etc.

2.3. Photoplethysmography

Photoplethysmography (PPG) is a simple, low-cost and noninvasive optical bio- monitoring technique used for a few decades to monitor the blood volume changes that occur in the human body vessels due to the blood circulation system. This technology consists of one light emitter and one light detector. With advances in the PPG-based physiological parameter monitoring system, the contact based PPG was extended to a contactless system using images ^[13]. It is not only applied for heart rate measurement, but also for measuring blood oxygen saturation and respiratory rate ^[14]. The implementation of such a system is widespread using various types of image capturing devices such as mobile phone cameras, webcams, and thermal cameras ^[14]^[15]^[16]^[17]. A common way to capture images is using Opto-Physiological Imaging (OPI) with a Complementary Metal-Oxide Semiconductor (CMOS) camera and webcam. Various research directions (scientific camera-based and webcam-based imaging) have been applied to monitor heart rate and blood pressure during rest, exercise, and recovery using PPG. However, only a few studies from the large amount of literature presenting PPG to measure heart rate ^[18]^[19]^[20]^[21] focus on HR monitoring during exercise ^[22]^[23].

One of the well-known devices to monitor exercise using PPG technology is the Microsoft Kinect sensor, which can sense the motion using image processing techniques on photoplethysmography images captured by the RGB camera mounted in it. The Kinect RGB camera detects changes in blood flow by mixing PPG signals and volumetric changes in the vessels. For a decade, these devices have been heavily used for monitoring parameters during exercise and rest ^[22]^[24]^[25]. Bosi ^[22] proposed a study using the Microsoft Kinects to detect HR at rest and exercise. From the photoplethysmography images captured by the Microsoft Kinect RGB camera, the average color intensity of each color channel of the RGB region of interest (ROI) selected from the forehead was processed through a bandpass digital filter ^[26], Independent Component Analysis (ICA) ^[27], and Principle Component Analysis (PCA) ^[28]^[29]. The spectral analysis using Fast Fourier Transform (FFT) of the principal components was applied. The results were compared with a pulse oximeter attached in the body simultaneously to the motion sensing device. A good correlation was found with the oxygen values in the rest situation, but not for the exercise situation, where participants performed rehabilitation exercises.

Remote PPG have also been proposed to improve heart rate measurement during physical exercise ^[23]. The main idea behind this version of PPG was to increase the degrees-of-freedom for noise reduction by decomposing the n-wavelength camera signals into multiple orthogonal frequency bands and extracting the pulse-signal per band. The videos of seven subjects were recorded in various practical challenges such as different skin colors of subjects, various light sources, illumination intensities, and exercising modes. The authors suggested that the proposed method increases the robustness of HR measurement in various fitness applications. Other research presented estimations body surface temperature using an Infrared (IR) camera during rest and various stages of exercise (sustained exercise at 50%, 60%, and 70% of age predicted maximum heart rate). HR and Respiratory Rate (RR) were measured with photoplethysmography and motion analysis (vPPG-MA). The results of body surface temperature using IR, and HR and RR using vPPG-MA showed the high level of agreement between the two measurements ^[30]. Naolean et al. ^[31] presented rPPG based HR estimation during physical exercise by analyzing the facial images. The authors implemented temporal super pixels and a convolutional neural network. The experiments were carried out on video and were recorded from eleven subjects during intense exercise. The root mean square error was 22.01, which was better than the similar studies.

2.4. Video-Based Image Processing

With the rapid advancement in the application of computer vision, researchers have also developed methods for the extraction of reliable heart rate measures ^[32]^[33]^[34]. One of them includes the use of face detection and feature extraction to obtain estimates of heart rate. For this, images or videos are recorded using digital cameras that are placed to frame the person’s face during exercise performance or at rest. The most used technique to detect face during heart rate estimation is the Viola Jones object detection algorithm. The blind-source separation of the digital signal algorithm ^[35] to measure heart rate during rest was then purposed by ^[33].

Digital image color analysis has also been proposed to help in HR estimation, and several digital filters has been proposed to increase the detection accuracy related to the color channels ^[36]^[37]^[38]^[39]. A case-study article introduced the use of ICA of each color channel of RGB in the raw images, followed by the application of short-time Fourier Transform (STFT) ^[39]. Using the proposed methods, root mean square error was less than 2.5 beats per minute (BPM) for heart rate variation between 80 BPM and 130 BPM.

Monkaresi ^[40] proposed the advancement of work proposed by Poh ^[33]^[34] using the application of the kNN algorithm after extracting features by ICA. The experiments were carried out in various intensities of exercise performed on stationary bicycles. Other non-contact techniques to measure heart rate include Blood Volume Pulse (BVP), which detects blood volume in cardiovascular dynamics using optical sensors ^[38]. Cui ^[15] proposed using facial image processing during exercise to measure BVP. The image pre-processing includes face recognition, band-pass filter ^[26], trend removal, and reconstruction of source signals to then retrieve BVP. The authors selected only the green channel for the ROI for further signal processing. This signal is applied to ICA and yields three independent signals. A series of image filters, Discrete Time Fourier Transform (DTFT), Discrete Short Time Fourier Transform (Discrete STFT), Discrete Wavelet Transform (DWT), Peak Counting, and Mean Value of Interbit Intervals (IBI) were applied to obtain the frequency spectrum of the raw signal. Both the articles used the image capture in ambient light, straight pose, and neutral emotion.

The recent development of an exergame connects physical exercise to entertainment, typical of a gaming activity. This could provide solutions to issues regarding treatments of age-related diseases ^[41]. The proper extraction and implementation of physiological features while using exergames is critical. Spinsante ^[42] presented a contactless HR measurement system based on motion-compensated video signals that can be implemented in the remote controllers used for exergames.

A completely new approach was presented by Lin ^[43] that measured heart rate and step count during exercise, detected by facial image processing. The authors implemented a chrominance-based adaptive filter and normalization method that enhanced accuracy. The detection rate was 99.52% and 99.7% for stepping and pulse rate using treadmill exercise, respectively. The estimated heart rate was compared to cardiofrequencimeter data during all the activities, showing positive correlation. The authors concluded that step count and heart rate can be measured synchronously during exercise.

2.5. Facial Expression

The facial expression is an important cue to monitor physical exercise, since people express feelings through the face, and it changes due to exertion ^[44]^[45]. Wu ^[46] proposed a method to estimate Rate of Perceived Expression automatically without any wearable devices and questionnaires. Camera-based heart rate and fatigue expression feature extractors were fused. The Rate of Perceived Exertion (RPE) was considered roughly ten times the heart rate in the calculations, and it was correlated with various approaches such as the PPG and fatigue feature, rPPG and fatigue feature, PPG, and rPPG. The authors reported the high correlation coefficient of RPE and heart rate for the ground truth value of RPE. From the results, it was noted that if the information of facial expressions related to fatigue is considered, the error in the estimation can be reduced.

3. Heart Rate Variability

Heart rate variability (HRV) is the physiological phenomenon of the variation of the time interval between heartbeats. It is an important physiological parameter to be considered to monitor intensity during exercise and many approaches have been proposed to measure HRV using contact and non-contact sensor technologies. ^[13]^[47]^[48]^[49].

A completely new approach was proposed by Hung to measure HRV by continuous monitoring of the Pupil Size Variability (PSV) during physical activity. The authors also measure blood pressure variability using the same approach. It was hypothesized that pupil size variability is highly correlated to both physiological parameters, which are also indicators of exercise intensity. Many parameters including electrocardiogram, respiration effort, finger arterial pressure, and pupil images were recorded from ten subjects before and after five minutes of exercise on a treadmill. The pupil images were captured by a 1/3 Charged Coupled Device (CCD) camera connected to an 8-bit monochrome video frame-grabber, set to capture at a resolution of 512 × 512 pixels and capture speed of 2 frames per second. All the signals, including pupil images, were acquired by a data acquisition unit ^[50] during exercise. The experiments underwent six phases of 5 min recording sessions. The findings suggested that PSV may be a valid indicator of cardiovascular variability.

4. Blood Pressure

Blood pressure level can signpost the effort level when an individual is performing exercise, helping to estimate the workload ^[51]. Blood pressure is also important to determine some patterns that could correspond to cardiac diseases, thus, it is significant for the monitoring of health output during exercise ^[52].

4.1. Photoplethysmography and Pulse Arrival Time

The Pulse Arrival Time (PAT) is the time between the peak of the electrocardiogram (ECG) and the arrival of the pulse detected by the PPG. Fatemeh Shirbani ^[53]^[54] investigated the correlation between image-based photoplethysmography pulse arrival time (iPPG-PAT) and diastolic BP (DBP) during one-minute seated rest and three minutes of isometric handgrip exercise. The video was recorded from the face using a standard web camera and estimates were compared to a ground-truth device. It was found that the beat-to-beat iPPG-PAT and DBP were negatively correlated.

4.2. Photoplethysmography and Pulse Transmit Time

Pulse Transit Time (PTT) is the time a pulse wave takes to travel between two different arterial points, which is an important cue to estimate blood pressure. Several studies were carried out to obtain the correlation between PTT and blood pressure. The authors of ^[55] introduced a contactless approach to estimate blood pressure using PPT, with seven healthy subjects. Image based (iPTT) and image-based PPG were recorded using a high speed camera at 426 Hz during physical exercise in a stationary bicycle. The exercises were carried out at three different times: rest, peak exercise, and recovery. The study found a highly positive correlation between iPTT and iPPG during exercise, concluding that measuring BP using PTT is reliable. It was shown that skin color changes due to blood pulsation and such changes could be identified by the three-color component processing in facial images.

5. Body Temperature

Body temperature rises due to physical exertion and relates to other physiological parameters, therefore, it can also be considered when monitoring physical exercise intensity ^[56]. Thermal imaging is the technology that allows for measuring the body temperature without any contact sensors.

Thermal imaging captures images of an object based on the infrared radiation emanated from it ^[57]^[58]^[59]. Its use in medical and sports environments is widespread ^[60]. However, it seems that body location is important in terms of assessment quality as thermal emission might differ depending on the body part aimed to be analyzed ^[61]. Ludwig ^[60] presented a critical comparison between the main methods used to obtain body temperature from images, and also proposed an alternative. It was found that the temperature obtained within an ROI selection of a well-defined area can be considered as the most reliable.

James ^[62] proposed an approach to investigate the validity and reliability of skin temperature measurement using a Telemetry Thermistor system (TT) and thermal camera during exercise in a hot and humid environment. Another similar study was presented by ^[57] to compare data loggers (skin adhesive), thermal imaging, and wired electrodes for the measurement of skin temperature during exercise in a similar environment. The authors concluded that data logger and thermal imaging can be used as alternative measures for skin temperature in exercising, especially on higher temperatures and humidity.

Another study explored temperature in several different body parts pre, during and after moderate aerobic exercise using infrared thermography, proposed by Fernandes ^[63]. The authors concluded that there are significant distinctions in the skin temperature distribution during exercise depending on the body part.

The changes in body temperature during endurance work-outs in highly trained male sprinters were analyzed by Korman ^[64] using a thermal camera. The aim was to characterize experiments: before the session (pre-exercise), warm-up, specific drills for athletes’ warming up by comparing body temperature in the four phases of the sprinting techniques, and endurance exercise. Significant differences were found between the temperature of athletes’ backs and from body profiles, as well as significant changes before and after exercise. However, the thermography results were not compared to ground-truth measurements of temperature.

6. Energy Expenditure

Energy expenditure is highly correlated to exercise intensity, thus is essential to the planning, prescription, and monitoring of physical exercise programs ^[65]^[66]^[67]. Although its estimation is challenging, the oxygen uptake has been considered the best direct way to measure energy expenditure ^[68]^[69]. Indirectly, it can be performed through heart rate and body acceleration ^[66], which involves the development and application of non-contact technology as presented in the following sections.

6.1. Thermal Imaging

The introduction of thermal imaging in exercise monitoring also allowed for the development of reliable contactless techniques for energy expenditure measurements. Thermal cameras capture infrared radiation in the mid or long-wavelength infrared spectrum, depending on sensor type, emitted from any object. Therefore, the pixel values in the images are converted to temperature values and finally mapped with energy expenditure.

Jenson ^[65] validated the thermal imaging method to estimate energy expenditure using oxygen uptake as a comparative. Fourteen endurance-trained subjects completed an incremental exercise test on a treadmill. Heart rate, gas exchange, and mean accelerations of the ankle, thigh, wrist, and hip were measured throughout the exercise. A linear correlation was found between the energy expenditure calculated using the optical flow of the thermal imaging and the oxygen uptake values. The contactless measurement of energy expenditure during exercise was also presented by Gade ^[68]. The authors used thermal video analysis to automatically extract the cyclic motion pattern in walking and running. The results indicated a linear correlation between the proposed method and oxygen uptake.

6.2. RGB Depth

One of the noticeable technologies used for image capturing is the RBG-Depth camera, which captures objects in three dimensions. Tao ^[67] presented a framework for the estimation of vision-based energy expenditure using a depth camera and validating the method with oxygen uptake measures. The method was found suitable for monitoring in a controlled environment, showing advantages as pose-variant and individual-independent way of measuring energy expenditure, in real time and remotely.

The deep learning technique has been considered one of the best tools to estimate, classify, and analyze quantitative data. Its application in sports has been increasing in recent years. This method is suitable for implementation in controlled environments, where the system first detects the presence of humans and then tracks the human body. This process is followed by a CNN-based feature extraction, then activity recognition and, finally, prediction of the calories produced ^[67].

A novel approach using a fully contactless and automatic method, based on computer vision algorithm, was presented by Koporec ^[66]. The RGB-Depth images are captured using Microsoft Kinect during exercise and a histogram of Oriented Optical Flow (HOOF) descriptors are extracted from the depth images and are used to predict heart rate. It feeds a regression model that finally estimates the energy consumption.

7. Respiratory Rate

Breathing is the process of taking air inside the body so oxygen can be absorbed, and then expelling carbon dioxide out of the body. Physical exertion not only increases the frequency of breathing, but also demands the exchange of a higher volume of air. Respiratory rate is directly linked to exertion, and thus, to energy expenditure as well ^[70]^[71]^[72]^[73]^[74].

7.1. Video-Based Image Processing

The ventilation threshold is an important variable to investigate physical exertion. It is related to the anaerobic threshold, an event characterized by the increase in ventilation at a faster rate than what the body is capable of absorbing. In recent decades, many methods have been proposed to measure the respiratory rate during exercise using contactless technology. Aoki ^[70] proposed a technique to measure respiratory rate during pedal stroke using optical techniques. A dot matrix optical element was arranged in front of the participant’s face and a laser was emitted to be captured by the camera CCD. After using low-pass digital filtering, a sinusoidal wave that vibrates at the respiratory frequency was calculated. The results showed high correlation when compared to data obtained using a gas analyzer.

7.2. RGB Depth

Aoki and colleagues not only proposed and evaluated the use of CCD cameras for respiratory rate measurements but also authored a series of publications exploring new trends in contactless sensors such as Kinect. In 2015, the use of the Kinect camera for respiratory rate measurement was validated ^[47]. RGB depth images were captured and processed to obtain sinusoidal waves during exercise performed in a stationary cycle ergometer. The frequency found in the set of waves represented the respiratory rate. Later, the authors investigated whether the new method could provide good estimates of the ventilation threshold (VT). The experimental setup was maintained, but was applied to an incremental test, specifically to identify this variable. The authors found that respiratory rate measures are possible from increments above 160 W and ventilatory threshold values can be estimated with ±10 W of deviation from the VT calculated by gas analyzer ^[75].

8. Muscle Fatigue

Fatigue is a subjective symptom of malaise and aversion to perform the activity or to objectively impaired performance ^[76]. It can be assessed by either self-report scales or performance-based measures ^[77]. Fatigue can be either physical or mental and both types are important to assess due to its high correlation to health-related parameters ^[78]^[79].

Facial expression is effective in assessing physical and mental fatigue ^[80]^[81]. Irani ^[82] proposed an approach to measure fatigue by tracking facial features during exercise. The main hypothesis of the study was that, towards a fatiguing state, the points of interest in the image would increase vibration, thus, it could be identified in the power spectral analysis of the signal. This model was tested in maximal and submaximal dumbbell lifting tests, against force measures obtained by a dynamometer. The results showed that the temporal point of interest in the face could be easily found using the method.

Deep learning and thermal imaging were fused to automatically detect in the face exercise-induced fatigue ^[83]. Different devices captured RGB, near infra-red, and thermal images, while the pre-trained CNN, Alexnet ^[84], and Visual Geometry Group-16 (VGG16)/VGG19 ^[85] were the deep learning methods used for the classification of different regions in the face according to fatigued/rested state. The authors found that the Alexnet applied to the region around the mouth showed the best classification of the fatiguing state.

9. Other Approaches

9.1. Muscle Oxygenation

Muscles need oxygen supply to work, thus, aerobic muscle performance increases muscle oxygenation. This parameter is related to heart rate and blood pressure during exercise. It is also closely related to muscle fatigue, thus, might bring important information to the research about physical exercise intensity ^[86].

9.2. Facial Expression

The human face is a door for expressing feelings yielded either by physical or mental condition. Pain, tiredness, and illness due to exertion is reflected in facial expressions; therefore, monitoring exercise intensity level by analyzing facial expression might be an interesting idea.

Khanal ^[44]^[87] explored various methods of automatic classification of exercise intensities using computer vision techniques of a subject performing sub-maximal incremental exercise on a cycle ergometer. The facial expression was analyzed by extracting 70 facial feature points. The exercise intensity was classified according to the distance between points and stage of the incremental exercise. The intensity was classified into two, three, and four classes using kNN, Support Vector Machine (SVM), and discriminant analyses. The results showed that facial expression is a good method to identify exercise intensity levels. A regression based facial color analysis to estimate the heart rate at particular instances of time was presented by Khanal et al. ^[87] where the autoregression model is proposed to predict the heart rate from the facial color changes.

A different way of using facial expression to analyze physical effort was presented by Uchida ^[81]. The facial images were analyzed at different levels of resistance training. The authors evaluated the changes in facial expression using Facial Action Coding System (FACS) and the facial muscle activity using surface electromyography. The association of these parameters was mild, however, statistically significant.

Miles ^[88] also presented an analysis of the reliability in tracking data from facial features across incremental exercise on a cycle ergometer. The results differed according to the face parts analyzed, but higher reliability was found for the lower face. A non-linear relationship between facial movement and power output were also determined. The power output, heart rate, RPE, blood lactate, positive and negative effects in corresponding exercise intensity were satisfied in the two blood lactate thresholds and maximum a posteriori probability MAP. These results show the potential in using the tracking of facial features as a non-invasive way of obtaining psychophysiological measures to access exercise intensity.

Still regarding the use of facial features to evaluate levels of exertion, the mouth and eyes are particularly interesting parts that express information by muscle actions. Thus, there is a variety of facial expressions and emotions heavily oriented by the eyes and mouth. Therefore, tracking the movement of these parts could be a key idea to analyze exercise intensity ^[45]. Recently, the eye-blink rate and open-close rate of the mouth were tracked using Viola and Jones algorithm for image processing ^[89] during sub-maximal exercise on a cycle ergometer. The eye-blinking rate was correctly identified with 96% accuracy. Additionally, the higher the exercise intensity, the higher the eye and mouth movement.

References

Paterson, D.H.; Warburton, D.E.R. Physical activity and functional limitations in older adults: A systematic review related to Canada’s Physical Activity Guidelines. Int. J. Behav. Nutr. Phys. Act. 2010, 7, 38.
Borg, R.L. Music In the Education of Children. J. Res. Music. Educ. 1962, 10, 79–80.
Faulkner, J.; Eston, R.G. Percieved exertion research in the 21st century: Developments, reflections and questions for the future. J. Exerc. Sci. Fit. 2008, 6, 1.
Huanga, D.H.; Chioua, W.K.; Chenb, B.H. Judgment of perceived exertion by static and dynamic facial expression. In Proceedings of the 19th Triennial Congress of the IEA, Melbourne, Australia, 9–14 August 2015.
Mei, M.; Leat, S.J. Quantitative assessment of perceived visibility enhancement with image processing for single face images: A preliminary study. Investig. Ophthalmol. Vis. Sci. 2009, 50, 4502–4508.
Prawiro, E.A.P.J.; Hu, C.C.; Chan, Y.S.; Chang, C.H.; Lin, Y.H. A heart rate detection method for low power exercise intensity monitoring device. In Proceedings of the 2014 IEEE International Symposium on Bioelectronics and Bioinformatics (IEEE ISBB 2014), Chung Li, Taiwan, 11–14 April 2014.
Wiles, J.D.A.; Allum, S.R.; Coleman, D.A.; Swaine, I.L. The relationships between exercise intensity, heart rate, and blood pressure during an incremental isometric exercise test. J. Sports Sci. 2008, 26, 155–162.
Knobloch, K.; Hoeltke, V.; Jakob, E.; Vogt, P.M.; Phillips, R. Non-invasive ultrasonic cardiac output monitoring in exercise testing. Int. J. Cardiol. 2008, 126, 445–447.
Pour Ebrahim, M.; Sarvi, M.; Yuce, M.R. A Doppler Radar System for Sensing Physiological Parameters in Walking and Standing Positions. Sensors 2017, 17, 485.
Jöbsis, F.F. Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science 1977, 198, 1264–1267.
Xu, G.; Mao, Z.; Wang, B. Noninvasive detection of gas exchange rate by near infrared spectroscopy. In Proceedings of the Seventh International Conference on Photonics and Imaging in Biology and Medicine, Wuhan, China, 8–10 August 2009.
Astaras, A.; Kokonozi, A.; Michail, E.; Filos, D.; Chouvarda, I.; Grossenbacher, O.; Koller, J.M.; Leopoldo, R.; Porchet, J.A.; Correvon, M.; et al. Pre-clinical physiological data acquisition and testing of the IMAGE sensing device for exercise guidance and real-time monitoring of cardiovascular disease patients. In XII Mediterranean Conference on Medical and Biological Engineering and Computing 2010; Springer: Berlin/Heidelberg, Germany, 2010; pp. 240–243.
Li, K.H.C.; White, F.A.; Tipoe, T.; Liu, T.; Wong, M.C.; Jesuthasan, A.; Baranchuk, A.; Tse, G.; Yan, B.P. The Current State of Mobile Phone Apps for Monitoring Heart Rate, Heart Rate Variability, and Atrial Fibrillation: Narrative Review. JMIR Mhealth Uhealth 2019, 7, 11606.
Kumar, M.; Veeraraghavan, A.; Sabharwal, A. DistancePPG: Robust non-contact vital signs monitoring using a camera. Biomed. Opt. Express 2015, 6, 1565–1588.
Cui, Y.; Fu, C.-H.; Hong, H.; Zhang, Y.; Shu, F. Non-contact time varying heart rate monitoring in exercise by video camera. In Proceedings of the 2015 International Conference on Wireless Communications & Signal Processing (WCSP), Nanjing, China, 15–17 October 2015; pp. 1–5.
Sun, Y.; Hu, S.; Azorin-Peris, V.; Zheng, J.; Greenwald, S.; Chambers, J.; Zhu, Y. Detection of physiological changes after exercise via a remote opto-physiological imaging system. In Proceedings of the Design and Quality for Biomedical Technologies IV, San Francisco, CA, USA, 28–29 January 2011.
Sun, Y.; Papin, C.; Azorin-Peris, V.; Kalawsky, R.; Greenwald, S.; Hu, S. Use of ambient light in remote photoplethysmographic systems: Comparison between a high-performance camera and a low-cost webcam. J. Biomed. Opt. 2012, 17, 3.
Ahmadi, A.K.; Moradi, P.; Malihi, M.; Karimi, S.; Shamsollahi, M.B. Heart Rate monitoring during physical exercise using wrist-type photoplethysmographic (PPG) signals. In Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 25–29 August 2015; pp. 6166–6169.
Fallet, S.; Vesin, J.-M. Robust heart rate estimation using wrist-type photoplethysmographic signals during physical exercise: An approach based on adaptive filtering. Physiol. Meas. 2017, 38, 155–170.
Suriani, N.S.; Jumain, N.A.; Ali, A.A.; Mohd, N.H. Facial Video based Heart Rate Estimation for Physical Exercise. In Proceedings of the 2021 IEEE Symposium on Industrial Electronics & Applications (ISIEA), Shah Alam, Malaysia, 10–11 July 2021; pp. 1–5.
Zhang, Z. Heart rate monitoring from wrist-type photoplethysmographic (PPG) signals during intensive physical exercise. In Proceedings of the 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Atlanta, GA, USA, 3–5 December 2014; pp. 698–702.
Bosi, I.; Cogerino, C.; Bazzani, M. Real-time monitoring of heart rate by processing of Microsoft KinectTM 2.0 generated streams. In Proceedings of the 2016 International Multidisciplinary Conference on Computer and Energy Science (SpliTech), Split, Croatia, 13–15 July 2016.
Wang, W.; den Brinker, A.C.; Stuijk, S.; de Haan, G. Robust heart rate from fitness videos. Physiol. Meas. 2017, 38, 1023–1044.
Gambi, E.; Agostinelli, A.; Belli, A.; Burattini, L.; Cippitelli, E.; Fioretti, S.; Pierleoni, P.; Ricciuti, M.; Sbrollini, A.; Spinsante, S. Heart Rate Detection Using Microsoft Kinect: Validation and Comparison to Wearable Devices. Sensors 2017, 17, 1776.
Xiao, H.; Xu, J.; Hu, D.; Wang, J. Combination of Denoising Algorithms for Video-Based Non-contact Heart Rate Measurement. In Proceedings of the 2022 3rd Information Communication Technologies Conference (ICTC), Nanjing, China, 6–8 May 2022; pp. 141–145.
Mitra, S.; Li, H.; Lin, I.-S.; Yu, T.-H. A new class of nonlinear filters for image enhancement. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Toronto, ON, Canada, 14–17 April 1991; IEEE: New York, NY, USA, 1991; pp. 2525–2526.
Comon, P. Independent component analysis, A new concept? Signal Process. 1994, 36, 287–314.
Flury, B. Common Principal Components & Related Multivariate Models; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 1988.
Jackson, J.E. A User’s Guide to Principal Components; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 1991.
Capraro, G.; Kobayashi, L.; Etebari, C.; Luchette, K.; Mercurio, L.; Merck, D.; Kirenko, I.; van Zon, K.; Bartula, M.; Rocque, M. ‘No Touch’ Vitals: A Pilot Study of Non-contact Vital Signs Acquisition in Exercising Volunteers. In Proceedings of the 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), Cleveland, OH, USA, 17–19 October 2018; pp. 1–4.
Napolean, Y.; Marwade, A.; Tomen, N.; Alkemade, P.; Eijsvogels, T.; van Gemert, J. Heart rate estimation in intense exercise videos. arXiv 2022, arXiv:2208.02509.
Balakrishnan, G.; Durand, F.; Guttag, J. Detecting pulse from head motions in video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 3430–3438.
Poh, M.-Z.; McDuff, D.J.; Picard, R.W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 2010, 18, 10762–10774.
Poh, M.Z.; McDuff, D.J.; Picard, R.W. Advancements in noncontact, multiparameter physiological measurements using webcam. IEEE Trans. Biomed. Eng. 2011, 58, 7–11.
Pal, M.; Roy, R.; Basu, J.; Bepari, M.S. Blind source separation: A review and analysis. In Proceedings of the 2013 International Conference Oriental COCOSDA held jointly with 2013 Conference on Asian Spoken Language Research and Evaluation (O-COCOSDA/CASLRE), Gurgaon, India, 25–27 November 2013.
More, A.V.; Wakankar, A.; Gawande, J.P. Automated heart rate measurement using wavelet analysis of face video sequences. In Innovations in Electronics and Communication Engineering; Saini, H.S., Sing, R.K., Patel, V.M., Santhi, K., Ranganayakulu, S.V., Eds.; Springer: Singapore, 2019; pp. 113–120.
Xie, K.; Fu, C.; Liang, H.; Hong, H.; Zhu, X.; Yang, J. Non-contact Heart Rate Monitoring for Intensive Exercise Based on Singular Spectrum Analysis. Int. J. Distrib. Syst. Technol. 2021, 12, 16–26.
Yang, Z.; Yang, X.; Jin, J.; Wu, X. Motion-resistant heart rate measurement from face videos using patch-based fusion. Signal Image Video Process. 2019, 13, 423–430.
Yu, Y.P.; Kwan, B.H.; Lim, C.L.; Wong, S.L.; Raveendran, P. Video-Based Heart Rate Measurement Using Short-time Fourier Transform. In Proceedings of the 2013 International Symposium on Intelligent Signal Processing and Communications Systems (Ispacs), Naha, Japan, 12–15 November 2013; pp. 704–707.
Monkaresi, H.; Calvo, R.A.; Yan, H. A Machine Learning Approach to Improve Contactless Heart Rate Monitoring Using a Webcam. IEEE J. Biomed. Health Inform. 2014, 18, 1153–1160.
Vojciechowski, A.S.; Natal, J.Z.; Gomes, A.R.S.; Rodrigues, E.V.; Villegas, I.L.P.; Korelo, R.I.G. Effects of exergame training on the health promotion of young adults. Fisioter. Mov. 2017, 30, 59–67.
Spinsante, S.; Ricciuti, M.; Scalise, L. Contactless Measurement of Heart Rate for Exergames Applications. In Proceedings of the 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Rome, Italy, 11–13 June 2018; pp. 1–6.
Lin, Y. Step Count and Pulse Rate Detection Based on the Contactless Image Measurement Method. IEEE Trans. Multimed. 2018, 20, 2223–2231.
Khanal, S.R.; Sampaio, J.; Barroso, J.; Filipe, V. Classification of Physical Exercise Intensity Based on Facial Expression Using Deep Neural Network; Springer: Cham, Switzerland, 2019; Volume 11573 LNCS.
Khanal, S.R.; Fonseca, A.; Marques, A.; Barroso, J.; Filipe, V. Physical exercise intensity monitoring through eye-blink and mouth’s shape analysis. In Proceedings of the 2nd International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW), Thessaloniki, Greece, 20–22 June 2018.
Wu, B.-F.; Lin, C.-H.; Huang, P.-W.; Lin, T.-M.; Chung, M.-L. A contactless sport training monitor based on facial expression and remote PPG. In Proceedings of the 2017 IEEE International Conference on Systems, Man and Cybernetics, Banff, AB, Canada, 5–8 October 2017; pp. 846–852.
Aoki, H.; Nakamura, H.; Fumoto, K.; Nakahara, K.; Teraoka, M. Basic Study on Non-contact Respiration Measurement during Exercise Tolerance Test by Using Kinect Sensor. In Proceedings of the 2015 IEEE/SICE International Symposium on System Integration (Sii), Nagoya, Japan, 11–13 December 2015; pp. 217–222.
Hunt, K.J.; Fankhauser, S.E. Heart rate control during treadmill exercise using input-sensitivity shaping for disturbance rejection of very-low-frequency heart rate variability. Biomed. Signal Process. Control 2016, 30, 31–42.
Hung, K.; Zhang, Y. Preliminary investigation of pupil size variability: Toward non-contact assessment of cardiovascular variability. In Proceedings of the 2006 3rd IEEE/EMBS International Summer School on Medical Devices and Biosensors, Cambridge, MA, USA, 4–6 September 2006.
Li, S.; Li, X.; Lv, Q.; Zhang, D. WiFit: A Bodyweight Exercise Monitoring System with Commodity Wi-Fi. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, Singapore, 8–12 October 2018.
Inder, J.D.; Carlson, D.J.; Dieberg, G.; McFarlane, J.R.; Hess, N.C.; Smart, N.A. Isometric exercise training for blood pressure management: A systematic review and meta-analysis to optimize benefit. Hypertens. Res. 2016, 39, 88.
Palatini, P. Blood Pressure Behaviour During Physical Activity. Sport. Med. 1988, 5, 353–374.
Shirbani, F.; Blackmore, C.; Kazzi, C.; Tan, I.; Butlin, M.; Avolio, A.P. Sensitivity of Video-Based Pulse Arrival Time to Dynamic Blood Pressure Changes. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA, 18–21 July 2018.
Shirbani, F.; Moriarty, A.; Hui, N.; Cox, J.; Tan, I.; Avolio, A.P.; Butlin, M. Contactless video-based photoplethysmography technique comparison investigating pulse transit time estimation of arterial blood pressure. In Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Guadalajara, Jalisco, Mexico, 26–30 July 2021; pp. 5650–5653.
Jeong, I.C.; Finkelstein, J. Introducing Contactless Blood Pressure Assessment Using a High Speed Video Camera. J. Med. Syst. 2016, 40, 77.
Sawka, M.N.; Wenger, C.B.; Young, A.J.; Pandolf, K.B. Physiological Responses to Exercise in the Heat. In Nutritional Needs in Hot Environments: Applications for Military Personnel in Field Operations; Marriott, B.M., Ed.; National Academies Press (US): Washington, DC, USA, 1993; Volume 3.
McFarlin, B.; Venable, A.; Williams, R.; Jackson, A. Comparison of techniques for the measurement of skin temperature during exercise in a hot, humid environment. Biol. Sport 2015, 32, 11–14.
Manullang, M.C.T.; Lin, Y.-H.; Lai, S.-J.; Chou, N.-K. Implementation of Thermal Camera for Non-Contact Physiological Measurement: A Systematic Review. Sensors 2021, 21, 7777.
Lopez, M.B.; Del-Blanco, C.R.; Garcia, N. Detecting exercise-induced fatigue using thermal imaging and deep learning. In Proceedings of the 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA), Montreal, QC, Canada, 28 November–1 December 2017.
Ludwig, N.; Formenti, D.; Gargano, M.; Alberti, G. Skin temperature evaluation by infrared thermography: Comparison of image analysis methods. Infrared Phys. Technol. 2014, 62, 1–6.
de Andrade Fernandes, A.; dos Santos Amorim, P.R.; Brito, C.J.; Sillero-Quintana, M.; Marins, J.C.B. Regional Skin Temperature Response to Moderate Aerobic Exercise Measured by Infrared Thermography. Asian J. Sport. Med. 2016, 7, e29243.
James, C.A.; Richardson, A.J.; Watt, P.W.; Maxwell, N.S. Reliability and validity of skin temperature measurement by telemetry thermistors and a thermal camera during exercise in the heat. J. Therm. Biol. 2014, 45, 141–149.
Fernandes, A.A.; Gomes Moreira, D.; Brito, C.J.; da Silva, C.D.; Sillero-Quintana, M.; Mendonca Pimenta, E.; Bach, A.J.E.; Silami Garcia, E.; Bouzas Marins, J.C. Validity of inner canthus temperature recorded by infrared thermography as a non-invasive surrogate measure for core temperature at rest, during exercise and recovery. J. Therm. Biol. 2016, 62, 50–55.
Korman, P.; Straburzynska-Lupa, A.; Kusy, K.; Kantanista, A.; Zielinski, J. Changes in body surface temperature during speed endurance work-out in highly-trained male sprinters. Infrared Phys. Technol. 2016, 78, 209–213.
Jensen, M.M.; Poulsen, M.K.; Alldieck, T.; Larsen, R.G.; Gade, R.; Moeslund, T.B.; Franch, J. Estimation of Energy Expenditure during Treadmill Exercise via Thermal Imaging. Med. Sci. Sports Exerc. 2016, 48, 2571–2579.
Koporec, G.; Vučković, G.; Milić, R.; Perš, J. Quantitative Contact-Less Estimation of Energy Expenditure from Video and 3D Imagery. Sensors 2018, 18, 2435.
Tao, L.; Burghardt, T.; Mirmehdi, M.; Damen, D.; Cooper, A.; Hannuna, S.; Camplani, M.; Paiement, A.; Craddock, I. Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home. In Proceedings of the Computer Vision—ACCV 2016 Workshops, Proceedings of ACCV 2016 International Workshops, Revised Selected Papers, Taipei, Taiwan, 20–24 November 2016; Springer: Cham, Switzerland, 2016.
Gade, R.; Larsen, R.G.; Moeslund, T.B. Measuring energy expenditure in sports by thermal video analysis. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, 21–26 July 2017; pp. 187–194.
Koehler, K.; Drenowatz, C. Monitoring Energy Expenditure Using a Multi-Sensor Device—Applications and Limitations of the SenseWear Armband in Athletic Populations. Front. Physiol. 2017, 8, 983.
Aoki, H.; Ichimura, S.; Kiyooka, S.; Koshiji, K. Non-contact measurement method of respiratory movement under pedal stroke motion. In Proceedings of the 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France, 22–26 August 2007; IEEE: New York, NY, USA, 2007; pp. 374–377.
Anbu, A.; Selvakumar, K. Non-contact breath cycle analysis for different breathing patterns using RGB-D videos. Smart Health 2022, 25, 100297.
Persinger, R.; Foster, C.; Gibson, M.; Fater, D.C.W.; Porcari, J.P. Consistency of the Talk Test for exercise prescription. Med. Sci. Sports Exerc. 2004, 36, 1632–1636.
Sun, G.; Matsui, T.; Watai, Y.; Kim, S.; Kirimoto, T.; Suzuki, S.; Hakozaki, Y. Vital-SCOPE: Design and Evaluation of a Smart Vital Sign Monitor for Simultaneous Measurement of Pulse Rate, Respiratory Rate, and Body Temperature for Patient Monitoring. J. Sens. 2018, 2018, 4371872.
Aliverti, A. The respiratory muscles during exercise. Breathe 2016, 12, 165–168.
Aoki, H.; Nakamura, H. Non-Contact Respiration Measurement during Exercise Tolerance Test by Using Kinect Sensor. Sports 2018, 6, 23.
Sharpe, M.; Wilks, D. Fatigue. BMJ 2002, 325, 480–483.
Krupp, L.B. Fatigue in multiple sclerosis: Definition, pathophysiology and treatment. CNS Drugs 1972, 17, 225–234.
Hulme, K.; Safari, R.; Thomas, S.; Mercer, T.; White, C.; Van der Linden, M.; Moss-Morris, R. Fatigue interventions in long term, physical health conditions: A scoping review of systematic reviews. PLoS ONE 2018, 13, 203367.
Karlsen, K.; Larsen, J.P.; Tandberg, E.; Jørgensen, K. Fatigue in patients with Parkinson’s disease. Mov. Disord. 1999, 14, 237–241.
Haque, M.A.; Irani, R.; Nasrollahi, K.; Thomas, M.B. Facial video based detection of physical fatigue for maximal muscle activity. IET Comput. Vis. 2016, 10, 323–329.
Uchida, M.C.; Carvalho, R.; Tessutti, V.D.; Pereira Bacurau, R.F.; Coelho-Junior, H.J.; Portas Capelo, L.; Prando Ramos, H.; dos Santos, M.C.; Teixeira, L.F.M.; Marchetti, P.H. Identification of muscle fatigue by tracking facial expressions. PLoS ONE 2018, 13, 208834.
Irani, R.; Nasrollahi, K.; Moeslund, T.B. Contactless measurement of muscle fatigue by tracking facial feature points in video. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Paris, France, 27–30 October 2014; pp. 4181–5186.
Lopes, A.T.; Aguiar, E.; Souza, A.F.; Oliveira-Santos, T. Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order. Pattern Recognit. 2017, 61, 610–628.
Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012; p. 1.
Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556.
Ellwein, L.; Samyn, M.M.; Danduran, M.; Schindler-Ivens, S.; Liebham, S.; LaDisa, J.F. Toward translating near-infrared spectroscopy oxygen saturation data for the non-invasive prediction of spatial and temporal hemodynamics during exercise. Biomech. Model. Mechanobiol. 2017, 16, 75–96.
Khanal, S.R.; Sampaio, J.; Exel, J.; Barroso, J.; Filipe, V. Using Computer Vision to Track Facial Color Changes and Predict Heart Rate. J. Imaging 2022, 8, 245.
Miles, K.H.; Clark, B.; Périard, J.D.; Goecke, R.; Thompson, K.G. Facial feature tracking: A psychophysiological measure to assess exercise intensity? J. Sports Sci. 2018, 36, 934–941.
Viola, P.; Jones, M. Robust Real-time Object Detection. In Proceedings of the Second International Workshop on Statistical and Computational Theories of Vision, Vancouver, BC, Canada, 13 July 2001; pp. 1–25.

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Computer Science, Artificial Intelligence

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

View Times: 564

Update Date: 06 Feb 2023

Table of Contents

Video Upload Options

Confirm