Multi-Modal, Remote Breathing Monitor: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Catherine Yang and Version 3 by Dean Liu.

Monitoring breathing is important for a plethora of applications including but not limited to baby monitoring, sleep monitoring and elderly care. This paper presents a way to fuse both vision based and RF based modalities for the task of estimating the breathing rate of a human. The modalities used are the F200 Intel's RealSense RGB and depth (RGBD) sensor, and an ultra wideband (UWB) radar. RGB image based features and their corresponding image coordinates are detected on the human body and are tracked using the famous optical flow algorithm of Lucas and Kanade. The depth at these coordinates is also tracked. The synced-radar received signal is processed to extract the breathing pattern. All of these signals are then passed to a harmonic signal detector which is based on a generalized likelihood ratio test. Finally, a spectral estimation algorithm based on the reformed Pisarenko algorithm tracks the breathing fundamental frequencies in real time, which are then fused into a one optimal breathing rate in a maximum likelihood fashion. RsWearchers tested this multi-modal setup on 14 human subjects and we report a maximum error of $0.5$ BPM compared to the true breathing rate.

  • vital signs
  • radar
  • micro-doppler
  • realsense
  • video

1. Introduction

Vital signs extraction has been a research topic both in the computer vision and radar research community. The computer vision-based algorithms are tackling this problem from two different angles: One is the color-based algorithms [[1][2][3][4]], which capture the minute color variation of the human skin during a heartbeat [[5]]. Color based algorithms are primarily used for heartbeat estimation which is not the scope of this paper. The other is known as motion magnification [[6]] that magnify minute movements in a video. This is used mainly for heartbeat but can be used also for breathing rate estimation.

Intel’s RealSense camera was used in [[7]] to estimate the heartrate of a human subject. They used the infrared (IR) channel for estimating the heartrate. The depth channel was used as well for estimating the pose of the human head. The use of optical flow for breathing estimation was presented in [[8]]; however, the detailed algorithm and result performance was not reported. Finally, in [[9]] we describe in details the algorithm to reliably extract breathing from a RGB video alone in real-time.

Lately, biological signals monitoring utilizing uDoppler has been the topic of the research community. Respiration rate extraction with a pulse-Doppler architecture is presented in [[10]]. The wavelet transform was used in [[11]] to overcome the Discrete Fourier Transform (DFT) resolution insufficiency, and in [[12]], the chirp Z transform was used on a IR-UWB radar echos to extract respiration rate. The same transform was used in [[13]] coupled with an analytical model for the remote extraction of both respiration and heartrate. Moreover, they verified the validity of a model in which the thorax and the heart are considered vibrating-scatterers, such that the total uDoppler return is a superposition of two sinusoids with different frequencies and amplitudes in which, the breathing frequency is smaller than the heartrate, and its amplitude is much larger.

 

2. Radar Measurement Setup

The radar measurement setup is given in Figure 1 and Figure 2. We use a IR-UWB impulse XeThru X4 radar module that transmits on a human subject (hereinafter “subject”). The raw data is collected at the PC through a USB interface and fed into the algorithm which is running real time. The radar is synced to the F200 Intel® RealSenseTM RGBD sensor. The radar operating parameters is given in Table 1. This single setup cost is ~$400. The cost can be reduced substantially after system design and large quantities discounts.

 
Sensors 20 01229 g001 550

Figure 1. Xethru X4 radar mounted to a laptop with RealSense sensor on it.

Sensors 20 01229 g002 550

Figure 2. Subject under test will be sitting on this chair infront of the radar and RealSense sensor.

Table 1. Radar parameters.

  Value Units Comment
Pulse Repetition Frequency 40.5 MHz  
Center frequency 7.29 GHz  
Bandwidth 1.5 GHz  
Peak pulse power –0.7 dBm  
Frame rate ∼10 Hz ∼70 dB processing gain

References

  1. Sungjun Kwon; Hyunseok Kim; Kwang Suk Park; Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society 2012, 2012, 2174-2177, 10.1109/EMBC.2012.6346392.
  2. Xiaobai Li; Jie Chen; Guoying Zhao; Matti Pietikäinen; Remote Heart Rate Measurement from Face Videos under Realistic Situations. 2014 IEEE Conference on Computer Vision and Pattern Recognition 2014, null, 4264-4271, 10.1109/cvpr.2014.543.
  3. Ming-Zher Poh; Daniel McDuff; Rosalind W. Picard; Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express 2010, 18, 10762, 10.1364/oe.18.010762.
  4. Ming-Zher Poh; Daniel McDuff; Rosalind W. Picard; Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam. IEEE Transactions on Biomedical Engineering 2010, 58, 7-11, 10.1109/TBME.2010.2086456.
  5. Balakrishnan, G.; Durand, F.; Guttag, J. Detecting Pulse from Head Motions in Video. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, Portland, OR, USA, 23–28 June 2013; IEEE Computer Society: Washington, DC, USA, 2013; pp. 3430–3437.
  6. Hao-Yu Wu; Michael Rubinstein; Eugene Shih; John Guttag; Frédo Durand; William Freeman; Eulerian video magnification for revealing subtle changes in the world. ACM Transactions on Graphics 2012, 31, 1-8, 10.1145/2185520.2185561.
  7. Chen, J.; Chang, Z.; Qiu, Q.; Li, X.; Sapiro, G.; Bronstein, A.; Pietikäinen, M. RealSense= Real Heart Rate: Illumination Invariant Heart Rate Estimation from Videos. In Proceedings of the 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), Oulu, Finland, 12–15 December 2016.
  8. K Nakajim; Y Matsumoto; T Tamura; Development of real-time image sequence analysis for evaluating posture change and respiratory rate of a subject in bed.. Physiological Measurement 2001, 22, N21.
  9. Regev, N.; Wulich, D. A simple, remote, video based breathing monitor. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Seogwipo, Korea, 11–15 July 2017; pp. 1788–1791.
  10. Koo, Y.S.; Ren, L.; Wang, Y.; Fathy, A.E. UWB MicroDoppler Radar for human Gait analysis, tracking more than one person, and vital sign detection of moving persons. In Proceedings of the 2013 IEEE MTT-S International Microwave Symposium Digest (IMS), Seattle, WA, USA, 2–7 June 2013; pp. 1–4.
  11. Tariq, A.; Shiraz, H.G. Doppler radar vital signs monitoring using wavelet transform. In Proceedings of the 2010 Loughborough, Antennas and Propagation Conference (LAPC), Loughborough, UK, 8–9 November 2010; pp. 293–296.
  12. Hsieh, C.H.; Shen, Y.H.; Chiu, Y.F.; Chu, T.S.; Huang, Y.H. Human respiratory feature extraction on an UWB radar signal processing platform. In Proceedings of the 2013 IEEE International Symposium on Circuits and Systems (ISCAS), Beijing, China, 19–23 May 2013; pp. 1079–1082.
  13. Antonio Lazaro; David Girbau; Ramon Villarino; Analysis of vital signs monitoring using an IR-UWB radar.. Progress In Electromagnetics Research 2010, 100, 265-284, 10.2528/pier09120302.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback