[논문]얼굴 영상 기반의 심박수 추정을 위한 딥러닝 모델의 경량화 기법

황규태; 박명근; 이상준

doi:10.14372/iemek.2023.18.2.51

얼굴 영상 기반의 심박수 추정을 위한 딥러닝 모델의 경량화 기법
Lightweight Deep Learning Model for Heart Rate Estimation from Facial Videos 원문보기

대한임베디드공학회논문지 = IEMEK Journal of embedded systems and applications, v.18 no.2, 2023년, pp.51 - 58

황규태 (Jeonbuk National University) , 박명근 (Polaris3D Research) , 이상준 (Jeonbuk National University)

Abstract ▼ AI-Helper

This paper proposes a deep learning method for estimating the heart rate from facial videos. Our proposed method estimates remote photoplethysmography (rPPG) signals to predict the heart rate. Although there have been proposed several methods for estimating rPPG signals, most previous methods can not be utilized in low-power single board computers due to their computational complexity. To address this problem, we construct a lightweight student model and employ a knowledge distillation technique to reduce the performance degradation of a deeper network model. The teacher model consists of 795k parameters, whereas the student model only contains 24k parameters, and therefore, the inference time was reduced with the factor of 10. By distilling the knowledge of the intermediate feature maps of the teacher model, we improved the accuracy of the student model for estimating the heart rate. Experiments were conducted on the UBFC-rPPG dataset to demonstrate the effectiveness of the proposed method. Moreover, we collected our own dataset to verify the accuracy and processing time of the proposed method on a real-world dataset. Experimental results on a NVIDIA Jetson Nano board demonstrate that our proposed method can infer the heart rate in real time with the mean absolute error of 2.5183 bpm.

주제어

표/그림 (10)

그림 그림 2. 영상-PPG 데이터 수집 시스템 구성도 Fig. 2. Configuration diagram of video-PPG data acquisition system
그림 그림 1. UBFC-rPPG 데이터 영상 예시 Fig. 1. Example image of UBFC-rPPG dataset
그림 그림 3. 전처리 영상 시각화 예시 Fig. 3. Visualization of pre-processed image
그림 그림 4. Teacher 모델과 Student 모델 구조 및 경량화 과정 Fig. 4. Pipeline of Knowledge distillation with structure of teacher and student model
표 표 1. UBFC-rPPG 데이터 셋에서의 비교실험 결과 Table 1. Performence comparison on the UBFC-rPPG dataset
표 표 2. Teacher와 student 모델의 성능 비교 실험 Table 2. Performance comparison of the teacher and student models (KD: knowledge distillation)
표 표 3. 실제 데이터셋에 대한 심박수 추정 성능 검증 Table 3. Evaluation of the proposed method on the real-world dataset (KD: knowledge distillation)
표 표 4. 젯슨나노 임베디드 보드에서의 처리시간 비교 Table 4. Processing time of the teacher and student models in Jetson Nano embedded board
그림 그림 5. 딥러닝 모델이 추론한 rPPG 및 이에 대한 파워스펙트럼 (심박수 추정 MAE가 1 bpm 이하인 경우) Fig. 5. Predicted rPPG and its power spectrum with MAE below 1 bpm
그림 그림 6. 딥러닝 모델이 추론한 rPPG 및 이에 대한 파워스펙트럼 (심박수 추정 MAE가 10 bpm 이상인 경우) Fig. 6. Predicted rPPG and its power spectrum with MAE over 10 bpm

참고문헌 (18)

M. Poh, D. J. McDuff, R. W. Picard, "Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam," IEEE Transactions on Biomedical Engineering, Vol. 58, No. 1, pp. 7-11, 2011.？

상세보기
M. Lewandowska, J. Ruminski, T. Kocejko, J. Nowak, "Measuring Pulse Rate with a Webcam - A Non-contact Method for Evaluating Cardiac Activity," Proceedings of 2011 Federated Conference on Computer Science and Information Systems, pp. 405-410, 2011.？
G. de Haan, V. Jeanne, "Robust Pulse Rate From Chrominance-Based rPPG," IEEE Transactions on Biomedical Engineering, Vol. 60, No. 10, pp. 2878-2886, 2013.？

상세보기
W. Wang, A. C. den Brinker, S. Stuijk, G. de Haan, "Algorithmic Principles of Remote PPG," IEEE Transactions on Biomedical Engineering, Vol. 64, No. 7, pp. 1479-1491, 2017.？

상세보기
W. Chen, D. McDuff, "DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks," Proceedings of the European Conference on Computer Vision, pp. 349-365, 2018.？
Z. Yu, X. Li, G. Zhao, "Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-temporal Networks," Proceedings of the British Machine Vision Conference, 2019.？
E. M. Nowara, D. McDuff, A. Veeraraghavan, "The Benefit of Distraction: Denoising Camera-Based Physiological Measurements Using Inverse Attention," Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4955-4964, 2021.？
Z. Yu, Y. Shen, J. Shi, H. Zhao, P. H. S. Torr, G. Zhao, "PhysFormer: Facial Video-Based Physiological Measurement With Temporal Difference Transformer," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4186-4196, 2022.？
S. Bobbia, R. Macwan, Y. Benezeth, A. Mansouri, J. Dubois, "Unsupervised Skin Tissue Segmentation for Remote Photoplethysmography," Pattern Recognition Letters, Vol. 124, pp. 82-90, 2019.？

상세보기
K. Zhang, Z. Zhang, Z. Li, Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," IEEE Signal Processing Letters, Vol. 23, No. 10, pp. 1499-1503, 2016.？

상세보기
G. E. Hinton, O. Vinyals, J. Dean, "Distilling the Knowledge in a Neural Network," ArXiv:1503.02531, 2015.？
S. Lin, H. Xie, B. Wang, K. Yu, X. Chang, X. Liang, G. Wang, "Knowledge Distillation via the Target-Aware Transformer," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10915-10924, 2022.？
E. Lee, E. Chen., Lee, C. Y. Lee, "Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-learner," Proceedings of the European Conference on Computer Vision, pp. 392-409, 2020.？
X. Niu, H. Han, S. Shan, X. Chen, "SynRhythm: Learning a Deep Heart Rate Estimator from General to Specific," Proceedings of the International Conference on Pattern Recognition, pp. 3580-3585, 2018.？
F. Bousefsaf, A. Pruski, C. Maaoui, "3d Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video," Applied Sciences 9, No. 20: 4364, 2019.？
P. Gupta, B. Bhowmick, A. Pal, "Accurate Heart-rate Estimation from Face Videos Using Qualitybased Fusion," Proceedings of the 2017 International Conference on Image Processing, pp. 4132-4136, 2017.？
R. Song, H. Chen, J. Cheng, C. Li, Y. Liu, X. Chen, "PulseGAN: Learning to Generate Realistic Pulse Waveforms in Remote Photoplethysmography," IEEE Journal of Biomedical and Health Informations, Vol. 25. No. 5, pp. 1373-1384, 2021.？
Z. Yu, W. Peng, X. Li, X. Hong, G. Zhao, "Remote Heart Rate Measurement from Highly Compressed Facial Videos: An End-to-end Deep Learning Solution with Video Enhancement," Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 151-160, 2019.

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증