robust remote heart rate estimation from face utilizing ...scenario on vipl-hr effectiveness of da...

Xuesong Niu 1, *, Xingyuan Zhao 2, *, Hu Han 1 , Abhijit Das 3 , Antitza Dantcheva 3 , Shiguang Shan 1 , and Xilin Chen 1 Institute of Computing Technology, Chinese Academy of Sciences Zhejiang University Inria [email protected], [email protected], {hanhu, sgshan, xlchen}@ict.ac.cn, {abhijit.das, antitza.dantcheva}@inria.fr 中科院计算所 Robust Remote Heart Rate Estimation from Face Utilizing Spatial-temporal Attention Proposed Method Problem Remote heart rate estimation from face video 70 bpm Heart Rate Estimator Motivation Spatial-temporal map generation procedure Results Within-database testing ) Cross-database testing Various applications of remote heart rate estimation Existing learning based methods failed to remove the polluted HR signals, and an attention mechanism can be helpful for filtering the salient features in HR signals. 70 bpm Remote HR measurement HR Sleep monitor Health monitor Defection detection Stress analysis PRV … … … … HR signal estimated using rPPG Face video Estimated HR 68bpm 70bpm 90bpm 69bpm The HR databases are limited in size, and suffer from the problem of data imbalance in term of different HRs. The ground-truth HR distribution of VIPL-HR Data augmentation (DA) Original Video Down- sampled video Up- sampled video Video sequence Spatial- temporal map Pseudo HR 89bpm 71.2bpm 133bpm Attention machinem End-to-end HR estimation Ablation Study on VIPL-HR 7.99 8.14 8.94 Proposed ResNet-18+DA ResNet-18 RMSE(bpm) 8.27 7.85 w/o attention with attention RMSE(bpm) Effectiveness of Attention RMSE of the head movement scenario on VIPL-HR Effectiveness of DA RMSE of videos with different HR ranges on VIPL-HR Paper #: 236 7.99 8.94 17.2 21 16.9 Proposed Niu2018 Wang2017 Tulyakov2016 Haan2013 RMSE(bpm) 10.1 10.58 11.37 13.97 19.95 Proposed Niu2018 Tulyakov2016 Haan2013 Li2014 RMSE(bpm) Databases Database No. Subj. No. Vids. Video length protocol VIPL-HR 107 2,378 30s within- database MMSE-HR 40 102 30s cross- database Download VIPL-HR http://vipl.ict.ac.cn/vie w_database.php?id=15

Upload: others

Post on 10-Aug-2021

1 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Robust Remote Heart Rate Estimation from Face Utilizing ...scenario on VIPL-HR Effectiveness of DA RMSE of videos with different HR ranges on VIPL-HR Paper #: 236 7.99 8.94 17.2 21

Xuesong Niu1,*, Xingyuan Zhao2,*, Hu Han1, Abhijit Das3, Antitza Dantcheva3 , Shiguang Shan1, and Xilin Chen1

𝟏𝟏Institute of Computing Technology, Chinese Academy of Sciences𝟐𝟐Zhejiang University

𝟑𝟑[email protected], [email protected],

{hanhu, sgshan, xlchen}@ict.ac.cn, {abhijit.das, antitza.dantcheva}@inria.fr

中科院计算所

Robust Remote Heart Rate Estimation from Face Utilizing Spatial-temporal Attention

Proposed Method

ProblemRemote heart rate estimation from face video

70 bpmHeart Rate Estimator

Motivation

Spatial-temporal map generation procedure

Results

Within-database testing

)

Cross-database testing

Various applications of remote heart rate estimation

Existing learning based methods failed to remove the polluted HRsignals, and an attention mechanism can be helpful for filtering thesalient features in HR signals.

70 bpm

Remote HR measurement

Sleep monitor

Health monitor

Defection detection

Stress analysis

PRV

…

…… …

HR signal estimated using

rPPG

Face video