ECG signal reconstruction based on facial videos via combined explicit and implicit supervision

Accurate and continuous electrocardiogram (ECG) signal measurements are of paramount importance for the prevention of cardiovascular disease. Existing physiological measurement methods require the placement of sensors on the human body, which proves uncomfortable and inconvenient during long-term monitoring. Recent research has found that heart rates can be extracted from noncontact RGB facial videos by measuring subtle color changes in the skin region. However, designing an accurate feature representation structure for physiological measurements remain a challenge. This study presents a novel end-to-end ECG signal synthesis network using short facial video sequences. We combined explicit and implicit supervision structures, which allows the network to perceive physiological-related features from unlabeled feature sets and adopts a channel and frame aggregation attention mechanism to boost the network acquisition of periodic spatiotemporally correlated features. Finally, the generated features were fed into a physiological estimator consisting of a serially connected “Generative Adversarial” module for ECG signal reconstruction. Experimental results on publicly available datasets and a new dataset collected by us demonstrate that the proposed network outperforms state-of-the-art methods in ECG signal measurements. This is a powerful tool for use in telemedicine and health monitoring.