|PARACCHINI MARCO BRANDO MARIO||Cycle: XXXIII |
Tutor: MARTELLI PAOLO
Advisor: MARCON MARCO Major Research topic
:Remote PhotoPlethysmoGraphy using SPAD Camera and Deep LearningAbstract:
Remote PhotoPlethysmoGraphy (rPPG) allows the extraction of cardiac information by just analyzing a video stream of a person face. In this work we propose the use of a Single-Photon Avalanche Diode (SPAD) camera in order to perform rPPG with higher accuracy, especially in low illumination conditions, exploiting the higher sensitivity of the SPAD sensors. In particular, we explored the adoption of a rPPG application in an automotive environment in order to monitor, in a non invasive fashion, the driver's health state and potentially avoid accidents caused by acute illness states. In order to compensate for the SPAD camera’s low spatial resolution, a novel facial skin detection method, based on a deep learning architecture, is proposed. This method is able to precisely associate a skin label to each pixel of a given image depicting a face even when working with low resolution grayscale face images (64x32 pixel) and is able to work in presence of general environment condition regarding illumination, facial expressions, object occlusions and regardless of the gender, age and ethnicity of the subject. Moreover, some metrics were developed in order to monitor the dependability of the heart rate estimation and detect situations where an optical solution, such as rPPG, could fail. Finally, we developed an application that runs in real time on a small ARM device equipped on a car that is able to receive data from the SPAD camera, extract the heart signal and analyze it in order to constantly monitor the driver's health condition.