End-to-end facial and physiological model for affective computing and applications

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Comas, Joaquim
• dc.contributor.author Aspandi, Decky
• dc.contributor.author Binefa i Valls, Xavier
• dc.date.accessioned 2021-03-26T09:25:23Z
• dc.date.issued 2020
• dc.description Comunicació presentada al 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), celebrat del 16 al 20 de novembre de 2020 a Buenos Aires, Argentina.
• dc.description.abstract In recent years, affective computing and its applications have become a fast-growing research topic. Furthermore, the rise of deep learning has introduced significant improvements in the emotion recognition system compared to classical methods. In this work, we propose a multi-modal emotion recognition model based on deep learning techniques using the combination of peripheral physiological signals and facial expressions. Moreover, we present an improvement to proposed models by introducing latent features extracted from our internal Bio Auto-Encoder (BAE). Both models are trained and evaluated on AMIGOS datasets reporting valence, arousal, and emotion state classification. Finally, to demonstrate a possible medical application in affective computing using deep learning techniques, we applied the proposed method to the assessment of anxiety therapy. To this purpose, a reduced multimodal database has been collected by recording facial expressions and peripheral signals such as electrocardiogram (ECG) and galvanic skin response (GSR) of each patient. Valence and arousal estimates were extracted using our proposed model across the duration of the therapy, with successful evaluation to the different emotional changes in the temporal domain.en
• dc.description.sponsorship This work is partly supported by the Spanish Ministry of Economy and Competitiveness under project grant TIN2017-90124-P and the donation bahi2018-19 to the CMTech at the UPF.
• dc.format.mimetype application/pdf
• dc.identifier.citation Comas J, Aspandi D, Binefa X. End-to-end facial and physiological model for affective computing and applications. In: Štruc V, Gómez-Fernández F, editors. 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020); 2020 Nov 16-20; Buenos Aires, Argentina. New Jersey: IEEE; 2020. p. 93-100. DOI: 10.1109/FG47880.2020.00001
• dc.identifier.doi http://dx.doi.org/10.1109/FG47880.2020.00001
• dc.identifier.uri http://hdl.handle.net/10230/46949
• dc.language.iso eng
• dc.publisher Institute of Electrical and Electronics Engineers (IEEE)
• dc.relation.ispartof Štruc V, Gómez-Fernández F, editors. 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020); 2020 Nov 16-20; Buenos Aires, Argentina. New Jersey: IEEE; 2020. p. 93-100
• dc.relation.projectID info:eu-repo/grantAgreement/ES/2PE/TIN2017-90124-P
• dc.rights © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. http://dx.doi.org/10.1109/FG47880.2020.00001
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.subject.keyword Biological system modelingen
• dc.subject.keyword Physiologyen
• dc.subject.keyword Feature extractionen
• dc.subject.keyword Medical treatmenten
• dc.subject.keyword Estimationen
• dc.subject.keyword Electrocardiographyen
• dc.subject.keyword Computational modelingen
• dc.title End-to-end facial and physiological model for affective computing and applicationsen
• dc.type info:eu-repo/semantics/conferenceObject
• dc.type.version info:eu-repo/semantics/acceptedVersion