An enhanced adversarial network with combined latent features for spatio-temporal facial affect estimation in the wild
Mostra el registre complet Registre parcial de l'ítem
- dc.contributor.author Aspandi, Decky
- dc.contributor.author Sukno, Federico Mateo
- dc.contributor.author Schuller, Björn
- dc.contributor.author Binefa i Valls, Xavier
- dc.date.accessioned 2021-03-26T09:26:04Z
- dc.date.available 2021-03-26T09:26:04Z
- dc.date.issued 2021
- dc.description Comunicació presentada al 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, celebrat del 8 al 10 de febrer de 2021 a Setúbal, Portugal.
- dc.description.abstract Affective Computing has recently attracted the attention of the research community, due to its numerous applications in diverse areas. In this context, the emergence of video-based data allows to enrich the widely used spatial features with the inclusion of temporal information. However, such spatio-temporal modelling often results in very high-dimensional feature spaces and large volumes of data, making training difficult and time consuming. This paper addresses these shortcomings by proposing a novel model that efficiently extracts both spatial and temporal features of the data by means of its enhanced temporal modelling based on latent features. Our proposed model consists of three major networks, coined Generator, Discriminator, and Combiner, which are trained in an adversarial setting combined with curriculum learning to enable our adaptive attention modules. In our experiments, we show the effectiveness of our approach by reporting our competitive results on both the AFEW-VA and SEWA datasets, suggesting that temporal modelling improves the affect estimates both in qualitative and quantitative terms. Furthermore, we find that the inclusion of attention mechanisms leads to the highest accuracy improvements, as its weights seem to correlate well with the appearance of facial movements, both in terms of temporal localisation and intensity. Finally, we observe the sequence length of around 160 ms to be the optimum one for temporal modelling, which is consistent with other relevant findings utilising similar lengths.en
- dc.description.sponsorship This work is partly supported by the Spanish Ministry of Economy and Competitiveness under project grant TIN2017-90124-P, the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502), and the donation bahi2018-19 to the CMTech at UPF. Further funding has been received from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 826506 (sustAGE).
- dc.format.mimetype application/pdf
- dc.identifier.citation Aspandi D, Sukno F, Schuller B, Binefa X. An enhanced adversarial network with combined latent features for spatio-temporal facial affect estimation in the wild. In: Farinella GM, Radeva P, Braz J, Bouatouch K, editors. Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications; 2021 Feb 8-10; Setúbal, Portugal. Setúbal: Scitepress; 2021. p. 172-81. DOI: 10.5220/0010332001720181
- dc.identifier.doi http://dx.doi.org/10.5220/0010332001720181
- dc.identifier.uri http://hdl.handle.net/10230/46961
- dc.language.iso eng
- dc.publisher Scitepress
- dc.relation.ispartof Farinella GM, Radeva P, Braz J, Bouatouch K, editors. Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications; 2021 Feb 8-10; Setúbal, Portugal. Setúbal: Scitepress; 2021. p. 172-81
- dc.relation.projectID info:eu-repo/grantAgreement/ES/2PE/TIN2017-90124-P
- dc.relation.projectID info:eu-repo/grantAgreement/ES/1PE/MDM-2015-0502
- dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/826506
- dc.rights CC BY-NC-ND 4.0. https://creativecommons.org/licenses/by-nc-nd/4.0/
- dc.rights.accessRights info:eu-repo/semantics/openAccess
- dc.rights.uri https://creativecommons.org/licenses/by-nc-nd/4.0/
- dc.subject.keyword Affective computingen
- dc.subject.keyword Temporal modellingen
- dc.subject.keyword Adversarial learningen
- dc.title An enhanced adversarial network with combined latent features for spatio-temporal facial affect estimation in the wilden
- dc.type info:eu-repo/semantics/conferenceObject
- dc.type.version info:eu-repo/semantics/acceptedVersion