Towards estimating the upper bound of visual-speech recognition: the visual lip-reading feasibility databas

Mostra el registre complet Registre parcial de l'ítem

  • dc.contributor.author Fernandez-Lopez, Adrianaca
  • dc.contributor.author Martinez, Oriolca
  • dc.contributor.author Sukno, Federico Mateoca
  • dc.date.accessioned 2017-09-01T16:58:09Z
  • dc.date.available 2017-09-01T16:58:09Z
  • dc.date.issued 2017
  • dc.description Comunicació presentada a: FG 2017 12th IEEE International Conference on Automatic Face and Gesture Recognition, celebrada del 30 de maig al 3 de juny de 2017 a Washington, Estats Units d'Amèrica.ca
  • dc.description.abstract Speech is the most used communication method between humans and it involves the perception of auditory and visual channels. Automatic speech recognition focuses on interpreting the audio signals, although the video can provide information that is complementary to the audio. Exploiting the visual information, however, has proven challenging. On one hand, researchers have reported that the mapping between phonemes and visemes (visual units) is one-to-many because there are phonemes which are visually similar and indistinguishable between them. On the other hand, it is known that some people are very good lip-readers (e.g: deaf people). We study the limit of visual only speech recognition in controlled conditions. With this goal, we designed a new database in which the speakers are aware of being read and aim to facilitate lip-reading. In the literature, there are discrepancies on whether hearingimpaired people are better lip-readers than normal-hearing people. Then, we analyze if there are differences between the lip-reading abilities of 9 hearing-impaired and 15 normalhearing people. Finally, human abilities are compared with the performance of a visual automatic speech recognition system. In our tests, hearing-impaired participants outperformed the normal-hearing participants but without reaching statistical significance. Human observers were able to decode 44% of the spoken message. In contrast, the visual only automatic system achieved 20% of word recognition rate. However, if we repeat the comparison in terms of phonemes both obtained very similar recognition rates, just above 50%. This suggests that the gap between human lip-reading and automatic speechreading might be more related to the use of context than to the ability to interpret mouth appearance.en
  • dc.description.sponsorship This work is partly supported by the Spanish Ministry of Economy and Competitiveness under the Ramon y Cajal fellowships and the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502), and the Kristina project funded by the European Union Horizon 2020 research and innovation programme under grant agreement No 645012.en
  • dc.format.mimetype application/pdf
  • dc.identifier.citation Fernandez-Lopez A, Martinez O, Sukno FM. Towards estimating the upper bound of visual-speech recognition: the visual lip-reading feasibility database. In: FG 2017 12th IEEE International Conference on Automatic Face and Gesture Recognition; 2017 May 30–June 3; Washington, DC, USA. [place unknown]: IEEE, 2017. p. 208-15. DOI: 10.1109/FG.2017.34
  • dc.identifier.doi http://dx.doi.org/10.1109/FG.2017.34
  • dc.identifier.uri http://hdl.handle.net/10230/32726
  • dc.language.iso eng
  • dc.publisher Institute of Electrical and Electronics Engineers (IEEE)ca
  • dc.relation.ispartof FG 2017 12th IEEE International Conference on Automatic Face and Gesture Recognition; 2017 May 30–June 3; Washington, DC, USA. [place unknown]: IEEE, 2017. p. 208-15.
  • dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/645012
  • dc.rights © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The final published article can be found at http://ieeexplore.ieee.org/document/7961743/
  • dc.rights.accessRights info:eu-repo/semantics/openAccess
  • dc.subject.keyword Visualizationen
  • dc.subject.keyword Speech recognitionen
  • dc.subject.keyword Speechen
  • dc.subject.keyword Contexten
  • dc.subject.keyword Visual databasesen
  • dc.subject.keyword Decodingen
  • dc.title Towards estimating the upper bound of visual-speech recognition: the visual lip-reading feasibility databasca
  • dc.type info:eu-repo/semantics/conferenceObject
  • dc.type.version info:eu-repo/semantics/acceptedVersion