Welcome to the UPF Digital Repository

Hand gestures as visual prosody: BOLD responses to audio–visual alignment are modulated by the communicative nature of the stimuli

Show simple item record

dc.contributor.author Biau, Emmanuel, 1985-
dc.contributor.author Morís Fernández, Luis, 1982-
dc.contributor.author Holle, Henning
dc.contributor.author Ávila, César
dc.contributor.author Soto-Faraco, Salvador, 1970-
dc.date.accessioned 2016-11-14T08:09:27Z
dc.date.issued 2016
dc.identifier.citation Biau E, Moris Fernández L, Holle H, Avila C, Soto-Faraco S. Hand gestures as visual prosody: BOLD responses to audio–visual alignment are modulated by the communicative nature of the stimuli. NeuroImage. 2016;132:129-137. DOI: 10.1016/j.neuroimage.2016.02.018
dc.identifier.issn 1053-8119
dc.identifier.uri http://hdl.handle.net/10230/27496
dc.description.abstract During public addresses, speakers accompany their discourse with spontaneous hand gestures (beats) that are tightly synchronized with the prosodic contour of the discourse. It has been proposed that speech and beat gestures originate from a common underlying linguistic process whereby both speech prosody and beats serve to emphasize relevant information. We hypothesized that breaking the consistency between beats and prosody by temporal desynchronization, would modulate activity of brain areas sensitive to speech–gesture integration. To this aim, we measured BOLD responses as participants watched a natural discourse where the speaker used beat gestures./nIn order to identify brain areas specifically involved in processing hand gestures with communicative intention, beat synchrony was evaluated against arbitrary visual cues bearing equivalent rhythmic and spatial properties as the gestures. Our results revealed that left MTG and IFG were specifically sensitive to speech synchronized with beats, compared to the arbitrary vision–speech pairing. Our results suggest that listeners confer beats a function of visual prosody, complementary to the prosodic structure of speech. We conclude that the emphasizing function of beat gestures in speech perception is instantiated through a specialized brain network sensitive to the communicative intent conveyed by a speaker with his/her hands.
dc.description.sponsorship This research was supported by the Ministerio de Economia y Competitividad (PSI2013-42626-P), AGAUR Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145).
dc.format.mimetype application/pdf
dc.language.iso eng
dc.publisher Elsevier
dc.relation.ispartof NeuroImage. 2016;132:129-137
dc.rights © Elsevier http://dx.doi.org/10.1016/j.neuroimage.2016.02.018
dc.title Hand gestures as visual prosody: BOLD responses to audio–visual alignment are modulated by the communicative nature of the stimuli
dc.type info:eu-repo/semantics/article
dc.identifier.doi http://dx.doi.org/10.1016/j.neuroimage.2016.02.018
dc.subject.keyword Speech perception
dc.subject.keyword Gestures
dc.subject.keyword Audiovisual speech
dc.subject.keyword Multisensory Integration
dc.subject.keyword MTG
dc.subject.keyword FMRI
dc.relation.projectID info:eu-repo/grantAgreement/EC/FP7/263145
dc.relation.projectID info:eu-repo/grantAgreement/ES/1PE/PSI2013-42626-P
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/acceptedVersion

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Search DSpace


Advanced Search

Browse

My Account

Statistics

In collaboration with Compliant to Partaking