Gesture-speech temporal integration in language development

Abstract

In everyday interactions, speakers integrate gestures and speech sounds at a temporal level. One of the linguistic functions of temporally synchronous gesture-speech combinations is to provide prominence to specific parts of a discourse. While a bulk of evidence has explored the gesture-speech co-expressiveness at a semantic level, little is known about the children’s ability to use synchronized gestural and prosodic prominences in the benefit of language. This PhD thesis investigates gesture-speech temporal integration abilities in development and its beneficial impact for children’s language.

The dissertation includes three independent studies at different time points in development, each one described in one chapter. The first two studies aim at investigating the role of perceiving gesture-speech temporal synchronizations functioning as markers of prominence, and its linkage to language abilities. First, a study investigated whether three- to- five- year- old children responded better to a word recall task when the word was presented with a contrast of prominence expressed with a synchronous beat gesture (i.e., a hand gesture synchronized with prominence in speech). The results indicated a beneficial local effect of the beat gesture on the recall of the temporally synchronous word. Second, a study examined whether six- to- eight- year-old children processed pragmatic inferences online more rapidly when the relevant information was presented together with a beat gesture. Additionally, this study investigated whether these potential benefits were due to the prominence expressed in the gesture or to its concomitant prosodic prominence. Results showed that children’s processing of a pragmatic inference was improved by both prosodic and beat gesture prominence contributions to the discourse. The last study focused on the predictive role of the first infant’s uses of temporally synchronous gesture-speech combinations on later language development. To do so, a longitudinal study correlated the infants’ production of synchronous pointing gesture-speech combinations during controlled socio-communicative interactions at 12 months with linguistic measures at 18 months. Results demonstrated that synchronous productions positively correlated with lexical and grammatical development at 18 months of age.

Overall, the three studies show evidence that infant's synchronous gesture-speech abilities (a) function as multimodal markers of prominence; (b) when perceived in a discourse context synchronies have positive impact on children’s word recall (Study 1) and pragmatic inference resolution (Study 2); and (c) infants’ first productions of synchronous gesture-speech combinations serve a communicative strategy which is correlated to later language abilities (Study 3). The findings of the studies presented in this thesis point out the importance of synchronous gesture-speech combinations in highlighting information, as well as their beneficial effects in language acquisition.

Programa de doctorat en Traducció i Ciències del Llenguatge