Multimodal communication in child-directed and adult-directed narrative speech: Speakers’ body movements and prosodic variance

Abstract

It has been demonstrated that adults adapt their speech when speaking to infants and children (child-directed speech or CDS) in comparison with adult-directed speech (ADS), exhibiting increased vocal pitch and pitch variation, and exaggerating use of co-speech gestures such as head movements and manual gestures. It has also been observed that the vocal aspects (i.e., speech itself) and the visual aspects (gesture) of human communication work in nuanced coordination with one another. However, whether augmentations of vocal cues in CDS are directly related to augmentations of gesture is unclear. The current study examined the relationship between augmentations of vocal and visual cues in CDS using TEACH-TALK, an audiovisual corpus of 40 future teachers who told the same story in two simulated conditions (ADS and CDS). Results revealed that vocal pitch, vocal pitch variance, and rates of body movement were clearly augmented in the CDS condition. Results showed no statistically meaningful correlations between vocal pitch variance and rates of body movement in ADS or in CDS, and no statistically significant relationship between augmentations in vocal pitch variance and body movement from ADS to CDS. These findings highlight the complex nature of speech-gesture coordination and the need for further study in order to better understand speech-gesture relationships in different communicative registers.