Modeling word interpretation with deep language models: the interaction between expectations and lexical information

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Aina, Laura
• dc.contributor.author Brochhagen, Thomas
• dc.contributor.author Boleda, Gemma
• dc.date.accessioned 2021-09-27T11:34:32Z
• dc.date.available 2021-09-27T11:34:32Z
• dc.date.issued 2020
• dc.description.abstract How a word is interpreted depends on the context it appears in. We study word interpretation leveraging deep language models, tracing the contribution and interaction of two sources of information that have been shown to be central to it: contextinvariant lexical information, represented by the word embeddings of a model, and a listener’s contextual expectations, represented by its predictions. We define operations to combine these components to obtain representations of word interpretations. We instantiate our framework using two English language models, and evaluate the resulting representations in the extent by which they reflect contextual word substitutes provided by human subjects. Our results suggest that both lexi cal information and expectations codify information pivotal to word interpretation; however, their combination is better than either on its own. Moreover, the division of labor between expectations and the lexicon appears to change across contexts
• dc.description.sponsorship This project has re ceived funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and inno vation programme (grant agreement No 715154), and from the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research, and the computer resources at CTE-POWER and the technical support pro vided by Barcelona Supercomputing Center (RES-IM2019-3-0006). This paper reflects the authors view only, and the EU is not responsible for any use that may be made of the information it contains.
• dc.format.mimetype application/pdf
• dc.identifier.citation Aina L, Brochhagen T, Boleda G. Modeling word interpretation with deep language models: the interaction between expectations and lexical information. In: Denison S, Mack M, Xu Y, Armstrong BC. Proceedings for the 42nd Annual Meeting of the Cognitive Science Society; 2020 Jul 29 - 1 Aug. [Toronto]: Cognitive Science Society, 2020. 1518-24 p.
• dc.identifier.uri http://hdl.handle.net/10230/48507
• dc.language.iso eng
• dc.publisher Cognitive Science Society
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/715154
• dc.rights ©2020 The Author(s). This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY)
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri https://creativecommons.org/licenses/by/4.0
• dc.subject.keyword Expectations
• dc.subject.keyword Word meaning
• dc.subject.keyword Language models
• dc.subject.keyword Distributional semantics
• dc.subject.keyword Deep learning
• dc.subject.keyword Ambiguity
• dc.title Modeling word interpretation with deep language models: the interaction between expectations and lexical information
• dc.type info:eu-repo/semantics/conferenceObject
• dc.type.version info:eu-repo/semantics/publishedVersion