How to represent a word and predict it, too: improving tied architectures for language modelling

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Gulordava, Kristina
• dc.contributor.author Aina, Laura
• dc.contributor.author Boleda, Gemma
• dc.date.accessioned 2018-10-30T11:45:32Z
• dc.date.available 2018-10-30T11:45:32Z
• dc.date.issued 2018
• dc.description Comunicació presentada a la EMNLP 2018 Conference on Empirical Methods in Natural Language Processing, celebrada a Brussel·les (Bèlgica), del 31 d'octubre al 4 de novembre de 2018.
• dc.description.abstract Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks.
• dc.description.sponsorship This project as received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907) and the Catalan government (SGR 2017 1575).
• dc.format.mimetype application/pdf
• dc.identifier.citation Gulordava K, Aina L, Boleda G. How to represent a word and predict it, too: improving tied architectures for language modelling. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing; 2018 Oct 31 - Nov 4; Brussels, Belgium. Stroudsburg: Association for Computational Linguistics; 2018. p. 2936–41.
• dc.identifier.uri http://hdl.handle.net/10230/35675
• dc.language.iso eng
• dc.publisher ACL (Association for Computational Linguistics)
• dc.relation.ispartof Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing; 2018 Oct 31 - Nov 4; Brussels, Belgium. Stroudsburg: Association for Computational Linguistics; 2018. p. 2936–41.
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/715154
• dc.rights © ACL, Creative Commons Attribution 4.0 License
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri http://creativecommons.org/licenses/by/4.0/
• dc.subject.keyword Language models
• dc.subject.keyword Word embeddings
• dc.subject.keyword Neural networks
• dc.subject.keyword Tied representations
• dc.title How to represent a word and predict it, too: improving tied architectures for language modelling
• dc.type info:eu-repo/semantics/conferenceObject
• dc.type.version info:eu-repo/semantics/publishedVersion