Show simple item record

dc.contributor.author Boleda, Gemma
dc.contributor.author Pham, Nghia The
dc.contributor.author Kruszewski, German
dc.date.accessioned 2017-08-25T17:17:01Z
dc.date.available 2017-08-25T17:17:01Z
dc.date.issued 2016
dc.identifier.citation Pham N, Kruszewski G, Boleda G. Convolutional neural network language models. In: Su J, Duh K, Carreras X, editors. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin: Association for Computational Linguistics; 2016. p. 1153-1162.
dc.identifier.uri http://hdl.handle.net/10230/32701
dc.description.abstract Convolutional Neural Networks (CNNs) have shown to yield very strong results in several Computer Vision tasks. Their application to language has received much less attention, and it has mainly focused on static classification tasks, such as sentence classification for Sentiment Analysis or relation extraction. In this work, we study the application of CNNs to language modeling, a dynamic, sequential prediction task that needs models to capture local as well as long-range dependency information. Our contribution is twofold. First, we show that CNNs achieve 11-26% better absolute performance than feed-forward neural language models, demonstrating their potential for language representation even in sequential tasks. As for recurrent models, our model outperforms RNNs but is below state of the art LSTM models. Second, we gain some understanding of the behavior of the model, showing that CNNs in language act as feature detectors at a high level of abstraction, like in Computer Vision, and that the model can profitably use information from as far as 16 words before the target.
dc.description.sponsorship This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe); ERC 2011 Starting Independent Research Grant n. 283554 (COMPOSES) and the Erasmus Mundus Scholarship for Joint Master Programs.
dc.format.mimetype application/pdf
dc.language.iso eng
dc.publisher ACL (Association for Computational Linguistics)
dc.relation.ispartof Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin: Association for Computational Linguistics; 2016. p. 1153-1162
dc.rights © ACL, Creative Commons Attribution 4.0 License
dc.rights.uri http://creativecommons.org/licenses/by/4.0/
dc.title Convolutional neural network language models
dc.type info:eu-repo/semantics/conferenceObject
dc.subject.keyword Convolutional Neural Networks
dc.subject.keyword CNNs
dc.subject.keyword Natural Language Processing
dc.subject.keyword Deep learning
dc.subject.keyword Language models
dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/655577
dc.relation.projectID info:eu-repo/grantAgreement/EC/FP7/283554
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/publishedVersion

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Search DSpace


Advanced Search

Browse

My Account

Statistics

Compliant to Partaking