Controlled tasks for model analysis: retrieving discrete information from sequences

Mostra el registre complet Registre parcial de l'ítem

  • dc.contributor.author Sorodoc, Ionut-Teodor
  • dc.contributor.author Boleda, Gemma
  • dc.contributor.author Baroni, Marco
  • dc.date.accessioned 2022-02-09T13:23:44Z
  • dc.date.available 2022-02-09T13:23:44Z
  • dc.date.issued 2021
  • dc.description Comunicació presentada al 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP celebrat l'11 de novembre de 2021 de manera virtual.
  • dc.description.abstract In recent years, the NLP community has shown increasing interest in analysing how deep learning models work. Given that large models trained on complex tasks are difficult to inspect, some of this work has focused on controlled tasks that emulate specific aspects of language. We propose a new set of such controlled tasks to explore a crucial aspect of natural language processing that has not received enough attention: the need to retrieve discrete information from sequences. We also study model behavior on the tasks with simple instantiations of Transformers and LSTMs. Our results highlight the beneficial role of decoder attention and its sometimes unexpected interaction with other components. Moreover, we show that, for most of the tasks, these simple models still show significant difficulties. We hope that the community will take up the analysis possibilities that our tasks afford, and that a clearer understanding of model behavior on the tasks will lead to better and more transparent models.
  • dc.description.sponsorship This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
  • dc.format.mimetype application/pdf
  • dc.identifier.citation Sorodoc I, Boleda G, Baroni M. Controlled tasks for model analysis: retrieving discrete information from sequences. In: Bastings J, Belinkov Y, Dupoux E, Giulianelli M, Hupkes D, Pinter Y, Sajjad H, editors. Proceedings of the 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP; 2021 Nov 11; Punta Cana, (DO). Punta Cana: ACL; 2021. p. 468–78. DOI: 10.18653/v1/2021.blackboxnlp-1.37
  • dc.identifier.doi http://dx.doi.org/10.18653/v1/2021.blackboxnlp-1.37
  • dc.identifier.uri http://hdl.handle.net/10230/52449
  • dc.language.iso eng
  • dc.publisher ACL (Association for Computational Linguistics)
  • dc.relation.ispartof Bastings J, Belinkov Y, Dupoux E, Giulianelli M, Hupkes D, Pinter Y, Sajjad H, editors. Proceedings of the 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP; 2021 Nov 11; Punta Cana, (DO). Punta Cana: ACL; 2021. p. 468–78.
  • dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/715154
  • dc.rights © ACL, Creative Commons Attribution 4.0 License
  • dc.rights.accessRights info:eu-repo/semantics/openAccess
  • dc.rights.uri https://creativecommons.org/licenses/by/4.0/
  • dc.title Controlled tasks for model analysis: retrieving discrete information from sequences
  • dc.type info:eu-repo/semantics/conferenceObject
  • dc.type.version info:eu-repo/semantics/publishedVersion

Col·leccions

Congressos (Departament de Traducció i Ciències del Llenguatge)
Documents OpenAIRE (Open Access Infrastructure for Research in Europe)