Assessing the syntactic capabilities of transformer-based multilingual language models

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Pérez-Mayos, Laura
• dc.contributor.author Táboas García, Alba
• dc.contributor.author Mille, Simon
• dc.contributor.author Wanner, Leo
• dc.date.accessioned 2023-03-01T07:21:37Z
• dc.date.available 2023-03-01T07:21:37Z
• dc.date.issued 2021
• dc.description Comunicació presentada a Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, celebrat de l'1 al 6 d'agost de 2021 de manera virtual.
• dc.description.abstract Multilingual Transformer-based language models, usually pretrained on more than 100 languages, have been shown to achieve outstanding results in a wide range of crosslingual transfer tasks. However, it remains unknown whether the optimization for different languages conditions the capacity of the models to generalize over syntactic structures, and how languages with syntactic phenomena of different complexity are affected. In this work, we explore the syntactic generalization capabilities of the monolingual and multilingual versions of BERT and RoBERTa. More specifically, we evaluate the syntactic generalization potential of the models on English and Spanish tests, comparing the syntactic abilities of monolingual and multilingual models on the same language (English), and of multilingual models on two different languages (English and Spanish). For English, we use the available SyntaxGym test suite; for Spanish, we introduce SyntaxGymES, a novel ensemble of targeted syntactic tests in Spanish, designed to evaluate the syntactic generalization capabilities of language models through the SyntaxGym online platform.
• dc.description.sponsorship This work has been partially funded by the European Commission via its H2020 Research Program under the contract numbers 779962, 786731, 825079, and 870930.
• dc.format.mimetype application/pdf
• dc.identifier.citation Pérez-Mayos L, Táboas García A, Mille S, Wanner L. Assessing the syntactic capabilities of transformer-based multilingual language models. In: Zong C, Xia F, Li Wenjie, Navigli R. Findings of the Association for Computational Linguistics (ACL-IJCNLP 2021); 2021 Aug 1-6; online. Stroudsburg: Association for Computational Linguistics; 2021. p. 3799-812. DOI: 10.18653/v1/2021.findings-acl.333
• dc.identifier.doi http://dx.doi.org/10.18653/v1/2021.findings-acl.333
• dc.identifier.uri http://hdl.handle.net/10230/55970
• dc.language.iso eng
• dc.publisher ACL (Association for Computational Linguistics)
• dc.relation.ispartof Zong C, Xia F, Li Wenjie, Navigli R. Findings of the Association for Computational Linguistics (ACL-IJCNLP 2021); 2021 Aug 1-6; online. Stroudsburg: Association for Computational Linguistics; 2021. p. 3799-812.
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/779962
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/786731
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/825079
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/870930
• dc.rights © ACL, Creative Commons Attribution 4.0 License
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri https://creativecommons.org/licenses/by/4.0/
• dc.subject.other Lingüística computacional
• dc.title Assessing the syntactic capabilities of transformer-based multilingual language models
• dc.type info:eu-repo/semantics/conferenceObject
• dc.type.version info:eu-repo/semantics/publishedVersion