The BEA 2024 shared task on the Multilingual Lexical Simplification Pipeline
The BEA 2024 shared task on the Multilingual Lexical Simplification Pipeline
Citació
- Shardlow M, Alva-Manchego F, Batista-Navarro R, Bott S, Calderon Ramirez S, Cardon R, et al. The BEA 2024 shared task on the Multilingual Lexical Simplification Pipeline. In: Kochmar E, Bexte M, Burstein J, Horbach A, Laarmann-Quante R, Tack A, Yaneva A, Yuan Z, editors. Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024); 2024 June 20; Mexico City, Mexico. Kerrville: Association for Computational Linguistics; 2024. p 571-89.
Enllaç permanent
Descripció
Resum
We report the findings of the 2024 Multilingual Lexical Simplification Pipeline shared task. We released a new dataset comprising 5,927 instances of lexical complexity prediction and lexical simplification on common contexts across 10 languages, split into trial (300) and test (5,627). 10 teams participated across 2 tracks and 10 languages with 233 runs evaluated across all systems. Five teams participated in all languages for the lexical complexity prediction task and 4 teams participated in all languages for the lexical simplification task. Teams employed a range of strategies, making use of open and closed source large language models for lexical simplification, as well as feature-based approaches for lexical complexity prediction. The highest scoring team on the combined multilingual data was able to obtain a Pearson’s correlation of 0.6241 and an ACC@1@Top1 of 0.3772, both demonstrating that there is still room for improvement on two difficult sub-tasks of the lexical simplification pipeline.Descripció
Comunicació presentada al 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), celebrat a Ciutat de Mèxic (Mèxic) el 20 de juny de 2024