Fomicheva, MarinaSpecia, Lucia2024-03-132024-03-132016Fomicheva M, Specia L. Reference bias in monolingual machine translation evaluation. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers); 2016 Aug 7-10; Berlin, Germany. Stroudsburg (PA): Association for Computational Linguistics; 2016. p. 77-82. DOI: 10.18653/v1/P16-2013http://hdl.handle.net/10230/59399Comunicació presentada a la 54th Annual Meeting of the Association for Computational Linguistics, celebrada del 7 al 10 d'agost de 2016 a Berlín, Alemanya.In the translation industry, human translations are assessed by comparison with the source texts. In the Machine Translation (MT) research community, however, it is a common practice to perform quality assessment using a reference translation instead of the source text. In this paper we show that this practice has a serious issue – annotators are strongly biased by the reference translation provided, and this can have a negative impact on the assessment of MT quality.application/pdfengACL materials are Copyright © 1963–2023 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.Traducció automàticaTraducció automàtica -- Avaluacióinfo:eu-repo/semantics/conferenceObjecthttp://dx.doi.org/10.18653/v1/P16-2013info:eu-repo/semantics/openAccess