Reference bias in monolingual machine translation evaluation

Fomicheva, Marina; Specia, Lucia

doi:http://dx.doi.org/10.18653/v1/P16-2013

Reference bias in monolingual machine translation evaluation

License

Citation

Fomicheva M, Specia L. Reference bias in monolingual machine translation evaluation. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers); 2016 Aug 7-10; Berlin, Germany. Stroudsburg (PA): Association for Computational Linguistics; 2016. p. 77-82. DOI: 10.18653/v1/P16-2013

Abstract

In the translation industry, human translations are assessed by comparison with the source texts. In the Machine Translation (MT) research community, however, it is a common practice to perform quality assessment using a reference translation instead of the source text. In this paper we show that this practice has a serious issue – annotators are strongly biased by the reference translation provided, and this can have a negative impact on the assessment of MT quality.

Description

Comunicació presentada a la 54th Annual Meeting of the Association for Computational Linguistics, celebrada del 7 al 10 d'agost de 2016 a Berlín, Alemanya.