Linguistic-based evaluation criteria to identify statistical machine translation errors

Farrús, Mireia; Costa-jussà, Marta R.; Mariño Acebal, José B.; Fonollosa, José A Rodriguez

Linguistic-based evaluation criteria to identify statistical machine translation errors

Citation

Farrús M, Costa-Jussà MR, Mariño JB, Fonollosa JAR. Linguistic-based evaluation criteria to identify statistical machine translation errors. In: Yvon F, Hansen V, editors. Proceedings of the 14th annual conference of the European Association for Machine Translation; 2010 May 27-28; Saint-Raphaël (France). [Allschwil]: European Association for Machine Translation, 2010. [7 p.].

Abstract

Machine translation evaluation methods are highly necessary in order to analyze the performance of translation systems. Up to now, the most traditional methods are the use of automatic measures such as BLEU or the quality perception performed by native human evaluations. In order to complement these traditional procedures, the current paper presents a new human evaluation based on the expert knowledge about the errors encountered at several linguistic levels: orthographic, morphological, lexical, semantic and syntactic. The results obtained in these experiments show that some linguistic errors could have more influence than other at the time of performing a perceptual evaluation.

Description

Comunicació presentada a: the 14th annual Conference of the European Association for Machine Translation,, celebrada del 27 al 28 de maig de 2010 a Saint-Raphaël, França.