Large language models "ad referendum": How good are they at machine translation in the legal domain?

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Briva-Iglesias, Vicent
• dc.contributor.author Dogru, Gokhan
• dc.contributor.author Cavalheiro Camargo, João Lucas
• dc.date.accessioned 2025-01-28T15:20:11Z
• dc.date.available 2025-01-28T15:20:11Z
• dc.date.issued 2024
• dc.description.abstract This study evaluates the machine translation (MT) quality of two state-of-the-art large language models (LLMs) against a traditional neural machine translation (NMT) system across four language pairs in the legal domain. It combines automatic evaluation metrics (AEMs) and human evaluation (HE) by professional translators to assess translation ranking, fluency and adequacy. The results indicate that while Google Translate generally outperforms LLMs in AEMs, human evaluators rate LLMs, especially GPT-4, comparably or slightly better in terms of producing contextually adequate and fluent translations. This discrepancy suggests LLMs' potential in handling specialized legal terminology and context, highlighting the importance of human evaluation methods in assessing MT quality. The study underscores the evolving capabilities of LLMs in specialized domains and calls for reevaluation of traditional AEMs to better capture the nuances of LLM-generated translations.
• dc.format.mimetype application/pdf
• dc.identifier.citation Briva-Iglesias V, Dogru G, Cavalheiro Camargo JL. Large language models "ad referendum": How good are they at machine translation in the legal domain? MonTI. Monographs in Translation and Interpreting. 2024;16:75–107. DOI: 10.6035/MonTI.2024.16.02
• dc.identifier.doi http://dx.doi.org/10.6035/MonTI.2024.16.02
• dc.identifier.issn 1889-4178
• dc.identifier.uri http://hdl.handle.net/10230/69348
• dc.language.iso eng
• dc.publisher Universitat d'Alacant
• dc.relation.ispartof MonTI. Monographs in Translation and Interpreting. 2024;16:75–107
• dc.rights Este trabajo se comparte bajo la licencia de Atribución-NoComercial-CompartirIgual 4.0 Internacional de Creative Commons (CC BY-NC-SA 4.0): https://creativecommons.org/licenses/by-nc-sa/4.0/.
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri http://creativecommons.org/licenses/by-nc-sa/4.0/
• dc.subject.keyword Machine translation
• dc.subject.keyword Large language model
• dc.subject.keyword Legal translation
• dc.subject.keyword Human evaluation
• dc.subject.keyword Automatic evaluation
• dc.title Large language models "ad referendum": How good are they at machine translation in the legal domain?
• dc.type info:eu-repo/semantics/article
• dc.type.version info:eu-repo/semantics/publishedVersion