Building a Catalan-Chinese parallel corpus from Wikipedia for use in machine translation
Mostra el registre complet Registre parcial de l'ítem
- dc.contributor.author Zhou, Chenyue
- dc.contributor.other Melero i Nogués, Maite
- dc.date.accessioned 2022-09-21T16:55:53Z
- dc.date.available 2022-09-21T16:55:53Z
- dc.date.issued 2022-09-21
- dc.description Treball de fi de màster en Lingüística Teòrica i Aplicada. Directora: Dra. Maite Meleroca
- dc.description.abstract The lack of parallel corpora is one of the biggest challenges hindering progress in Machine Translation for low-resource languages. In this work, we crawl and filter parallel sentences in Catalan and Chinese from Wikipedia in order to compile a parallel corpus of good quality. This paper describes the processes we follow to build the corpus, including mining the text data, computing sentence embeddings, extracting sentence alignment and filtering for better corpus quality. We manually audit the corpus quality based on an error taxonomy. Results show that the automatic filtering we applied makes a great improvement in the quality of our web-crawled corpus. The corpus is later used as training data to finetune a multilingual Machine Translation (MT) system in both CA→ZH and ZH→CA directions. Results show that finetuning with our corpus successfully managed to improve BLEU score in both directions on the Flores-101 public benchmark test sets, which demonstrates the importance of corpus in MT and the quality of our Catalan-Chinese parallel corpus.EN
- dc.format.mimetype application/pdf*
- dc.identifier.uri http://hdl.handle.net/10230/54140
- dc.language.iso engca
- dc.rights Llicència CC Reconeixement-NoComercial-SenseObraDerivada 4.0 Internacional (CC BY-NC-ND 4.0)ca
- dc.rights.accessRights info:eu-repo/semantics/openAccessca
- dc.rights.uri https://creativecommons.org/licenses/by-nc-nd/4.0/ca
- dc.subject.keyword Parallel corpusen
- dc.subject.keyword Data miningen
- dc.subject.keyword Corpus qualityen
- dc.subject.keyword Machine translationen
- dc.subject.keyword Catalanen
- dc.subject.keyword Chineseen
- dc.subject.keyword Low-resource languagesen
- dc.title Building a Catalan-Chinese parallel corpus from Wikipedia for use in machine translationen
- dc.type info:eu-repo/semantics/masterThesisca