Enhancing Neural Machine Translation Through Incorporation of Unsupervised Language Understanding and Generation Techniques: The Case of English-Afaan Oromo Translation
No Thumbnail Available
Date
2024-05
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Breaking down language barriers is a paramount pursuit in the realm of Artificial Intelligence.
Machine Translation (MT), a domain within Natural Language Processing (NLP), holds the
potential to bridge linguistic gaps and foster global communication. Enhancing cross-cultural
communication through MT will be realized only if we succeed in developing accurate and
adaptable techniques which in turn demands adequate availability of linguistic resources. Unluckily,
under-resourced languages face challenges due to limited linguistic resources and sparse
parallel data. Previous studies tried to solve this problem by using monolingual pre-training
techniques. However, such studies solely rely on either Language Understanding (LU) or Language
Generation (LG) techniques resulting in skewed translation. This study aims to enhance
translation outcomes beyond the capabilities of previous studies by marrying the concepts of
LU and LG and hence boosting the quality of MT in both directions. Our proposed model,
the BERT-GPT incorporated Transformer, combines SOTA language models, BERT and GPT,
trained on monolingual data into the original Transformer model and demonstrates substantial
improvements. Experimental results shows that translation quality leaps forward, as evidenced
by a significant increase in the BLEU score reaching 42.09, from the baseline score of 35.75 for
English to Afaan Oromo translation, and 44.51 from the baseline score of 40.35 for Afaan Oromo
to English translation on test dataset. Notably, our model unveils a deep understanding of Afaan
Oromo’s linguistic nuances, resulting in translations that are precise, contextually appropriate,
and faithful to the original intent. By leveraging the power of unsupervised pre-training and
incorporation of unsupervised LU and LG techniques to the transformer model, we pave the
way for enhanced cross-cultural communication, advanced understanding and inclusivity in our
interconnected world.
Description
Keywords
BERT, GPT, NMT, Unsupervised pre-training