Exploring Machine Translation for code-switching between English and Setswana in South African classrooms

Abstract

It has been observed in South Africa that non-native English speaking learners that are taught mathematics in English as the medium of instruction tend to perform poorer than their counterparts that are taught in their first language. To try and address this, teachers tend to code-switch between English and the learners’ first language in mathematics classrooms. Code-switching is a topic of interest in NLP research and is impacted by the lack of data. This research is concerned with leveraging the use of existing pretrained language models such as mT5 (multilingual T5) and m2m-100 (Many-to-Many multilingual) for the task of machine translation of mathematical text written in English, into English text mixed with Setswana text. A small corpus of parallel sentences with the source text being in English and the target being Tswanglish (English mixed with Setswana) was transcribed from audio recordings of mathematical lessons given to Grade 10 learners who are native Setswana speakers but are taught in English in South Africa. This corpus was used to fine-tune the mT5 and m2m-100 models respectively. The mT5model returned a BLEU score of 12.9. The m2m-100 registered an impressive BLEU score of 46.5. During inference, the m2m-100 managed to produce an accurate Tswanglish sentence, whereas the mT5 returned a string of letters, suggesting that the m2m-100 is a better choice for this task. Though there was evidence of overfitting, it was observed that given a bigger dataset containing more English-Tswanglish parallel sentences, the m2m-100 has the potential of being used for the purpose of machine translation for code-switched mathematical English text to Tswanglish.