Evaluation of MOLTO Technologies

TitleEvaluation of MOLTO Technologies
Publication TypeSlide Presentation
AuthorsRautio, J, Koponen, M, Carlson, L
PublisherThe MOLTO Consortium
Place PublishedBarcelona, Spain
Year of Publication2013
Date Published05/2013
Publication LanguageEnglish
Abstract

We present the results of an evaluation of the translation quality in the MOLTO use cases, namely, the tourist phrasebook, ACE-in-GF and AceWiki, patents and mathematics. The results for cultural heritage are still pending. All evaluations were made by native or near-native level speakers of each language, 34 evaluators in all. The MOLTO translations were compared to Google, Bing and Systran, and the evaluators were to choose the best, either to accept as it is or for post-editing. Both automatic evaluation metrics and the percentage of the evaluators' preferred translations suggest that MOLTO method fares significantly better for the task; however, there is variation between languages. The time and effort needed to fix the languages that get the poorest results is a factor which we continue to evaluate.

KeywordsFinal Project Meeting - Barcelona 2013, MT quality, MT quality evaluation, WP9
Type of WorkPresentation
URLhttp://www.molto-project.eu/sites/default/files/Evaluation.pdf
AttachmentSize
Evaluation.pdf411.01 KB