3. Mt Errors and Post-mt Editing

MT errors refer to the inappropriate translations at the lexical, pragmatic and syntactic levels in the MT output. Since the current English-to-Ukrainian MT system cannot produce a satisfactory MT output, human editing is required to improve the quality, and this is known as post-MT editing. Post-MT editing, as defined by Juan C. Sager, is "the adaptation and revision of output of a machine translation system either to eliminate errors which impede comprehension or to make the output read like a natural-language text". Melby defined this term as "the process of revising a translation after the draft translation has been completed". In short, post-MT editing is mainly undertaken to improve the quality of the MT output for publication purposes.

MT errors can be classified into lexical, syntactic and pragmatic errors due to the lexicon-specific, syntax-specific and pragmatics-specific limitations of the MT system in operation. For the convenience of calculating MT errors in this project, the instructor asked students to group the lexical and pragmatic errors together into a single category. Table 1 shows some examples of three types of MT errors and certain hidden reasons. In the column of examples, the source language sentence, its MT output and post-MT editing are provided.

4. Methodology

Since we conduct an MT-based project with students to test the effectiveness of the cognitive learning of text types in translation, this section needs to introduce the method we have adopted, the students participating in this project and the teaching procedures.

This project integrates quantitative measurement and qualitative analysis. Students' statistical results of MT errors and post-MT editing were used as a quantitative means to see whether students could identify MT errors and use appropriate strategies to correct the errors. At the end of this project, a questionnaire was administered and the responses were statistically assessed to determine the students' understanding of the concept of text types with a comparative analysis of MT errors across text types. In addition, the students' reflections as recorded in the weekly assignment were exclusively used for a qualitative analysis.

To obtain the statistical figures, students were asked to edit MT errors and then count the frequency of the occurrence of particular MT errors at the lexical, syntactical and, if necessary, pragmatic levels. Students eventually calculated the average number of MT errors for each text type and then used the statistics to rank the syntactic complexity and the degree of lexical ambiguity or pragmatic clarity. This ranking allowed students to infer and observe the distinctive syntactic, lexical or pragmatic features of the three text types because these features were crucial in affecting and governing MT performance across text types.

The questionnaire consisted of twenty multiple-choice questions in the four areas of "Effectiveness of MT error analysis," "Relevance of text types to translation," "Learning distinctive linguistic features of text types," and "Affective and cognitive contribution." The multiple-choice questions were easy for students to answer and students were asked to answer honestly because the result of the questionnaire would be used only for the instructor's research. In addition, students were asked to write down their reflections at the end of their post-MT editing. Students had to examine the hallmarks of the three given text types and then identify their similarities and differences. Doing so could help students to understand the concept of text types.

This MT-based project was completed in a weekly three-hour machine translation class that ran for nine hours over three consecutive weeks. The project tasks moved from data analysis to concept building. During the first week, students were asked to use the MT system (TransWhiz) to translate an excerpt from instructions for buying an Olympus lithium-polymer battery (the informative text type) and an excerpt from one advertisement for the Gap Incorporation (the evocative/operative text type). Students had to edit and count the MT errors, and then they compared MT performance across two text types in the areas of lexical choice and syntactic structures.

During the remaining two weeks, students were asked to use the same MT system to translate an excerpt from a recipe for instant noodles and a user's manual on the installation of a refrigerator (representative of the informative text type), an excerpt from a short speech and one advertisement of the tour package for newlyweds (representative of the evocative/operative text type) and an excerpt from Shakespeare's play, Hamlet, and from Lord of the Rings (representative of the expressive text type). After that, students had to edit and count the MT errors, and then to identify the distinctive linguistic features of the three text types through analysis of the limitations of the MT system. In addition, students were asked to write an assessment of the MT performance and then to discriminate the dominant linguistic features of the three text types that constitute certain constraints which govern and control MT processing and MT performance. The process of writing reflections helps students to be aware of the similarities and differences among the three text types.