Machine Translation Humor Update

It has been sometime since I first wrote a blog post about MT humor primarily because I really have not been able to find anything worth the mention, until now, and except for some really lame examples about how MT mistranslates (sic) I have not seen much to laugh heartily at. It seems a group of people on the web have discovered the humorous possibilities of MT in translating song lyrics which might be difficult even for good human translators. (It really seems strange to be saying “human translator”.) 

I should point out that in all these recent cases one does have to work at degrading the translation quality by running the same text through a whole sequence of preferably not closely related languages.

It has often surprised me that there are some in the MT industry who use “back translation” as a way to check MT quality, as from my vantage point it is an exercise that can only result in proving the obvious. MT back translation by definition should result in deterioration since to a very great extent MT will almost always be something less than a perfect translation. This point seems to evade many who advocate this method of evaluation, so let me clarify with some mathematics as math is one of the few conceptual frameworks available to man where proof is absolute or pretty damned certain at least.

If one has a perfect MT system then the Source and Target segments should be very close if not exactly the same. So mathematically we could state this as:

Source (1) x Target (1) = 1

since in this case we know our MT system is perfect ;-)

But in real life where humans play on the internet and you have DIY MT systems being used to determine what MT can produce, the results are less likely to equal 1 which is perfect as shown in the example above.

So lets say you and I do a somewhat serious evaluation of the output of various MT systems (each language direction should be considered a separate system) and find that the following table is true for our samples by running 5,000 sentences through various MT conversions and scoring each MT translation (conversion) as a percentage “correct” in terms of linguistic accuracy and precision.

Language Combination	Percentage Correct
English to Spanish	0.8 or 80%
Spanish to English	0.85 or 85%
English to German	0.7 or 70%
German to English	0.75 or 75%

So if we took 1,000 new sentences and translate them with MT we should expect that the percentage shown above would be “correct” (whatever that means). But if we now chain the results by making the output of one, the input of the other, we will find that results are different and and get continually smaller e.g.

EN > ES > EN = .8 x .85 = 0.68 or 68% correct

EN > DE > EN = .7 x .75 =  0.525 or 52.5% correct

So with MT we should expect that every back test will result in a lower or degraded results as we are multiplying the effect of two different systems. Since computers don’t really speak the language one cannot assume that they have equal knowledge going each way and if you provide a bad source from system A to system B you should expect a bad target as computers like some people, are very literal.

So now if we take our example and run it through multiple iterations we should see a very definite degradation of the output as we can see below.

EN > ES > EN(from MT) > DE > EN = .8 x .85 x .7 x .75 = 0.357 or 35.7%

So if you are trying to make MT look silly you have to run it through multiple iterations to get silly results. It would help further if you chose language combinations like EN to Japanese to Hindi to Arabic as this would cause more rapid degradation to the original English source. Try it and share your results in the comments. 

So here we have a very nicely done example and you should realize it takes great skill for the lead vocalist to mouth the MT words as if they were real lyrics and still maintain melodic and rhythmic integrity so be generous in your appreciation of their efforts.