Crowdsourced translations in video-games: do they work?
Crowdsourced translations are an everegreen point of debate. But what are the results so far, especially in the field of gaming? To learn more, this week we will study one community translation from its conception to the finished product.
And before we start, I'd say this warrants not one, but two big forewords.
I don't mind amateur and semi-professional translation (if clients understand what they are getting)
For many translators, "crowdsourcing", "amateur translation" and "community translation" are quite simply the devil incarnate.
Personally, I believe that amateurs and professionals offer a thoroughly different service. Amateurs can provide a good performance. Attend the shows of your local music school o comedy club and you may be surprised of how entertaining they can be. But professionals guarantee a satisfying result. I don't want to see my doctor and be surprised how accurate his diagnosis is, I expect him to do so.
And that guarantee alone completely changes the level of detail, care and attention required.
Do we really need three full-time translators with CAT, QA and full review to translate Hello Kitty's Magical Caramel Adventures? Not necessarily.Do we need them to guarantee it will still be fine despite the Japlish source, the mad deadlines, the ambiguous length limits, the required terminology on 75 multi-tab reference files, and all the other setbacks that can stop a less experienced or motivated translator? Absolutely, and it's a risk clients shoulder themselves when they rely on amateurs.So, if a developer hasn't got the budget for a full-blown localization (or they feel that a community translation can build some buzz around their title), that's absolutely fine by me, as long as they understand (and have been warned) that cheap may prove really expensive.
Hard doesn't necessarily mean impossible
One might argue that you cannot give a game translation to a group of amateurs and expect a decent level of quality. There's some truth in that, but it's a prejudice I cannot accept, because we have been subject to the same kind of bias.
Still today, there are colleagues that consider it impossible to offer a good translation without having seen the whole game beforehand. We call it sim-ship localization, they call it (with a shiver of disgust) blindfolded translation.
The feedback we had through the years, both through the press and through the clients themselves, seems to show that it's indeed possible to achieve good results this way. Does this mean that our translations really compare to traditional ones? Not at all! Working without context, we could never hope of reaching the same level of adaptation and detail, but we refined other aspects, from style, to terminology, to QA raising the profile of our text so that the initial handicap is less and less visible for the audience.
Anyone aiming for a good amateur game localization indeed has quite a challenge in front of them but, if they set some realistic targets, and put hard work and dedication into building upon their points of strength, they should be able to find their own space, exactly like we found ours.
Background
In order to see game localization at work in a crowdsourced environment, we will follow a community project called Trucks & Trailers through the blog posts of its maker SCS Software, a niche developer specialized in vehicle driving simulators. On the 20th of April 2011 they wrote this message.
Crowdsourced translationsWe have an idea that we would like to discuss with you.
In the past, we have paid significant amounts of money for localization of our games. Just translating German Truck / UK Truck generation games into 12 languages cost us about the same as a full year salary of a member of SCS Software's team. Even after the huge expense, there were still several important languages left out, so many of our fans and customers had to play the game without official support for their native language. We also heard complaints from some of you about the quality of the translations. It may be a better idea to spend the money on extra art or extra features and employ another designer or programmer.
What if we asked you for help? Would you be ready to donate your free time to help us localize Trucks & Trailers (which is not so much text), and ETS2 later (which will be quite a lot of text)?
There are technologies out there to facilitate the job, for orchestrating the crowd-sourced localization. Technically, multiple people could even work in parallel on translating to the same language, to distribute the burden. With voting on quality of other people's translations sentence by sentence or word by word, the system should even catch any mistakes.
With your help, we could hopefully cover more languages than we would dare to spend on with the usual approach, and make the games enjoyable for more people around the world.
Chances are our strong community would come up with much better translation than even professional translators who are never knowledgeable enough about transportation industry. What do you think?
So their main motivation for switching from professional to amateur translation were:
-Saving on localization costs, freeing up resources for development
- Covering a wider range of languages
- Improving the quality of the translation, thanks to the experience of the community
On the 30th of April 2010, with the project set up on Getlocalization.com, they posted enthusiastically:
The power of the community #2
Within 12 hours of launching the Trucks & Trailers community translation project, 4 languages were already completed. Now 3 more hours later, we are up to 6 completed languages, and every minute some 10 more strings are translated across the mix of 20+ languages. All this was happening on Friday evening and night (at least for European participants). If anything is a clear indicator of the dedication and strength of the truck sim fan community, this must be it. Amazing!
The work is not quite done yet, even for the languages that the system reports as 100% done the review process is still under way. People can vote for the best translation if there are several competing variants, or still suggest alternatives and improvements. It is still worth it to join in and help.
The initial list of languages entered by SCS seems to have been extended by people offering help with even more languages. However it seems that there is now also a bit a mess being created, for example we had "Turkish (Turkey)" as one of the choices, but in parallel somebody started "Turkish" sub-project, so now it's very likely that some of the effort is lost as the Turkish contributors are split into two uncooperative groups. Please concentrate just on the more-completed variant.
All in all, the call for translations appears to have been very successful, with multiple languages completed in less than one day. All that appears to be missing is voting (the system in place to ensure no mistakes slip through) and some coordination among minor languages.
The 7th of May, they summed up the progress as follows:
State of translations - what a difference a week makes
Last week at this moment, some 200 people were hard at work translating Trucks & Trailers. Today, only a few of the most loyal are still trying to polish the translations. Several languages are still unfinished, or almost but not quite complete, and we are still hoping for the final push. Please stay with us, as we are closer to the finish line, now is the time to help to tidy the translations up. We appreciate your support and help very much.
It has been indeed a very interesting experiment, and I hope many of you are enjoying it. We have witnessed (even if not always completely understood) very active discussions in some language groups, like for example in German or Russian, and it seems that those guys really are trying very hard to make their translations superb quality. We have witnessed (and not understood at all) some controversy (like fighting groups in split Turkish translations), but we hope that in the end the results will be worth it.
We may still need to pass the results through professional proofreaders, the quality and especially the consistency may not be 100%, but our overall impression is very positive. Consider that the community is giving itself a major gift. Even if the translations end up not being perfect, when did you last play a game on day of release localized into Greek, Turkish, Bulgarian or Norwegian Bokmål as well as Nynorsk?
T&T may be quite unique in this regard. And yes, you may be helping us in the selfish capitalist goal of making as much money as possible though sales to people who wouldn't understand the game otherwise. But the way we work, with everything being reinvested straight into future vehicle simulators, you are actually investing your sweat equity into helping us make our future games better, complete them sooner, and make them understandable to more people. THANK YOU!
Apparently, the process has led to some heated debate, sometimes turning into proper bickering, but the game is now understandable by a wider audience, helping the success of this ultra-niche developer.
The translation
So, how good this crowdsourced translation actually is? In order to find out, we took 500 words of English to Italian translation from the project and rated it using the same metrics usually applied to game translations.
The results are highly negative and include four strings completely generated with Google translate, three major grammar mistakes ("devi padroneggiare con le manovre di base", "alcuni grandi spedizioni"), obvious typos ("diffficoltà", "movimento terro") and inconsistencies (bay translated both as "baia" and "alloggiamento" in less than 20 lines).
To put things into perspective, if the lowest possible score given by the system is "D", this translation ranks as an "N".
Following projects
SCS Software itself seems to agree that results were not always perfect, given the new approach they took on the 23rd of September
ETS2 Translators needed
Community translation of ETS2 has been under way for some time behind the scenes already, and several languages are now reported as 100% complete. We have privately invited major contributors from our previous translations, and they are doing a fabulous job.
We are still tweaking things extensively in the game based on tester's feedback, so strings are changed or added frequently. Our translators fortunately are quite patient with us so far when "breaking" their translations this way.
Unlike with the previous projects, this time we do not intend to open up the translation system to just anybody. It was causing problems with too much "noise" and sometimes even malicious abuse of the system.
We are now ready to bring in more translators for languages which haven't got enough contributions yet, and we would kindly like to ask more of you for help
The languages we need help with are: all Nordic languages, Spanish, Portuguese, Ukrainian and Balkan languages. We DO NOT need any help though with German, Polish, Hungarian, Czech, Romanian, Silesian (yes, we are ready to cover even "small" languages if we find dedicated contributors, Basque anyone?), Italian, and French. We are setting the per-language maximum to 3 people to allow for productive discussion in smaller circles of dedicated people.
What we are looking for: mature approach, solid language skills, excellent English language comprehension, and team spirit. Experience with previous community translations a plus, provable history of community participation a big benefit, too.
What you are getting in exchange? This is a volunteer activity with no compensation, so the most you can get is feeling good about helping your favorite game to be available to players in your own language. We have a history of expressing gratitude to people helping us out by giving out free activation codes for our games (for Scania TDS help we have gifted over 100 free copies of the game), but this is really up to our discretion if we see that the quality (most important criteria) and the overall effort really were a solid contribution.
Conclusions
I had a quick email exchange with SCS and their experience matched what I saw from the outside. If you remember, these were the initial goals of the project.
- Saving on localization costs, freeing up resources for development
- Covering a wider range of languages
- Improving the quality of the translation, thanks to the experience of the community
We might say that the first and second were reached. Following the different localization projects surely required some time and effort, but it was surely cheaper than a professional translation.
Also the goal of covering more languages appears to be reached. After all, like the developer said, very few games can boast Bulgarian or Greek translations.
The real sticking point seems to be quality. In one year and a half the tone goes from "Chances are our strong community would come up with much better translation than even professional translators" to "We may still need to pass the results through professional proofreaders" to what is substantially a job post for translators.
What we can learn from that? Apparently, voting doesn't really work as a QA mechanism. Too many cooks indeed spoil the broth and, while it's quite exciting to see "200 people hard at work translating", the results seem far less thrilling, lagging way below the worst professional translations.
I find it quite telling that those that were initially called "Crowdsourced translations" finally became "Community translations" with the crowd gradually replaced by "our translators", personally invited into "smaller circles of dedicated people" with "mature approach, solid language skills, excellent English language comprehension, and team spirit."
The almighty Internet, that was supposed to turn amateur lead into crowdsourced gold, has been replaced by a tight-knit community of select fans, co-creating the ultra-niche game of their dreams hand in hand with their favorite developer - which gets a deeper understanding of translation in the process.
No matter how "amateurish" the end results may be, we cannot really complain about that.