How to get genuine, unbiased opinion about the quality of translated material Thread poster: GiuliaBa
|
|---|
I head the operational effort of a large organisation's in-house translation team. Although we have quality assurance procedures in place, I am always struggling to find a way to measure the efficacy of translated content by a genuine, unbiased audience. Any advice would be very appreciated.
Big thanks!
| | | |
opolt 
Germany
Local time: 23:33
English to German
+ ...
| Not sure what you mean | Feb 8, 2012 |
Because the intended audience of your documents won't normally be able to assess the quality of your translations (as such) -- if they were able to do so, they wouldn't need them translated (because they speak all those languages already).
If you are referring to an assessment of the effects of those translated works on your potential audience, personally I wouldn't know how answer that -- too much depends on specific circumstances. At any rate, I have my doubts... See more Because the intended audience of your documents won't normally be able to assess the quality of your translations (as such) -- if they were able to do so, they wouldn't need them translated (because they speak all those languages already).
If you are referring to an assessment of the effects of those translated works on your potential audience, personally I wouldn't know how answer that -- too much depends on specific circumstances. At any rate, I have my doubts that such an assessment can be objective at all.
Lastly, if you want to check (and improve) the quality of your translators' output, the best (and most expensive) way to achieve that would be to hire some external, hand selected translators experienced in proofreading and with many (15+) years of experience under their belt in the respective speciality field(s). You also should provide them with all the context information required (including info about the actual material conditions under which the translations are produced, such as time constraints, availability of reference sources, software used, quality of source material, etc.) -- the more, the better. After that, it would be nice to start a friendly, instructive dialogue between those proofreaders and your translators -- you don't want your translators to feel guilty and unsure about their capabilities, you want them to improve on what they are doing, and the only way to ensure that is by creating an environment which fosters learning.
[Edited at 2012-02-08 13:50 GMT]
[Edited at 2012-02-08 14:22 GMT] ▲ Collapse
| | | |
GiuliaBa
Local time: 22:33
TOPIC STARTER | Current practice | Feb 8, 2012 |
Thanks for your reply
Here's how it currently works:
1. The content (mainly, website content) is produced in English by a copywriter that works in tandem with designers.
2. Developers produce the web page / mobile app, etc and the source text is sent to the translators in single strings of text (i.e. no visual context is provided)
3. The text is translated and submitted by the translators and inserte... See more Thanks for your reply
Here's how it currently works:
1. The content (mainly, website content) is produced in English by a copywriter that works in tandem with designers.
2. Developers produce the web page / mobile app, etc and the source text is sent to the translators in single strings of text (i.e. no visual context is provided)
3. The text is translated and submitted by the translators and inserted into the localized webpages by developers.
4. The new page is loaded onto a testing site, where it is proofread by the same person who translated the content before being released to public. Any changes to the translations are implemented as this stage.
5. On a monthly basis, we collect random samples of each translator's recently completed work, and we send it on to an external agency, which provides proofreading services and suggests improvements where necessary.
6. The agency's feedback is passed on to the translators, and any eventual changes to copy are discussed with them.
This process works fine, but I would now like to gather a different feedback, from real users maybe, to understand whether the content is localised well enough to give the impression of being original content (not a translation of something else) and therefore provide an excellent user experience to our user-base worldwide, or if it is clearly a translation (more or less correct and understandable) of a foreign product/website.
Thanks for your feedback!! really looking forward to your reply ▲ Collapse
| | | |
| Not sure what you mean II | Feb 8, 2012 |
GiuliaBa wrote:
... I am always struggling to find a way to measure the efficacy of translated content by a genuine, unbiased audience.
Please define "efficacy of translated content". In my specialist areas we do use several methods such as "linguistic validation" or "readability testing" to ensure the quality of a translation or the readability for a specific audience. There are also methods to check what the reader understands/remembers when reading a certain text etc.
You can send me a PM (e. g. with your phone # or skype name) to discuss your needs. Depending on what your requirements are, I might be able to point you to some people who are experts in this area.
| | |
|
|
|
If by "genuine, unbiased audience" you mean persons that have not been involved in the translation process, then I agree with Opolt. . For that, obviously, you need to hire proofreaders. Said proofreaders must be a) native in the target language b) working in one of their speciality fields and, ideally, c) proficient in the source language as well (unless we are talking about "blind proofreading" where the source text isn't made available to the proofreader)
On the other hand, perha... See more If by "genuine, unbiased audience" you mean persons that have not been involved in the translation process, then I agree with Opolt. . For that, obviously, you need to hire proofreaders. Said proofreaders must be a) native in the target language b) working in one of their speciality fields and, ideally, c) proficient in the source language as well (unless we are talking about "blind proofreading" where the source text isn't made available to the proofreader)
On the other hand, perhaps you are talking about a marketing text (promotional material, advertising campaign, etc.) and what you are looking for is an "unbiased audience" made up of non-professionals, who will assess the translation AS CONSUMERS based on how effective the message is. I'm not sure what to advise there. I believe any good proofreader will be able to look at the text form that perspective as well.
I am wondering if you posted this question because your company does not employ proofreaders (in which case, you should) or if you do but don't trust their judgement. If it's the latter you need to ask yourself why: have you had any complaints from a client? If you have, you should evaluate your translators AND your proofreaders by hiring external proofreaders. If you see that some of your regular collaborators consistently deliver substandard work, don't risk your company's reputation. Keep only the best ones.
Edit to add: Oops, I posted this before I saw your second message, Giulia.
"The new page is loaded onto a testing site, where it is proofread by the same person who translated the content before being released to public. Any changes to the translations are implemented as this stage"
Ouch. Wrong. IMO, it should be proofread by the translator himself/herself first, sure, but then should ALWAYS be proofed by a different translator as well. Even the best translators are only human and might miss things, especially their own mistakes.
[Edited at 2012-02-08 14:28 GMT] ▲ Collapse
| | | |
opolt
Germany
Local time: 23:33
English to German
+ ...
| Seems to be about usability | Feb 8, 2012 |
Maybe you need to conduct some kind of (software) usability study then (if the focus is more on the interface and interaction, than on the "mere" reading experience/understandability/fluency).
http://en.wikipedia.org/wiki/Usability
PS I would like to add that to me, it seems improbable that you will get objective, useful feedback about the "nativeness/natural flow" of the ... See more Maybe you need to conduct some kind of (software) usability study then (if the focus is more on the interface and interaction, than on the "mere" reading experience/understandability/fluency).
http://en.wikipedia.org/wiki/Usability
PS I would like to add that to me, it seems improbable that you will get objective, useful feedback about the "nativeness/natural flow" of the interface from a non-translator/linguist. And a pure usability expert won't be aware of the linguistic aspects involved (maybe). But, given that the work was already submitted for proofreading before, I think you now need to go to the next higher level, in order to integrate all the aspects that you are interested in; thus my impression is that you need (at the very minimum) a person who combines experience in different computer/web/phone interfaces with a localization/translation background.
[Edited at 2012-02-08 14:42 GMT] ▲ Collapse
| | | |
GiuliaBa
Local time: 22:33
TOPIC STARTER | Here is what I mean | Feb 8, 2012 |
To: Siegfried Armbruster
Sorry for the confusion. What I want to find out is whether the translators have managed to adapt the original content to the target language so well that every aspect of the text (message, style, tone of voice, emotions, cultural background...) resonate with the audience, and it seems as if that text had been written in the target language, rather than translated.
I would like a Colombian user, for instance, to feel as if they had landed on a truly ... See more To: Siegfried Armbruster
Sorry for the confusion. What I want to find out is whether the translators have managed to adapt the original content to the target language so well that every aspect of the text (message, style, tone of voice, emotions, cultural background...) resonate with the audience, and it seems as if that text had been written in the target language, rather than translated.
I would like a Colombian user, for instance, to feel as if they had landed on a truly Colombian website, when they find us, rather than feel ours is just a foreign company's website more or less accurately translated from English.
To: opolt
Usability studies are generally conducted by another team before the product goes through the localization process. I am specifically looking for feedback about content and how well it has been adapted to the local language...
[Edited at 2012-02-08 15:50 GMT] ▲ Collapse
| | | |
opolt
Germany
Local time: 23:33
English to German
+ ...
GiuliaBa wrote:
...
To: opolt
Usability studies are generally conducted by another team before the product goes through the localization process. I am specifically looking for feedback about content and how well it has been adapted to the local language...
[Edited at 2012-02-08 15:50 GMT]
Well ok, I understand. Though both aspects are often quite interrelated -- the usability aspect is neglected in 99% of all cases, and it shows (even in simple online help texts).
Eliza has mentioned some good points. WRT the Colombian example, the best way would be to hire a Colombian native right from the start (rather than having it translated into some kind of generic Spanish first, for example).
I'm not sure whether your goal of 100% nativity is attainable. Maybe, but somehow I doubt it. You have an identity as a (say) British company, and most users will always be aware that you are what you are -- unless you want to hide it. Your goal, the way you've put it, is laudable but quite difficult to achieve in an everyday setting, I would say, especially if interactive contents/interfaces are involved, and it's more like a never-ending process than a static thing. And that goes double for anything IT related, for reasons related to the way the industry is run (to put it mildly :-]) and also because it is already heavily anglicised.
Also, wrt to sounding completely native, you can get into philosophical territory very quickly. Many would argue that it's impossible. The argument has been going on for centuries :-] In the end, a translation is a translation, i.e. not native.
I think we need a more concrete example -- my feeling is that it's difficult to evaluate you situation from a distance. Wish you good luck anyway.
| | |
|
|
|
| Website localization is a complex process | Feb 8, 2012 |
GiuliaBa wrote:
What I want to find out is whether the translators have managed to adapt the original content to the target language so well that every aspect of the text (message, style, tone of voice, emotions, cultural background...) resonate with the audience, and it seems as if that text had been written in the target language, rather than translated.
I would like a Colombian user, for instance, to feel as if they had landed on a truly Colombian website, when they find us, rather than feel ours is just a foreign company's website more or less accurately translated from English.
Ok, now I understand what you mean. To "translate" a website into another language/culture so that it reads natural in the translated version requires a lot of work.
a) actually this work starts with the source text. Very often source texts represent the culture of the language they are written in. E. g. in a text about participation in a clinical study, a text written by a US author will mention/follow the US regulations for clinical trial, in Europe we do have different regulations - this means that any direct translation e.g. into German will still follow the US regulations and therefore will just not apply to a German reader.
So adaptation of your source text is vital to producing results that read natural in a foreign culture.
b) not providing the translators with the visual layout of the website is a recipe for failure. If the translator does not know if a segment is a heading, a bullet point, etc, he/she is missing important information that is required for optimal translation quality
c) not using independent reviewers/editors is another negative road to disaster
d) ask native in country reviewers to check your website content before it goes online
The total workflow is a lot more complex, but we are regularly running this type of projects and we have not yet found any corner we can cut without risking to produce unsatisfactory results.
| | | |
Phil Hand
China
Local time: 05:33
Chinese to English
| "seems as if the text had been written in the target language" | Feb 9, 2012 |
Is a really interesting concept.
I know you're looking for practical processes rather than theoretical discussion, Giulia, so first I would say: the joy of websites is that it's possible to track user behaviour in great detail. How long do users of the original English site spend on the front page, how often do they click on each link - this kind of data is a goldmine.
But to stray into slightly more theoretical ground:
I don't do localisation, so I haven't though... See more Is a really interesting concept.
I know you're looking for practical processes rather than theoretical discussion, Giulia, so first I would say: the joy of websites is that it's possible to track user behaviour in great detail. How long do users of the original English site spend on the front page, how often do they click on each link - this kind of data is a goldmine.
But to stray into slightly more theoretical ground:
I don't do localisation, so I haven't though about this much before, but just recently I translated a document English>Chinese which included a quote from a the client's CEO. Now, more than most text, a translated quote creates a level of unreality. In my own reading, it's not normally an issue: when I read a quote from a non-English speaker that's been translated, I just accept it as original. But in Chinese there's a lot of bad "Globish-style" translation from English, and people are quite used it. Presumably it sends a signal that readers use to help orient themselves.
On a website, there could be even more dissonance. If a website reader in a non-white country sees a picture of a white CEO, with a bubble and quote in perfect Chinese(/whatever language), how do they interpret that? In that kind of context, should a translator aim for "local style"?
Sorry, I'm sure there's lots of research and thinking on this that I should just go and read. It's all a new area to me! ▲ Collapse
| | | |
Abba Storgen (X)
United States
Local time: 16:33
Greek to English
+ ...
"a different feedback, from real users maybe, to understand whether the content is localised well enough "
About the average user: The translation may be perfect, and natural, but an average user may not like it. An average user may like material that's actually inferior. Average "real" users - you do not know how well they know the language, and what are the criteria and personal preferences.
The eyes of linguists are different than the eyes of the average user.... See more "a different feedback, from real users maybe, to understand whether the content is localised well enough "
About the average user: The translation may be perfect, and natural, but an average user may not like it. An average user may like material that's actually inferior. Average "real" users - you do not know how well they know the language, and what are the criteria and personal preferences.
The eyes of linguists are different than the eyes of the average user. The average reader of a website can't see the errors that I am trained to spot.
About editors: do what everyone in the free world does with editors (except the translation industry which is highly immature): ask the editors to explain their changes. Changes on original content without explanations constitutes "incomplete work". You can't send a letter to Boeing telling them "I don't like your airplanes but I'm not telling you why".
About bias and how to eliminate it: Remove the anonimity between the two cooperating parties. Make the translator and the editor "one team". There's no other industry that accepts anonimity among an engineer and a second engineer (as an example) etc. The anonimity in this industry is a left-over from Soviet style academia and it's counter-productive. When mature professionals work together as a team, they know what they' re doing and they' re focusing on quality (because they are risking both their reputations).
"I would like a Colombian user, for instance, to feel as if they had landed on a truly Colombian website, when they find us, rather than feel ours is just a foreign company's website more or less accurately translated from English."
About people being creative and perform more than one task: Which means, you are asking them not only to translate, but to also make it look 100% natural. Make sure you are dealing with mature and highly experienced readers/writers. And pay them accordingly. Consider that today's translators, even the most experienced, can't afford to give much more time on a translation that won't pay what they want (after taxes). It's a creative process, it's not "fix my shoes". Provide as much time as you can (won't work well under pressure), and do not assume that unhappy translators (tight deadlines, low rates etc) are as creative as the happy ones
If you assemble a small, experienced team, under good terms both for you and for them, you do not really need to hire third agencies that will try to invalidate your work. Less anonimity, less conflicts, that's it. Because if you need your content to be natural and well produced, you don't really want to get into the trap of conflicting subjective opinions... ▲ Collapse
| | | |
| No visual context? | Feb 9, 2012 |
Hello Giulia,
Perhaps I've misunderstood you but, in your second post, you say that "no visual context" is supplied to your translators, just single strings of words.
I'm wondering whether it might help them to produce more accurate and natural-sounding translations if they could in fact see the context.
I've sometimes been asked to translate material without the illustrations (absent photographs, absent screenshots of instructions, etc.) and it's certainly difficult to describ... See more Hello Giulia,
Perhaps I've misunderstood you but, in your second post, you say that "no visual context" is supplied to your translators, just single strings of words.
I'm wondering whether it might help them to produce more accurate and natural-sounding translations if they could in fact see the context.
I've sometimes been asked to translate material without the illustrations (absent photographs, absent screenshots of instructions, etc.) and it's certainly difficult to describe accurately something one has never seen.
Just a thought.
Best wishes,
Jenny ▲ Collapse
| | |
|
|
|
Tom in London
United Kingdom
Local time: 22:33
Member (2008)
Italian to English
| the other side | Feb 9, 2012 |
I have the opposite experience:
On a regular basis, sometimes several times a day, I translate very short strings of text for the English part of the website of a very well known large Italian travel company.
As is my normal practice I always do a good job and my intention is for their English website to read in a completely natural English and not for it to seem like a translation.
But when I check back by going to the website, I find that they regularly c... See more I have the opposite experience:
On a regular basis, sometimes several times a day, I translate very short strings of text for the English part of the website of a very well known large Italian travel company.
As is my normal practice I always do a good job and my intention is for their English website to read in a completely natural English and not for it to seem like a translation.
But when I check back by going to the website, I find that they regularly change my translations, making a complete mess of the website and thereby putting off potential anglophone customers.
Silly them. ▲ Collapse
| | | |
juvera
Local time: 22:33
English to Hungarian
+ ...
| Ask the real users! | Feb 10, 2012 |
GiuliaBa wrote:
This process works fine, but I would now like to gather a different feedback, from real users maybe...
What I want to find out is whether the translators have managed to adapt the original content to the target language so well that every aspect of the text (message, style, tone of voice, emotions, cultural background...) resonate with the audience, and it seems as if that text had been written in the target language, rather than translated.
I would like a Colombian user, for instance, to feel as if they had landed on a truly Colombian website, when they find us, rather than feel ours is just a foreign company's website more or less accurately translated from English.
Usability studies are generally conducted by another team before the product goes through the localization process. I am specifically looking for feedback about content and how well it has been adapted to the local language
You answered your own question. Only real users can tell you what real users think of the website. Sophisticated experts may not provide you with the right answer. The websites need a feedback tab or a simple questionnaire to ask the users of the particular website for their opinion on its contents, language, ease of understanding and use, etc. It would be advantageous for the owner of the website too.
| | | |
Login to reply/comment 
Translation news
