As a Content Owner or Localization Project Manager who is responsible for translation quality, you need the right tools to help linguists produce high-quality translations. The process of evaluating and providing feedback on translations is instrumental to enriching quality, and to the evolution of your localization standards.
Smartling provides a range of Quality Features to encourage translation quality - the most versatile being Linguistic Quality Assurance.
What is LQA?
Linguistic Quality Assurance (LQA) is a process by which human linguists evaluate translations against a schema of objective errors. Errors are grouped into categories. Any category with high error counts helps you identify focus areas that need improvement. By evaluating translation quality in an objective scoring schema, it eliminates bias and allows you to make data-driven decisions to improve your localization process. Examples of objective translation errors could include, but are not limited to:
- Any inaccuracy or inconsistency with your Glossary, Style Guide, Translation Memory
- A disregard of your Quality Checks
- A disregard of instructions or information shared in an attachment
- Inadequate spelling or grammar
- A translation taken out of context
What makes LQA different?
While Quality Checks and standard review steps help to ensure your translations move through the workflow without mistakes, they are rigid and open to interpretation. LQA takes a step back and assesses the overall standard of each string against a catalog of errors.
Strings are essentially reverse-graded, with the lower the score equating to the higher the standard. The numbers generated for each error in each category accumulate to highlight specific areas in your translations that need action and the impact of that action; whether it's an update to style guides, glossaries, or training for the team.
Who performs LQA?
LQA can be performed by internal Localization teams or outsource LSPs. As it integrates into a workflow, any method of translation can be evaluated with LQA, including human translations, machine translations or machine translations with edits.
How it works
Content Owners create an LQA schema of errors to evaluate translations against. Once published, they can then configure the schema to a workflow that translations should be evaluated in. This workflow should contain a review workflow step-type specifically for LQA.
Translations are evaluated in the CAT Tool on the LQA review step. Each string is evaluated against the catalog of errors within the LQA schema. The evaluation recorded is accessible to Content Owners in a range of LQA reports.
LQA Schemas
To make evaluations as impartial as possible, a categorized list of errors is required. A translation quality error is a problem in the translation that can be objectively agreed on by stakeholders.
Each error is "rated" with a severity level. The industry standard is neutral-minor-major-critical, according to the MQM framework. Each severity level is given a numerical weight to calculate the quality score, or quality metric.
When setting up LQA, you have the choice to create a new schema, or use one of the schema templates. Smartling provides two industry standard schema templates compatible with MQM you can use to evaluate translations against. For more information, read our documentation on Schema Templates for LQA.
Whether you choose to create your own schema or use a template, we recommend using the same schema in all evaluations. This will ensure all translations have been evaluated against the same catalog of errors.
Workflows
LQA is enabled on a workflow step, so once the schema is created and published, LQA should then be configured to a workflow step of your choice. Although not a requirement, it is recommended that a workflow step be added specifically for LQA. Alternatively, you can enable LQA on any post-translation workflow step.
It is important to note that translation resource users can only access the LQA dialog to record errors and arbitrate on workflow steps they are assigned to that has enabled LQA. This means that if LQA is only enabled on one workflow step, they cannot access the LQA dialog to view errors or arbitration when and the content moves out of the step. For this reason, we recommend enabling LQA on multiple post-translations steps, or an entire LQA workflow.
Errors
When a reviewer records an error, the severity value will be the default severity unless they change it. When they record no errors on a string or skip the string entirely, the translation is evaluated as having no error.
If you use the MQM templates to evaluate translations, the severity count of each error recorded cumulates to the quality score of the translations. The quality score of translations can be found in Reports > Linguistic Quality Assurance Dashboard.
Smartling captures the density of errors recorded per 1,000 words in the Linguistic Quality Assurance Error Density Report.
String with LQA errors in the CAT Tool
String with LQA errors in Review Mode
Arbitration
Translators have the opportunity to dispute an error or to justify why the translation provided is appropriate within a certain context. This process of disputing and reviewing the disputes to determine the final judgment is called "arbitration". In some cases, you may have a resource assigned just to review and resolve the arbitration.
Any error can be arbitrated upon evaluation by simply clicking on the saved error and adding an arbitration comment. This creates a dialog under the disputed error. The arbitration comments are counted for each error under the Arbitration comment column.
Content Owners can further dispute the arbitration of the error, as well as view all errors and arbitration recorded on any Job under Reports > Linguistic Quality Assurance Errors & Arbitration.
Viewing Changes Made to Translations
An error is recorded on the translation that was submitted to your workflow step. You can edit a translation before or after you record an error. You can then view the difference between the unedited translation (the translation the error is recorded on) and the edited translation (the translation that you will submit to the next step of the workflow), by clicking the downward arrow beside Translation in the error dialog.
The change (or diff) is highlighted for your attention.
Find Strings Reviewed Under LQA
You can filter for strings that have been reviewed under LQA in the Strings View. Use the LQA filter to find strings that have been recorded as having errors, or having no errors.
Note that strings that have been submitted without recorded errors (i.e. submitted or skipped) are recorded as strings with "No Errors" by default.
You can also use this filter to find strings that have not been reviewed under LQA.
Additional Resources
For information on how to set up LQA, read our documentation on Getting Started with LQA.
For information on how translations are evaluated with LQA, read our documentation on Evaluating Translations with LQA.
Video Tutorial: Linguistic Quality Assurance in Smartling
Timestamps:
What is Linguistic Quality Assurance (LQA)? 00:08
How to set up an LQA process in Smartling 01:10
Step 1: Create an LQA Schema 01:45
- Choose a severity format 03:14
- Add error categories 03:51
- Add error types 04:27
- Publish your schema 05:13
Step 2: Enable LQA as part of your workflow 05:28
How your Reviewers record LQA errors 06:30
LQA and Issues: What is the difference? 08:34
Step 3: Run & analyze the LQA report 10:04
Arbitration: What is it and how can it be enabled? 11:31
Adding arbitration comments 13:55
LQA Errors and Arbitration report 14:46
Help & Support 15:52