Custom MT engine training is a service offered by Smartling that can help significantly enhance the Machine Translation output quality and tailor it to your stylistic and terminology preferences.
Generic NMT engines are domain-agnostic, which means that they can sometimes produce poor results on industry-specific language. By feeding an NMT engine with domain-specific vocabulary, style and usage patterns, it can produce more relevant and fine-tuned results.
How does it work?
Your Translation Memory and/or Glossary (including “Do Not Translate” terms) are first curated and optimized, and then fed into an NMT (neural MT) engine machine learning algorithm (Google AutoML). This training process helps achieve a Machine Translation output that is less generic and better represents your industry and brand tone of voice.
Smartling offers MT engine training as a full-service solution: From the preparation of your Linguistic Assets and curation of the most suitable data sets, to the training and subsequent evaluation of the MT engine - our expert team and LanguageAI platform help you get the most value at every step.
Steps to training an MT engine
Step 1: Reach out to your Customer Success Manager
For pricing information or to get started, please reach out to your Smartling Customer Success Manager. Our expert team will then discuss the exact process and next steps with you.
Step 2: Optimization of your Linguistic Assets
Depending on your preferences, MT engine training can be based on your Glossary, your Translation Memory or both. These assets need to contain a sufficient amount of reliable and domain-specific data (at least 6,000 translation units, with a recommended minimum of 10,000 translation units, or roughly 150,000 words).
Before they can be used to train an MT engine, your Glossary(ies) and Translation Memory(ies) need to be optimized: The more accurate the translations that the engine is trained on, the better the results. We don’t recommend training an MT engine with assets that are inaccurate, or otherwise unreliable.
MT engines prefer clear, unambiguous entries in a TMX format.
Incorrect, unreliable and inconsistent data needs to be identified and either removed or fixed.
For example, the following inconsistencies will be identified and flagged during this process:
- Translation units where there is a mismatch in tags, numbers, quotation marks, symbols or casing,
- translation units where the source text has multiple target translations,
- translation units where the target text has multiple source translations,
- translation units containing repeated words in either the source or translation, and
- duplicates.
Smartling’s AI-powered self-healing features and/or our review services by human linguists can be used to repair and clean the affected translation units.
Step 3: Data curation and engine training
The optimized TMX files are then curated and split into various data sets for the training process.
- The train set enables the model to generalize its learning from the training data to make accurate predictions on new, unseen data. Once the training is complete, the model becomes capable of making predictions based on the patterns it learned from the training dataset.
- The tune set helps the model to optimize the hyperparameters of the machine learning model to improve its performance.
- Lastly, the test set is a separate portion of the original dataset that was not used during training. It contains the "correct translations" against which the model’s performance is measured. The closer the trained engine matches the "correct translation", the better the engine will perform in production when faced with similar data for future translation.
Since effective MT training requires a highly granular approach to uploading linguistic content, this data selection and curation is crucial to achieving the desired results.
Step 4: Engine evaluation
To provide evidence of the improvements gained by using the trained engine over a generic engine, Smartling evaluates the results based on a range of quality scores:
- Four different textual scoring algorithms are used to measure the predicted edit effort compared to human translations (without considering the semantic meaning):
- BLEU (BiLingual Evaluation Understudy) is used to evaluate raw MT.
- TER (translation edit rate), edit distance and WER (word error rate) are used to evaluate MTPE production.
- Even a small number of edits can have a big impact on the actual meaning of a sentence. Therefore, Smartling also uses four different semantic scoring algorithms (such as Laser, Labse, BERT and Bleurt) to evaluate the accuracy of the translation output when it comes to their actual meaning.
Step 5: Retrain and maintain MT engines
As your business grows, your custom MT engine needs regular updates with new terminology, messaging, and style guidelines to continue producing accurate outputs.
Ideally, your MT engine should be retrained every 6-12 months, and the performance should be continually monitored.
Which MT engines support custom training?
Smartling offers custom training for Google AutoML. A list of supported languages is available here. For custom engine training, English must be used as either the source or target language.
Trained MT engines can be combined with Smartling’s AI-enhanced workflow options, including AI-Powered Human Translation. Using an intelligent scoring mechanism, the best possible Machine Translation is selected for each string. If a trained MT engine is available, it will typically provide the best initial translation for the vast majority of your content.
When a trained engine is used as part of a custom MT or MTPE workflow, it can also be integrated with Smartling’s AI Translation Toolkit.
Expected results
Training an MT engine can have a significant impact on aspects like translation quality and brand alignment. Trained engines achieve higher average BLEU scores compared to their untrained counterparts, and their outputs also require fewer edits than generic engines.
If a step for human post-editing is used, the higher quality MT output allows translators start with an improved first translation.
In Smartling, trained MT engines can be paired with innovative AI features (such as the AI Translation Toolkit) for best results. Dynamic features like AI Fuzzy Match Repair, AI Formality Adjustment and Glossary Term Insertion use Large Language Models to correct grammatical errors and ensure brand consistency, leading to even higher quality MT translations compared to generic engines or GenAI on its own.