GPT can be used as a translation provider in Smartling. Similar to traditional machine translation providers, GPT needs to be set up as an MT Profile in your Smartling account. During this process, you will need to set up a fully flexible translation prompt. If desired, you can also customize a number of additional parameters to achieve the desired translation output and reduce post-processing.
The MT Profile for GPT can then be used as part of a translation workflow or with any of Smartling's account-wide machine translation options.
Supported models
All GPT models. Bring your own API key.
Supported Languages
GPT does not have a list of supported languages. It will attempt to generate a translation for any language you prompt it to. All high-resource languages are supported, while low-resource languages could result in poor quality translations.
Prerequisites
An API key for GPT from OpenAI or Azure is required. API keys for GPT are not provided by Smartling.
Limitations
Token limits
GPT models use tokenizers, which break the text down into units called tokens. This counts the total number of tokens (both input and output tokens). Input token is everything that is fed into the model (that includes the prompt and the input source string.) The output token is the number of tokens that is returned from the model.
There is no universal tokenization model, which means that the text could be broken down based on words, characters or character sequences. Therefore, the number of translated words in Smartling will likely differ from the token usage.
Your GTP instance comes with a monthly token limit. If you exceed this limit, an error flag will be shown in Smartling, and your GTP profile will cease to complete translations.
The larger the source content size and the higher the number of target languages, the higher the risk of reaching the token limit will be.
Tip: To test tokenization, test your prompt in OpenAI's tokenizer.
Rate limits
If GPT is used as a translation provider, three types of rate limits restrict the number of translations that can be processed:
- Requests per minute
- Requests per day (only within the first 48 hours)
- Tokens per minute
If any of these rate limits are exceeded, translations can no longer be completed with GPT and an error message will be shown.
For more information about rate limits can be found in OpenAI and Azure documentation.
Setting up GPT as a translation provider
- Go to Account Settings > Machine Translation Settings
- Navigate to Profiles and click Create MT Profile.
- Select GPT (Azure) or GPT (OpenAI) based on the provider you are using for the API key.
- Insert a name for the new translation profile, which will appear in your list of MT Profiles in Smartling.
- Do one of the following:
- For GPT (OpenAI), enter the API key from OpenAI.
- OpenAI-Organization' header value: For users who belong to multiple organizations, you can pass a header to specify which organization is used for an API request. Usage from these API requests will count against the specified organization's subscription quota. For more information, read Open AI's documentation.
- For GPT (Azure), enter your Azure GPT access token, the name of your Azure OpenAI.
- Resource and the deployment ID which you chose when you deployed the model.
- Optionally, the Azure API version to use for this operation can be provided.
- For GPT (OpenAI), enter the API key from OpenAI.
- Select the GPT model which you would like to use.
- Example: gpt-3.5-turbo
- GPT model - max supported tokens: Max supported tokens of the GPT model
- This is an optional parameter where you can specify the max number of tokens your model can support per request. This is useful if you have a custom model or the model name is not obvious from the deployment ID. If no value is provided, the default is automatically set to 4K.
- To test tokenization, test your prompt in OpenAI's tokenizer.
- Set the translation parameters
- Write a translation prompt
- Test the integration
- Click Save.
Translation parameters
To further fine-tune the translation output, you can customize the following, optional parameters.
If you do not specify any values for these parameters in your MT Profile, Smartling's default values will be used.
Temperature
The temperature determines the output creativity. The lower the temperature, the more direct GPT's translation will be. A higher temperature can be used to achieve a more creative translation output, however the lack of repeatability makes it difficult to reproduce the same translation output. A high-temperature value also increases the probability of the GPT model to stray from the given context.
By default, the temperature is set to "0", to lower the risk of hallucinations.
Top P
Similarly to autocomplete options in other systems, GPT can determine which word is the most likely to follow the previous tokens in a sentence. Top P refers to the probability of a certain word being chosen in a particular context.
The default value is set to "1", to ensure that more probable words are used to complete the translations.
Presence penalty
This parameter penalizes tokens which appear repeatedly, by encouraging the model to include a more diverse range of tokens in the generated text. A high Presence Penalty will result in the model being more likely to generate tokens that have not yet been included in the output text and it can be used to prevent topic repetition.
The default value is set to "0", to allow for the translations to adhere to the source content.
Frequency penalty
This parameter penalizes tokens based on how many times they have already appeared in the text. The more frequently a token has appeared, the more it will be penalized. Therefore, the frequency penalty parameter can be used to avoid word repetitions and to encourage the use of synonyms.
The default value is set to "0", to allow for the translations to adhere to the source content, even if it contains word repetitions.
Translation prompt
Unlike with traditional MT providers, it is not sufficient to enter the API key, but you also need to set up a translation prompt. Using a customized translation prompt allows you to provide a tailored translation output based on your requirements.
We recommend using the word "translate" in the prompt to instruct GPT to provide a translation.
Additionally, the following two placeholders need to be used in the prompt: {sourceLanguage} and {targetLanguage}.
- {sourceLanguage} : refers to the source locale as specified in your Smartling project
- {targetLanguage} : refers to all target locale(s) as specified in your Smartling project
Example prompt: Translate from {sourceLanguage} to {targetLanguage}. The tone should be formal and friendly.
A previous version of Smartling's GPT integration required the use of the placeholder {sourceContent}. This is no longer needed. If your translation prompt still contains the placeholder {sourceContent}, we would recommend removing it. Please ensure that the updated prompt will follow a clear sentence structure once the placeholder has been removed.
Testing the integration
Once all values have been entered, the GPT translation prompt can be tested by selecting a source and target language and clicking Test Integration.
Note: There is a 250-character limit on the source text field. The translation text field cannot be edited.
Considerations
Compared to traditional machine translation providers, Large Language Models like GPT offer more flexibility, however they also present a number of challenges, which should be considered.
Fallback translation provider
When GPT is used on the Translation step of a workflow in Smartling, it is strongly recommended to set up an alternate MT profile and/or a fallback method. The alternative MT provider or method will be used if the translation with GPT should fail.
If GPT's services get a 500 or 503 error code, Smartling will retry to send your content for translation 3 times.
For 429 errors, Smartling stops trying immediately and tries the fallback translation provider, if one is provided. If no fallback is provided, Smartling retries with exponential backoff (retrying with an increased time period, up to 4 hours). This could take the Job days to complete, depending on how many tokens need to be translated.
If the content cannot be successfully translated at this point, the fallback translation method will be used instead.
If no fallback translation method is set up, Smartling will retry to send content for translation with GPT for 7 days.
Glossary term insertion
The insertion of glossary terms into translations provided by GPT is currently not supported.
Hallucinations
GPT models can hallucinate, which means that additional material which wasn’t requested is added to the translation output. A higher temperature value can generally lead to more hallucinations.
Tags and placeholders
At times, HTML tags and placeholders may be handled incorrectly by GPT. A post-editing step may be required to ensure the correct placement of HTML tags and placeholder.
Prompts
Prompts are clear instructions to the LLM on what you want it to do. You can include as much direction and contextual information as necessary, such as formality, tone and style preferences, just be aware of your model's token limits.
GPT doesn't know how to say "I don't know" to any given prompt. As a result, it won't always perform the action instructed in the prompt, but instead, hallucinate. A way to prevent this is to include instructions on what you want it to do if it is unsure.
Furthermore, results generated from prompts can be inconsistent and take longer to generate, depending on the prompt.
Quality
GPT does not translate lower resource languages very well. Fine-tuning the translation quality with linguistic assets is not possible, compared to MT custom trained engines. Overall, translation quality is much better in MT custom trained models. If you are translating with GPT, we recommend including a post-translate edit and review step in the translation workflow.