Translating with Google Gemini (Vertex AI)

Google's Vertex AI platform allows you to translate using the latest Gemini models. 

Google's Vertex AI can be used as a translation provider in Smartling, as part of a translation workflow or with any of Smartling's machine translation integrations.

Requirements

You will need an API key in the form of JSON credentials for Google Vertex AI. You can either bring your own key or ask your Smartling Customer Success Manager to provision a Smartling-provided key for translating with Gemini models.

Supported models 

Smartling supports translation using Gemini models version 2.0 and above via the Vertex AI API. This includes support for fine-tuned models.

Supported languages 

Please refer to the Vertex AI documentation for the full list of languages supported by Gemini models.  

Limitations

Token limits

LLM models use tokenizers to break text into units called tokens. Token usage includes both input and output tokens. Input tokens refer to everything fed into the model, including the prompt and the source string. Output tokens refer to the number of tokens returned by the model.

There is no universal tokenization method. Text can be broken down by words, characters, or character sequences, depending on the model. Therefore, the number of translated words shown in Smartling will likely differ from the token usage. 

Each model has a maximum token limit that applies to both the input prompt and the generated response. Please see the Vertex AI documentation for specific token limits. Vertex AI also enforces quotas and usage limits. If you are using a Gemini model, Google uses a Dynamic Shared Quota (DSQ) system. 

In addition to the length of the translation prompt sent with each request, larger source content and a greater number of target languages increase the risk of reaching the token limit. You can estimate your token usage and billable character count for a prompt using the Vertex AI API. More information on this is available here. 

Rate limits 

Rate limits control the number of requests or the volume of traffic allowed within a specific time period. Translation providers like Google Vertex AI define rate limits for their models. You must adhere to these limits to maintain successful translations.

Please refer to the Vertex AI documentation for more detailed information on rate limits.

Info: If a token or rate limit is exceeded, an error will appear in Smartling on the Translation Profiles page. Smartling will retry using the LLM Profile until a translation can be produced successfully. If the overall or monthly token limit has been reached, the LLM Profile may stop generating translations.

Setting up Google Gemini (Vertex AI) as a translation provider

To translate using Vertex AI, you first need to obtain an API key in the form of JSON credentials from Google.

Once this credential has been stored in Smartling, it can be used to create an LLM Profile. Within the profile, you can configure your translation prompt and set various translation parameters.

Once the LLM Profile is created, Vertex AI can be used as part of a translation workflow or with any of Smartling's machine translation integrations. 

Step 1: Setup in Google 

First, enable Google Vertex AI and ensure that your preferred model is deployed. You will need to obtain an API key from Google, which provides access to the model. 

Step 2: Add provider credentials in Smartling

You will need to store your Vertex AI credentials in Smartling if you are using your own key in order to use a Gemini model as a translation provider. 

Info: If you are using a Smartling-provided key, you do not need to complete this step. Your Customer Success Manager will provision your LLM profile with Smartling credentials.

1. From the top navigation of your Smartling dashboard, access the AI Hub.
2. Navigate to the Credentials section.
   
3. Click Add Provider Credential.
   
   This opens a modal where you can save your provider key. Please enter the following information: 
   • MT or LLM Provider (required):
     From the dropdown, select Google Gemini (Vertex-AI). 
   • Credential Name (required):
     Enter a name for your provider credential to help identify it. Ideally, this nickname should reflect which team owns the credential, which provider is used, and any other valuable information to help you identify this provider key.
     For example: Gemini Marketing Model
   • Credential Description (optional)
   • Project ID (required): A unique identifier for your project in Google. It is typically displayed below the project name in Google Cloud Console. 
     Example:
     My Project
     Project ID: my-project-id-123456
   • Location (required): The location of your data (i.e., the Vertex AI datacenter location). 

   • JSON credentials (required): Copy and paste the entire JSON credentials file into this field, including the brackets. See the instructions below for generating a JSON credentials file.

     Ensure that your credentials are associated with the Vertex AI User role in the Google Cloud Console. This role is required to use Vertex AI for Gemini models.

     

     • In the Google Cloud Console, navigate to Service Accounts > select the relevant project > select the account. 
       
     • Go to the Keys tab > click Add key > Create new key.  
       
        
     • Choose JSON and click Create. A file will be downloaded—this is your credentials file. Copy and paste its contents into the Credentials field.
       
   • Test Credential 
     Once you have provided all required information, please click "Test Credential" to check if the provider credential is fully functional.
     • A success message will be displayed if the credential is working correctly. You can now proceed to associate this credential with an LLM Profile.
     • In case of any issues, an error message will be displayed. Please check if a valid provider credential was obtained, and if the JSON credentials and other details were entered correctly.
4. Click Save to create the credential.

Step 3: Use the provider credentials in an LLM Profile

Once the provider credential has been saved and tested successfully, it can now be used to create an LLM Profile. An LLM Profile allows you to configure your translation prompt, as well as additional preferences to further customize the translation output.

1. From the top navigation of your Smartling dashboard, access the AI Hub.
2. Navigate to the Translation Profiles section.
   
3. Click Create Profile and select LLM Profile (RAG).

Tip: Follow our instructions on How to create an LLM Profile.

• Select Google Gemini (Vertex-AI) as your LLM Provider.
• Enter your provider and token details.
  • Under Gemini model, enter the exact name of the model including the version (e.g., gemini-2.0-flash-lite-001). If you are using a fine-tuned model, enter the entire URL path to the relevant model in your Google Cloud project.
• Adjust parameter details (optional): 
  • You can customize optional translation parameters to adjust the translation output. If you do not specify any custom parameters in your LLM Profile, your model's default values will be used. For more information on the available translation parameters, please see Translation Parameters for LLM Translation.
• Configure your translation prompt:
  • Unlike with traditional MT providers, you need to set up a translation prompt to instruct the LLM on how to translate your content. Using a customized translation prompt allows you to provide a tailored translation output based on your requirements. Please follow our instructions on how to Configure your translation prompt.
  • Optionally, Prompt tooling with RAG allows you to augment your translation prompt with contextual information extracted from your linguistic assets. By using RAG technology (Retrieval-Augmented Generation), your translation prompt is automatically injected with highly customized translation data, allowing the LLM to better understand your translation preferences and to produce a more tailored translation output.
  • Once all values have been entered, the translation prompt can be tested to ensure that it produces the desired outcome. Please follow our instructions for testing the prompt.

4. Click Create to create the LLM Profile.

Step 4: Use the LLM Profile in an MT workflow or MT integration 

Once the LLM Profile has been created, it can be used to translate your content, either within the Smartling platform (using an MT workflow or MT suggestions in the CAT Tool), or through one of Smartling's MT integrations to display machine translations directly where needed.

Tip: For more information, see Using the LLM Profile to translate your content.

Enable AI-Enhanced Glossary Term Insertion (optional)

AI-Enhanced Glossary Term Insertion is supported for translations generated by Google Gemini (Vertex AI) when used in a translation workflow or Smartling Translate. Glossary Term Insertion is not supported when Gemini is used for an MT integration, such as MT in the CAT Tool, MT API or the GDN.
 

Considerations

Compared to traditional machine translation providers, Large Language Models (LLMs) like Google Gemini provide more flexibility. However, they also come with a number of challenges that should be considered.

Fallback translation provider

When using Vertex AI in the Translation step of a workflow in Smartling, it is strongly recommended to configure an alternate MT profile and/or a fallback method. This backup will be used if translation with Vertex AI fails.

If Vertex AI returns a 500 or 503 error, Smartling will retry the request to send your content for translation up to three times.

For 429 errors, Smartling stops trying immediately and tries the fallback translation provider, if one is provided. If no fallback is provided, Smartling retries with exponential backoff (retrying with an increased time period, up to 4 hours). This could take the job days to complete, depending on how many tokens need to be translated.

If the content cannot be successfully translated at this point, the fallback translation method will be used instead.

If no fallback is configured, Smartling will continue retrying Vertex AI for up to 7 days.

Tags and placeholders

At times, HTML tags and placeholders may be handled incorrectly by LLMs. A post-editing step may be required to ensure the correct placement of HTML tags and placeholders.

Prompts

Prompts are instructions that guide the LLM on how to translate. You can include as much direction and context as needed such as tone, formality, or brand style. However, do always keep token limits in mind.

Because LLMs cannot indicate uncertainty, they may hallucinate instead of failing gracefully. To reduce this risk, include instructions in the prompt for situations when the model is unsure.

Prompt results may also vary in quality and response time, depending on the complexity of the instructions.

Quality

LLMs generally perform poorly when translating low-resource languages. They cannot be fine-tuned using linguistic assets like custom-trained MT engines can. Overall, custom-trained MT models tend to produce higher-quality translations.

LLMs can hallucinate, meaning they can generate content that was not requested. 

If you are using an LLM for translation, we strongly recommend including a human editing or review step in the workflow.