Tutorials

Tutorial 1 - Translation Jobs and Workflows

Learn about key Smartling concepts by using the API with translation jobs

Content translated in Smartling is organized into groupings known as ‘jobs’. Jobs contain a set of content to be translated, a list of target languages, and additional attributes such as due date. Translation agencies typically quote the cost and expected delivery date of a body of translation work based on a job. As a result, API integrations generally should package content for translation into reasonably sized jobs.

In this tutorial, we’ll create a job and add some content to it using the API; then translate that content in Smartling; check the translation status of the job via API; and finally download the completed translations using the API.

Here are the steps we'll follow:

  1. Authenticate
  2. Create translation job
  3. Create job batch
  4. Upload files to be translated
  5. Check translation status
  6. Review translation workflows
  7. Authorize job for translation
  8. Translate
  9. Re-check translation status
  10. Download translated files

We’ll play a number of different roles along the way: primarily, that of the integration developer, but also the localization project manager, and will touch on the translator role. Having an understanding of these different perspectives is very helpful for building an effective integration with Smartling.

Prerequisites

Make sure that you have the following in place before starting:

  1. Smartling API setup. If you're missing any of the below, please see the Getting Started tutorial for instructions on how to obtain them:
    • A test project in Smartling, with SmartMatch disabled
    • Project-manager access to the test project
    • API credentials for the test project
  2. Python requests package installed (along with Python itself). This is required to run the sample code below.

Sample code

Sample code in Python using the requests package is included below. This code can be entered or copied directly into the Python interactive interpreter, or gradually added to a script. If using the Python interpreter, make sure keep the same interpreter running so that the variables carry over between steps. 

Step 1. Authentication

The first step in using the API is to authenticate. The authentication API call requires your API credentials, i.e., the user identifier and user secret. For security reasons, it’s recommended to make these values available as environment variables that can be accessed from within your code, rather than storing them directly in your code.

First open a terminal window (Mac/Linux), or command prompt (Windows), and enter the commands below, replacing PROJECTID, USERIDENTIFIER and USERSECRET with your values:

export DEV_PROJECT_ID='PROJECTID'
export DEV_USER_IDENTIFIER='USERIDENTIFIER'
export DEV_USER_SECRET='USERSECRET'

set DEV_PROJECT_ID=PROJECTID
set DEV_USER_IDENTIFIER=USERIDENTIFIER
set DEV_USER_SECRET=USERSECRET

It’s helpful to store these commands in a script that can be run when required. If you do this, make sure to exclude the script from your source control system in order to avoid exposing your user secret.

The simplest way to try the code samples below is to enter them into the Python interactive interpreter. You can start the interpreter by entering the command below in the terminal window where you set the environment variables:

python
 or
python3

py

This launches the interpreter presenting a prompt >>> where you can enter python code. Note you may need to press Enter one or more times until the >>> prompt reappears after pasting the code samples into your interpreter window.

Once the environment variables are in place, we can read them into variables in our code:

import sys
import os
import requests
import json

# Read authentication credentials from environment
user_id = os.environ.get('DEV_USER_IDENTIFIER')
user_secret = os.environ.get('DEV_USER_SECRET')
project_id = os.environ.get('DEV_PROJECT_ID')

And then use them in the authentication API call:

# Authenticate
api_url = 'https://api.smartling.com/auth-api/v2/authenticate'
api_parameters = {
    'userIdentifier': user_id,
    'userSecret': user_secret
}
api_response = requests.post(api_url, json = api_parameters)
        
# Store access token for use in subsequent API calls    
if api_response.status_code == 200:
    access_token = api_response.json()['response']['data']['accessToken']
    refresh_token = api_response.json()['response']['data']['refreshToken']
else:
    print(api_response.status_code)
    print(api_response.text)

Refreshing access tokens

Access tokens expire after about 5-10 minutes and need to be ‘refreshed’. If the access token expires between some of the tutorial steps below you’ll see an authentication error response. If that happens, simply re-run the authentication code above, then retry the step you’re on.

For details on how authentication should be managed in a production integration, see the Authentication guide on managing access tokens.

Step 2. Create translation job

Now that we’re authenticated, we need to create the translation job. To do this, we use the Create job endpoint, passing the job name as a parameter:

job_name = 'Test Job 1'
api_url = 'https://api.smartling.com/jobs-api/v3/projects/' + project_id + '/jobs'
api_request_headers = {'Authorization': 'Bearer ' + access_token}
api_parameters = {
    'jobName': job_name
}
api_response = requests.post(api_url,
                             headers=api_request_headers,
                             json=api_parameters)
if api_response.status_code == 200:
    job_uid = api_response.json()['response']['data']['translationJobUid']
    print('job_uid = ' + job_uid)
else:
    print(api_response.status_code)
    print(api_response.text)

(If you run this code several times, you might get an error response indicating that a job with that name already exists. If this happens, simply change the value of ‘job_name’ above and retry. Alternatively, rename or delete the job in Smartling.)

Once the job is successfully created, you’ll be able to see it represented in the Smartling dashboard:

  1. Login to Smartling at https://dashboard.smartling.com
  2. If not in the test project, navigate to it by either clicking on it from the Account dashboard, or by selecting it it from the Projects menu in the upper left
  3. Click on the Jobs tab

You should see the new job listed:

newjobnocontent.png

In the ‘Progress’ column, you can see that the job currently has no content. We’ll add the content to it below.

Step 3. Create job batch

Although it’s possible to add content directly to a job via API, the recommended approach is to use ‘job batches’. Job batches simplify the process by handling the asynchronous elements, such as waiting for uploaded files to be processed.

For this example, we’ll use the Job batches V2 API. The API has two high-level steps:

  1. Create the batch, supplying the URIs of the files that will be uploaded to it
  2. Upload the files that were specified when creating the batch.

Here is the code to create our job batch. It specifies the URIs for two test files that we’ll create and upload in the subsequent step. (Remember that you might need to re-authenticate if more than five minutes have passed since you ran the authentication step above.)

file_uris = [
    job_uid + '/test-files/site-navigation.json',
    job_uid + '/test-files/products.json'
]

api_url = 'https://api.smartling.com/job-batches-api/v2/projects/' + project_id + '/batches'
api_request_headers = {'Authorization': 'Bearer ' + access_token}
api_parameters = {
    'authorize': False,
    'translationJobUid': job_uid,
    'fileUris': file_uris
}
api_response = requests.post(api_url,
                             headers=api_request_headers,
                             json=api_parameters)
if api_response.status_code == 200:
    batch_uid = api_response.json()['response']['data']['batchUid']
else:
    print(api_response.status_code)
    print(api_response.text)

URIs uniquely identify files within a Smartling project. They can be set to anything, but it usually makes sense for the URI to reflect the actual directory path and name of the file. We’ll create some test files to upload in the next step. For now, we just need the URIs of these files.

In order to avoid potential confusion in testing, we’re prepending the job ID to the URI. This ensures that each job will get its own copy of the file in Smartling in case you add the same file to multiple jobs in your testing. This would not typically be done in an actual integration.

Step 4. Upload files to be translated

The API endpoint we need for uploading files to the batch is Upload file to a batch. It takes the file URI, which must match one of the URIs specified when the batch was created, along with the file type and list of target locales for this file.

First, we need to create some test files. We’ll use a directory structure that matches the URIs we specified in the previous step.

Create a sub-directory named ‘test-files’ in the directory where you started the Python interpreter; then add two JSON files to it named site-navigation.json and products.json containing the content shown below:

test-files/site-navigation.json

{
    "app.home.label" : "Home"
}

test-files/products.json

{
    "product.ID1234.name" : "Some product name"
}

Here is the code for uploading these files to the batch we created in the previous step. You might need to adjust the LOCALE_LIST to match the locales defined in your test project.

FILE_LIST = [
    'test-files/site-navigation.json',
    'test-files/products.json'
]
LOCALE_LIST = ['fr-FR']

api_url = 'https://api.smartling.com/job-batches-api/v2/projects/' + project_id + '/batches/' + batch_uid + '/file'
api_request_headers = {'Authorization': 'Bearer ' + access_token}

for file_name in FILE_LIST:
    file_uri = job_uid + '/' + file_name
    api_parameters = {
        'fileUri': file_uri,
        'fileType': 'json',
        'localeIdsToAuthorize[]': LOCALE_LIST
    }
    file_param = {
        'file': open(file_name, 'rb')
    }
    api_response = requests.post(api_url,
                                 headers=api_request_headers,
                                 data=api_parameters,
                                 files=file_param)
    if api_response.status_code == 202:
        print('Uploaded file ' + file_name)
    else:
        print(api_response.status_code)
        print(api_response.text)

 

After running the above code, we can check the job in Smartling again and see that it now contains content:

 

newjobwithcontent.png

 

And if we click into the job and then select the job's Files tab, we should see the files that were added to the job:

 

jobfiles.png

 

If you don’t see the files uploaded to the job, you can use the code below to check the status of the batch, which should give information about what went wrong. This could happen if, for example, there was some problem with the file format.

api_url = 'https://api.smartling.com/job-batches-api/v2/projects/' + project_id + '/batches/' + batch_uid
api_request_headers = {'Authorization': 'Bearer ' + access_token}
api_response = requests.get(api_url, headers=api_request_headers)
print(api_response.status_code)
# pretty-print to see full response
print(json.dumps(api_response.json(), indent=4))

 

Once you make any required fixes, you might need to re-run batch-creation from the previous step in addition to the file uploads from this step.

File types

Smartling supports a variety of different file types in addition to JSON. The way a file is parsed, for example how keys and placeholders are identified, varies depending on the file type. Parsing behaviour can be customized through the use of parsing directives which can be included in the file itself or supplied as API parameters. For more information on this, see Supported File Types.

Languages and locale IDs

When uploading files to the job batch, we include the target locale IDs for the file as a parameter to the upload.

Locale IDs are codes that identify a specific language variant. For example, the locale ‘fr-CA’ identifies the variant of French spoken in Canada. Projects in Smartling are configured with whatever locales are expected to be required for content translated in that project. For example, a Smartling project for translating your website content would be configured with whatever locales your website is translated into. 

When content is sent for translation in Smartling, the target languages (i.e., the languages the content is translated into) must be chosen from the list of locales configured in the project. If required, additional locales can be added to the project. You can see what locales are defined in your project by navigating to Project Settings > Languages in the Smartling Dashboard, or by using the Get project details API call.

Integrations often need to perform a mapping between the language code used in the content platform and the locale ID used in Smartling. For example, the language in your website content platform might be identified as “French” or “fr”, but in Smartling might be configured as “fr-FR”, i.e., French (France).

Uploading files

Note that a file is uploaded only once regardless of the number of target languages it is to be translated into.

Step 5. Check translation status

Translation takes time; even machine translation, if the volume is large, takes time to process and may encounter errors requiring retries and fallback processes. In addition, translation work often must be edited or reviewed before being considered complete. So we need a programmatic way to know when translations are completed and ready for download. We have a number of different options for this:

  • Callbacks (webhooks) can be sent when a file is completed in a particular locale or when a job is completed. See Webhooks and Callbacks.
  • Status API endpoints (e.g., Status of file for each locale) can report whether a file is completed in a particular locale, or whether a job is completed
  • Recently completed endpoint provides a list of recently completed work.

For now, we’ll just check the status of the job using the Get progress of job endpoint:

api_url = 'https://api.smartling.com/jobs-api/v3/projects/' + project_id + '/jobs/' + job_uid + '/progress'
api_request_headers = {'Authorization': 'Bearer ' + access_token}
api_response = requests.get(api_url, headers=api_request_headers)

if api_response.status_code == 200:
    progress = api_response.json()['response']['data']['progress']
    if progress is not None:
        print('Job Progress: ' + str(progress['percentComplete']) + '% complete.')
    else:
        print('Job Progress: 0% complete!.')
else:
    print(api_response.status_code)
    print(api_response.text)

Since nothing has been translated yet, this should report progress of '0%'. As content moves through the translation workflow (covered next), the progress number will increase. Note however that it’s also possible for the progress number to decrease. This can happen for a variety of reasons, such as when editors send content back to translators or if new content is added to an in-progress job.

In the following couple of steps, we’ll take a detour into the role of the localization manager to get a sense of what happens to our content after we’ve sent it for translation.

Step 6. Review translation workflows

A translation workflow is a sequence of steps through which content moves as part of the translation process. Below are examples of common translation workflows:

 

workflows.png

 

Content is sent into a workflow by an action called ‘authorization’. The final step of all workflows is called ‘Published’. Once all content in a file reaches the Published step for a particular locale, it’s considered completed for that locale. When all files are completed in all locales in a job, the job is considered completed. Completion of a locale or job triggers any associated callbacks that have been configured.

Smartling allows you to configure different workflows depending on your needs, and to choose which workflow a particular set of content should use.

For our test, we’ll use the simplest workflow: one with a single Translation step. This simple workflow is created by default when a new project is created in Smartling.

Confirm that your project is configured to use this simple workflow using these steps:

  1. Click on the gear icon in the upper right, under the Help menu, and select Project Settings:

    project-settings.png

  2. Select ‘Workflows’ from the left-navigation menu:

    workflow.png

You should see a workflow like the one above.

Now check that each language is configured to use that workflow by default by clicking on the Languages menu in the same Settings page:

languages.png


Confirm that your languages are set to use a one-step Translation-only workflow.

Step 7. Authorize job for translation

Authorization is the action that starts the translation process by moving the content into the first step of a translation workflow.

Navigate to the Jobs tab of the project, then authorize the job manually by clicking the Authorize button next to the job. You will be presented with an option to choose a workflow–just take the default. Once this action completes, the content will be in the Translation step of the workflow, and the job is said to be ‘in progress’.

 

authorize.png

 

The authorization step exists because a job might need to be assessed and its cost estimated before work begins on it. Although authorization is often done manually by, for example, a localization project manager, it’s also possible to authorize translation jobs via API. In our case, we chose not to authorize the job via API (see authorize api parameter in step 3 above).

You choose a job’s workflow when authorizing it–either in the dashboard or via the Authorize job API endpoint. Each language in a project is configured with a default workflow, which is used if no workflow is specified when authorizing the job. It’s possible for a job’s content to be distributed among several workflows, but in our case we sent all content to its default workflow, which was configured the same for each language.

When planning your production integration, you’ll need to determine if you’ll be authorizing jobs via API. If so, you will also need to know whether you need to specify particular workflows or can rely on the default settings that have been configured by your localization manager.

Step 8. Translate content in the Smartling dashboard

It's possible to configure a machine-translation workflow which will enable you to run integration tests without having to login to the Smartling dashboard; but for now, we’ll just enter translations manually. Note that the translation method we use here is not the one used by translators: translators use a special Smartling tool, known as a CAT (computer-assisted translation) Tool.

After authorizing the content, click into the job, and then click on the ‘Workflows’ tab of the job. You should see a number under the Translation workflow step. If a number doesn’t appear, it could be because the authorization process is still in progress: wait a moment, then refresh. The number shown is the number of source words in the translation workflow step for the given language.

 

job-workflows.png

 

Click on the number to take you to the project ‘Strings view’, filtered to show strings from this job and language.

Enter a ‘translation’ for each string, by hovering over the string and clicking the Edit Translation link that appears:

 

edit-translation.png

 

The translation can be an actual translation if you know the language, or some text that will indicate that it’s been ‘translated’. Make sure to click ‘Save’ for each one:

 

translate2.png

 

Submit completed translations

When you have entered all the translations, select all the strings in the strings view, then click Submit to Next Step from the Actions dropdown menu.

 

submit.png

 

This action moves our translations to the next step of the workflow, which in this case, is the Published step. Once all content reaches the Published step, the job is completed and ready for download.

Step 9. Re-check translation status

Before downloading the completed translations, let’s check the status of the job using the same method as before to confirm that it’s ready. You'll probably need to re-run the authentication step before doing this. Assuming all the translations in all selected languages have been submitted to the Published step, it should show 100%:

api_url = 'https://api.smartling.com/jobs-api/v3/projects/' + project_id + '/jobs/' + job_uid + '/progress'
api_request_headers = {'Authorization': 'Bearer ' + access_token}
api_response = requests.get(api_url, headers=api_request_headers)

if api_response.status_code == 200:
    progress = api_response.json()['response']['data']['progress']
    if progress is not None:
        print('Job Progress: ' + str(progress['percentComplete']) + '% complete.')
    else:
        print('Job Progress: 0% complete.')
else:
    print(api_response.status_code)
    print(api_response.text)

If it’s not yet at 100%, recheck where the content is in the workflow by clicking into the job, and then the workflows tab within the job. If any content shows in a step other than Published, click into it, then make sure it’s translated and submitted.

Step 10. Download translated files

Our last step is to download the translated files. We loop through our file list and locale list, making a separate call for each file/locale combination, writing the translated files into the test-files directory with the locale as part of the file name:

for file_name in FILE_LIST:
    for locale_id in LOCALE_LIST:
        file_uri = job_uid + '/' + file_name
        api_url = 'https://api.smartling.com/files-api/v2/projects/' + project_id + '/locales/' + locale_id + '/file'
        api_request_headers = {'Authorization': 'Bearer ' + access_token}
        api_parameters = {
            'fileUri': file_uri,
            'retrievalType' : 'published',
            'includeOriginalStrings' : True
        }
        api_response = requests.get(api_url,
                                    headers=api_request_headers,
                                    params=api_parameters)        
        if api_response.status_code == 200:            
            # insert locale_id before '.json' extension in translated file name
            translated_file_name = file_name[:-5] + '_' + locale_id + file_name[-5:]
            with open(translated_file_name, 'wb') as f:
                f.write(api_response.content)
        else:
            print(api_response.status_code)
            print(api_response.text)

We included the retrievalType parameter set to published above. This is what you would typically use in a production integration. It means that translations will be included in the download only if they’re in the Published step; for saved translations that are still being edited or reviewed the source string is returned instead of the in-progress translation.

We also set the includeOriginalStrings parameter to True, which is the default. This means that if a translation isn’t available then use the source string as the translation. Setting the parameter to False causes an empty string to be returned if no translation is available. This parameter is only supported for certain file types.

Now that we’ve downloaded our files, we’ve finished our test. However, a production integration would need to account for the fact that translations can sometimes be updated in Smartling after they’ve already been downloaded. This could be due to a translation issue being found post-launch, for example. To account for this possibility, a production integration needs to continue to watch for updates, which it can do by continuing to respond to callbacks, and/or by checking for recently published files (see ‘recently published files’ endpoint).

Summary

In this tutorial, we translated files using job batches. Along the way, we learned about some key Smartling concepts:

  • Jobs and job batches
  • Workflows
  • Authorization
  • Locale IDs

Next steps

Review the Key Concepts guide as well as the Overview of the API.

 

Was this article helpful?