Release notes

R2024-03

Deployment strategy on a Remote Engine cluster

Feature Description Related API documentation
Route and data service deployment

When configuring how a Route or data service task runs on a Talend Remote Engine cluster using API, you can use the optional parameter deploymentStrategy to determine how this task should be deployed.

  • Rolling: Tasks are deployed one by one to the engines of the cluster. This means that for a short time, different engines will be running different versions of the task, therefore you can avoid service interruption.
  • Parallel: Tasks are simultaneously deployed to all the engines of the cluster. Before deploying a new version of a task, you will need to undeploy the currently running version, which will result in service interruption.

Related endpoints:

R2023-11

Plan step update

Feature Description Related API documentation
New plan step condition

The Always Proceed to Next Step condition has been added to the existing conditions on a given step to allow your plan to continue the execution flow regardless of the success or failure of individual tasks within this step.

This new condition enhances flexibility in plan design and increases resilience in plan executions, ensuring subsequent steps are executed irrespective of the outcomes of each individual task within a given step.

Related endpoints:

R2023-10

Feature Description Related API documentation
Search for historical executions

New Monitoring APIs have been added to help you easily search for historical executions by filtering on a wide range of execution fields. These fields include the types of the execution like a plan or a task, the category of an artifact such as a standard Job, a Route, or a Pipeline, and the runtime configuration like the engine, the workspace, or the environment.

Additionally, These APIs allow you to conveniently download the results in a CSV file for further processing.

Related endpoint:

R2023-09

Observability metrics API enhancements

Feature Description Related API documentation
Search for component metrics

The Search raw component metrics endpoint has been improved to be more accurate when searching for component metrics data by the start or the end time of the components.

The updated filters are:

  • startTime: start time of metrics generation and collection. This time corresponds approximately to when your Job task starts to work.
  • endTime: end time of metrics generation and collection. This time corresponds approximately to when this component in your Job task finishes its work
  • component_start_time_seconds: this filter has been deprecated

Related endpoint:

R2023-06

Remote Engine unpairing

Feature Description Related API documentation
New unpairing options

When unpairing a Remote Engine using the Unpair Remote Engine API endpoint, you can use new options to maintain uninterrupted your running data service tasks and Route tasks:

  • keepOsgiDeployments: set this option to true to keep OSGi services running
  • keepMicroserviceDeployments: set this option to true to keep microservices running

These options offer greater flexibility and control for your unpairing process.

Related endpoint:

R2023-05

API usability enhancement

Feature Description Related API documentation
API discrepancies

Consistent endeavor is continuously being carried out to address API discrepancy issues to help you efficiently develop your API solutions.

Highlighted fixes:

Run profile

Feature Description Related API documentation
Run profile deletion

When deleting a run profile, use the newRunProfileId query parameter to specify the run profile that replaces the deleted one.

In addition, a list of tasks using the run profile to be deleted is returned:

  • If you do not provide a new run profile ID for the replacement, the deletion fails because the tasks on this list still use this run profile and you have not provided an alternative.
  • If you provide a new run profile ID, the deletion succeeds. Then you need to update the tasks on that list with the new run profile ID.

Related endpoints:

R2023-04

Time range filter

Feature Description Related API documentation
Execution search: updated policy on time range filters

When filtering task executions using the from and the to filters, the executions to be included in the response are:

  • Executions that started in the specified timeframe between from and to.
  • Executions that ended in the specified timeframe between from and to.
  • Executions that started and ended in the specified timeframe between from and to.
  • Executions that ran throughout the entire specified timeframe between from and to.

Updated endpoint:

R2023-03

Talend Cloud Management Console account limits

Feature Description Related API documentation
New workload threshold for TMC API

Maximum Transaction Per Minute(TPM) rates have been increased to more effectively limit the concurrent API calls from a single IP address:

  • Monitoring: 600
  • Orchestration: 1800
  • Processing: 1800
  • Seats and subscription: 60
  • Workspace permissions: 600

These limits serve to safeguard against potential misuse or attacks, ensure equitable access to resources, better handle sudden spikes, and enable smooth functioning under diverse scenarios, ranging from standard to exceptional.

Related endpoints:

License information

Feature Description Related API documentation
Seat and subscription endpoint

A new endpoint has been added to help you easily retrieve details of your license, such as:

  • Number of tokens you acquired
  • How they are allocated for various usage scenarios
  • Your current subscription options

Related endpoint:

R2023-02

Task update

Feature Description Related API documentation
Protected artifact update

When publishing the new version of an artifact, by default, all the tasks that meet the following conditions are automatically updated, no matter in which workspace this publishing happens:

  • they are configured with Always use the latest artifact version
  • they are using an artifact that has the same name as the artifact being published

You can now protect a workspace and therefore its tasks from this kind of unintended update, by setting the protectedArtifactUpdate parameter to be true, when invoking the workspace update endpoint. With this parameter on, tasks in this protected workspace are updated only when the artifact is published in the same workspace.

Updated endpoint:

R2023-01

Task execution filter

Feature Description Related API documentation
Mandatory time range limit

A narrower yet more reasonable time range limit has been applied on the task execution filtering to optimize the efficiency of your execution history queries and prevent against misuse of the resources.

  • When setting a time range with the from and the to parameters, the allowed maximum value is now 60 days.
  • If no time range is explicitly set, the default value is the last 60 days.

OAS CLI generation

Feature Description Related API documentation
API specification improvements

The downloads of public API specifications are now bug free and ready for a smooth OAS (OpenAPI Specification) CLI generation.

Related endpoints:

Pausing or resuming a task

Feature Description Related API documentation
More operation details

You can now easily read who paused or resumed a task and on which date this operation was performed.

Simulating promotion runs

Feature Description Related API documentation
Simulating promotions

The endpoint to simulate runs of a given promotion has been relocated from the Processing domain to the Orchestration domain.

The old endpoint now becomes unavailable.

During this migration, some missing artifact version fields are also added in the response of the API call.

New endpoint:

API usability enhancement

Feature Description Related API documentation
API discrepancies

A number of discrepancy issues have been fixed for API endpoints to help you efficiently develop your API solutions.

Related endpoints:

New support for service accounts

Feature Description Related API documentation
More APIs supporting service accounts

All the project management endpoints, the Observability endpoints, and the smart timeout endpoints now support service account tokens. This means that you can use service accounts to invoke these APIs without having to rely on a user account.

R2022-12

Smart timeout

Feature Description Related API documentation
Smart timeout of a task

You can now use API requests to turn on or turn off smart timeout for a specific task.

For further information about how a smart timeout is computed, see Enabling smart Job task execution timeout.

These new endpoints support only the personal access tokens. They do not support the service account tokens.

New endpoints:

R2022-11

SCIM v2

Feature Description Related API documentation
SCIM provisioning

You can now use the SCIM v2 API endpoints to automate the processes to create users, assign users to groups, assign roles to users and retrieve user profiles.

SCIM v2

SSO Role mapping

Feature Description Related API documentation
SSO role mapping

These new endpoints are designed to help you easily synchronize role assignments and manage their changes between your SSO provider and Talend Cloud.

SSO Role Mapping

R2022-10

Pausing or resuming a plan

Feature Description Related API documentation
Pause and resume plans

A new endpoint has been added to help you easily pause or resume plan executions, when required by a certain circumstance, for example, when a system maintenance window imposes a pause of plan executions.

Accordingly, related endpoints have been updated to take the plan pause status into account.

New endpoint:

Updated endpoints:

Talend Cloud Management Console API workload thresholds

Feature Description Related API documentation
API transaction limits

Maximum TPS (transaction Per Second) of API calls from an IP address has been set to 30 to ensure the resilience of the product for every API user.

This limit is meant to better handle peaks, protect from misuse or attacks, ensure equal capacities to all requests, and better adapt to situations from standard to specific.

For more limits about a Talend Cloud Management Console related acount, see Talend Cloud Management Console account limits.

R2022-09

Crawler

Feature Description Related API documentation
New API A new API allows you to retrieve a full list of assets contained in a connection, in a single operation. For examples of this API in use, see Crawler. Crawler

Simulating promotion runs with artifact version details

Feature Description Related API documentation
Simulating promotions

A new endpoint has been created to simulate runs of a given promotion to build a detailed report accordingly.

This report helps you easily manage your software life cycle by better understanding the differences of artifacts and tasks between the source and the target environments of that promotion.

For the same purpose, the promotion execution endpoint is updated to include the artifact version information in its response.

New endpoint:

Updated endpoint:

Workspace and environment management endpoints

Feature Description Related API documentation
Workspace and environment management

In addition to the creation endpoints, more endpoints have been created to help you automate the management of workspaces and environments.

With these new endpoints, you can now set up automatic processes to perform the following operations:

  • Environments:
    • get available environments
    • update a specific environment
    • delete an environment
  • Workspaces:
    • get available workspaces
    • update a specific workspace
    • delete a workspace

Environment endpoints:

Workspace endpoints:

R2022-07

Pausing or resuming a task

Feature Description Related API documentation
Pause and resume tasks

A new endpoint has been added to help you easily pause or resume task executions, when required by a certain circumstance, for example, when a system maintenance window imposes a pause of task executions.

Accordingly, related endpoints have been updated to take the task pause status into account.

New endpoint:

Updated endpoints:

Timeout on triggers

Feature Description Related API documentation
Timeout settings

In addition to the configuration level and the execution level, you can now set a timeout parameter in the trigger level of a Job task.

A timeout set on a trigger applies on the task run started by this trigger only. It overrides the task configuration timeout if that task has one.

Set this timeout when you append a trigger to a schedule or update a trigger of that schedule.

When you set a timeout value, be aware:
  • Only time-based triggers support it; a webhook trigger does not.
  • Its accuracy is 10 seconds.
  • It is defined in minutes.
  • It applies only on Job tasks running on Remote Engine Gen1 v2.12.0 and onwards, Remote Engine Cluster and Cloud Engine.

Observability metrics

Feature Description Related API documentation
New endpoint You can now search for raw component metrics data to build reports about your Data Integration tasks, running on a Cloud Engine or a Remote Engine v2.12.0 onwards. Search raw component metrics

R2022-06

API domains

Feature Description Related API documentation
New business domains The Talend Cloud Management Console specific API endpoints have been carefully tailored to new business domains:
  • Orchestration
  • Processing
  • Security
  • Monitoring

These domains also contain a variety of subsets to bring the business oriented endpoints to an even finer granularity.

All these endpoints support both Personal Access Tokens (PATs) and Service Account Tokens (SATs)

See the following page to discover all these endpoints in their subsets and domains:

Orchestration

Feature Description Related API documentation
Multiple time triggers New schedule-specific endpoints have been created to help you manage up to 15 time-based triggers for a task or plan:
  • directly define multiple triggers for a task or plan. You do not need to duplicate your tasks or plans for each of those triggers anymore.

  • simulate all the events to be triggered before applying a schedule to a task or plan
  • manage changes in a schedule without taking any risk upon your current executions
  • reuse and adapt existing schedules for other tasks or plans
  • avoid redundant trigger definitions and tight event frequency, including the bissextile years "effect"
  • protect against accidental deletion of a schedule when it is still used by a task or plan

Note that a schedule supports time triggers only. For this reason, a webhook trigger, which is merely event related, must be used as a separate option, that is to say, without being mixed with the time triggers.

For detailed use cases about how to use these endpoints with a task or a plan, see Using API to schedule task or plan runs

Task filtering endpoint You can now select tasks by setting filters on their tags and trigger types in the request body of a new endpoint.
Environment endpoint New endpoint has been added for environment creation.
Workspace endpoint You can now create a workspace via API.

Job task timeout

Feature Description Related API documentation
Timeout settings A timeout parameter is now available to help you automatically terminate task runs when their durations exceed intended timeframes:

You can set this parameter at two different levels to dynamically manage the timeout of your tasks:

  • Task configuration level: set in the PUT request to the Configure Task executions endpoint. This timeout is added to the configuration of a task and thus applies on all the runs of this task.
  • Task execution level: set in the POST request to the Execute Task endpoint. This timeout dynamically applies on the current task run only. It overrides the task configuration timeout if that task has one.
When you set a timeout value, be aware:
  • Its accuracy is 10 seconds.
  • It is defined in minutes.
  • It applies only on Job tasks running on Remote Engine Gen1 v2.12.2 and onwards, Remote Engine Cluster and Cloud Engine.

R2022-05

Processing

Feature Description Related API documentation
Task executions You can now obtain task runs by setting filters on their tags. The TMC_OPERATOR role is required to use this endpoint. Get available task executions
Plan executions Specific criteria for filtering plan runs have been added: environmentId and workspaceId. The TMC_OPERATOR role is required to use this endpoint. Get available Plans executions

Orchestration

Feature Description Related API documentation
Event retrieval A new Orchestration API endpoint has been added to retrieve a list of all types of runs scheduled for a given period. The TMC_OPERATOR role or the TMC_ENGINE_USE permission is required to use this endpoint. Get scheduled executions

Observability metrics

Feature Description Related API documentation
New endpoint A new endpoint has been added to get the observability metrics of your Data Integration tasks. Get component metrics of task runs

R2022-04

Execution logs

Feature Description Related API documentation
New parameters New optional query parameters to select the format or define the log file name:
  • fileFormat: the format of the log file to be generated, JSON or TEXT.
  • newFileName: this parameter indicates whether you need to include task IDs and execution IDs in the filename of the log to be generated. It is recommended to set this value to be true as the old filename format is going to be deprecated.
Generate full logs for a completed or ongoing execution
New date format A new isoDate field has been added to the content of the logs. This field displays human readable dates.

R2022-03

Service Account

Feature Description Related API documentation
New API A new API allows you to create and manage service accounts. For examples of this API in use, see Service accounts. Service Accounts

Orchestration

Feature Description Related API documentation
New endpoints Two new endpoints have been added to the task and the plan APIs to retrieve a list of the upcoming scheduled runs.

Processing

Feature Description Related API documentation
New field in promotion data types Data types used in promotion-related operations have been updated to include a new context field. This optional field contains the description of the circumstances related to a promotion run.