Processing

Configure Engines and Clusters, manage and browse executions of your Tasks, Plans and Promotions.

Endpoints

Global security

These security schemes apply to the entire API

Security scheme

This scheme can be referenced across the API

Public
This Bearer authentication can be set in the Authorization header of your requests. Authentication tokens and Personal Access Tokens are supported. Personal Access Tokens can be generated in the Profile Preferences page of the Talend Cloud Portal.
Bearer authentication
Name Description
Format Bearer <token>

Task Executions

Executabletask

Object
Name Description Type Attributes and examples
executable Executable identifier string Required
parameters Executable parameters ObjectOptional
logLevel Execution log level string Optional
timeout Execution timeout in minutes. Minimal value is 1. integer Optional
INT32

JobExecutionStatus

Execution info
Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
startTimestamp Start time of job execution (UTC) datetime Required
RFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
RFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
RFC3339
userId User who triggered or scheduled the execution string Required
userType Type of user who triggered or scheduled the execution string Required
jobId Job ID string Required
jobVersion The version of the executed job string Required
environmentVersion Execution environment version string Required
executionStatus Execution status string Required
executionType Execution type string Required
executionDestination Destination of execution string Required
containerId Container ID string Optional
runProfileId Run profile ID string Optional
remoteEngineId Remote engine ID string Optional
remoteEngineClusterId Remote engine cluster ID string Optional
numberOfProcessedRows Number of processed rows integer Optional
INT64
numberOfRejectedRows Number of rejected rows integer Optional
INT64
accountId Account ID string Required
workspaceId Workspace ID string Required
planId Plan execution ID string Optional
errorType Error type if error occurs string Optional
errorMessage Error message if error occurs string Optional
timeout Execution timeout if set (in minutes) integer Optional
INT32

JobExecutionStatusV21

Execution info
Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
startTimestamp Start time of job execution (UTC) datetime Required
RFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
RFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
RFC3339
userId User who triggered or scheduled the execution string Required
userType Type of user who triggered or scheduled the execution string Required
jobId Job ID string Required
jobVersion The version of the executed job string Required
environmentVersion Execution environment version string Required
executionStatus Execution status string Required
executionType Execution type string Required
executionDestination Destination of execution string Required
containerId Container ID string Optional
runProfileId Run profile ID string Optional
remoteEngineId Remote engine ID string Optional
remoteEngineClusterId Remote engine cluster ID string Optional
numberOfProcessedRows Number of processed rows integer Optional
INT64
numberOfRejectedRows Number of rejected rows integer Optional
INT64
accountId Account ID string Required
workspaceId Workspace ID string Required
planId Plan execution ID string Optional
errorType Error type if error occurs string Optional
errorMessage Error message if error occurs string Optional
status Task execution status string Required

WorkspaceInfo

Object
Name Description Type Attributes and examples
id Workspace identifier string Optional
name Workspace name string Optional
description Workspace description string Optional
owner Workspace owner string Optional
type Workspace type string Optional
environment EnvironmentInfo Optional
protectedArtifactUpdate

Allows to restrict the scope of task updates when a new artifact version is published. 

  • true : the tasks of this workspace are protected from the unintended updates triggered by artifacts being published in other workspaces. These tasks are updated only when artifacts are being published in the same workspace
  • false : tasks that have been set to use the latest version of the published artifact, or any other artifact that has the same name as the newly published artifact, are all updated, whatever workspace is used for this artifact publishing
boolean Optional

TaskExecutionsFilters

Object
Name Description Type Attributes and examples
environmentId Environment filter string Optional
workspaceId Workspace filter string Optional
status Filter on status of execution string Optional
tags array of string Optional
Datatype details
Type Description Attributes and examples
array Task multiple tags filter
string One elementary tag to filter execution
lastDays Number of days in the past. Must be in range [1, 60]. Default value - 60 number Optional
from

Time period filter start date time (milliseconds).
With the from and to time period filters, the executions to be included in the response are:

  • Executions that started within the specified timeframe between from and to.
  • Executions that ended within the specified timeframe between from and to.
  • Executions that started and ended within the specified timeframe between from and to.
  • Executions that ran throughout the entire specified timeframe between from and to.
number Optional
to

Time period filter end date time (milliseconds).
With the from and to time period filters, the executions to be included in the response are:

  • Executions that started within the specified timeframe between from and to.
  • Executions that ended within the specified timeframe between from and to.
  • Executions that started and ended within the specified timeframe between from and to.
  • Executions that ran throughout the entire specified timeframe between from and to.
number Optional
limit Number of items to return. Must be in range [1, 100]. Default value - 100 number Optional
offset Position of the first item. Must be greater than or equal to 0. Default value - 0 number Optional
TaskExecutionsFilters
{
  "environmentId": "57f64991e4b0b689a64feed3",
  "workspaceId": "57f64991e4b0b689a64feed2",
  "tags": [
    "critical",
    "supply-asia"
  ],
  "limit": 100,
  "offset": 10,
  "status": "execution_rejected",
  "from": 1646159400000,
  "to": 1649183400000,
  "lastDays": 5
}

Get available task executions

POST /processing/executables/tasks/executions

Get available task executions. Available filters are :

  • some specific environment as the alternate filter to workspaces
  • some specific workspace as the alternate filter to environments
  • status of execution
  • tags associated with tasks : when multiple tags are provided, each of them will be used as a filter, that is to say, the logical AND is applied
  • limit to a number of days in recent history (for example, 1 : yesterday, 5 : past five days)
  • datetime period in the past, not exceeding a 60-day range

All those parameters are optional and can be combined at your convenience. Note that the workspace should belong to the environment when both filters (environmentId and workspaceId) are specified in the same call.

When neither the number of days nor a datetime period is provided, the default 60 days time range is applied.

Request

Body
TaskExecutionsFilters
application/json
{
  "limit": 100,
  "offset": 10,
  "status": "executing",
  "from": 1646159400000,
  "to": 1649183400000,
  "lastDays": 20,
  "environmentId": "57f64991e4b0b689a64feed3",
  "workspaceId": "57f64991e4b0b689a64feed2",
  "tags": [
    "critical",
    "finance-eu"
  ]
}

Response

200Status 200
Ok
Body
Page
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get Tasks Executions

GET /processing/executables/tasks/executions

Get task executions. Available filters are :

  • some specific environment as the alternate filter to workspaces
  • some specific workspace as the alternate filter to environments
  • status of execution
  • limit to a number of days in recent history (for example, 1 : yesterday, 5 : past five days)
  • datetime period in the past, not exceeding a 60-day range

All those parameters are optional and can be combined at your convenience. Note that the workspace should belong to the environment when both filters (environmentId and workspaceId) are specified in the same call.

When neither the number of days nor a datetime period is provided, the default 60 days time range is applied.

Request

Query parameters
Name Description Type Attributes and examples
environmentId Environment filter string Optional
workspaceId Workspace filter string Optional
status Filter on status of execution string Optional
lastDays Number of days in the past. Must be in range [1, 60]. Default value - 60 integer Optional
INT32
from

Time period filter start date time (milliseconds).
With the from and to time period filters, the executions to be included in the response are:

  • Executions that started within the specified timeframe between from and to.
  • Executions that ended within the specified timeframe between from and to.
  • Executions that started and ended within the specified timeframe between from and to.
  • Executions that ran throughout the entire specified timeframe between from and to.
integer Optional
INT64
to

Time period filter end date time (milliseconds).
With the from and to time period filters, the executions to be included in the response are:

  • Executions that started within the specified timeframe between from and to.
  • Executions that ended within the specified timeframe between from and to.
  • Executions that started and ended within the specified timeframe between from and to.
  • Executions that ran throughout the entire specified timeframe between from and to.
integer Optional
INT64
limit Number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
INT32
offset Position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
INT32

Response

200Status 200
OK
Body
Page
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get Task Executions

GET /processing/executables/tasks/{taskId}/executions
Get task executions

Request

Path variables
Name Description Type Attributes and examples
taskId task ID string Required
Query parameters
Name Description Type Attributes and examples
status execution status string Optional
lastDays Number of days in the past. Must be in range [1, 60]. Default value - 60 integer Optional
INT32
from

Time period filter start date time (milliseconds).
With the from and to time period filters, the executions to be included in the response are:

  • Executions that started within the specified timeframe between from and to.
  • Executions that ended within the specified timeframe between from and to.
  • Executions that started and ended within the specified timeframe between from and to.
  • Executions that ran throughout the entire specified timeframe between from and to.
integer Optional
INT64
to

Time period filter end date time (milliseconds).
With the from and to time period filters, the executions to be included in the response are:

  • Executions that started within the specified timeframe between from and to.
  • Executions that ended within the specified timeframe between from and to.
  • Executions that started and ended within the specified timeframe between from and to.
  • Executions that ran throughout the entire specified timeframe between from and to.
integer Optional
INT64
limit the number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
INT32
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
INT32

Response

200Status 200
OK
Body
Page
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Execute Task

POST /processing/executions
Execute Task

Request

Body
Executabletask

Response

201Status 201
Execution started
Body
Executionidentifier
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
ErrorResponse
404Status 404
Not found - resource not found
Body
ErrorResponse
405Status 405
Current Task is paused, it can’t be executed
500Status 500
Server error - something went wrong on server side
Body
ErrorResponse
501Status 501
Task cannot be deployed because it is not configured
Body
ErrorResponse

Get Task execution status

GET /processing/executions/{executionId}
Get Task execution status

Request

Path variables
Name Description Type Attributes and examples
executionId execution ID string Required

Response

200Status 200
Successful response
Body
JobExecutionStatusV21
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
ErrorResponse
404Status 404
Not found - resource not found
Body
ErrorResponse
429Status 429
Too many requests
Body
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
ErrorResponse

Terminate Task execution

DELETE /processing/executions/{executionId}
Terminate Task execution

Request

Path variables
Name Description Type Attributes and examples
executionId execution ID string Required

Response

200Status 200
Execution already finished
Body
string
204Status 204
Execution stopped succesfully
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
ErrorResponse
404Status 404
Not found - resource not found
Body
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
ErrorResponse

Plan Executions

PlanExecutable

Object
Name Description Type Attributes and examples
executable Executable identifier string Required
executionPlanId The plan execution identifier in case of re-execution string Optional
stepId The step identifier to start with string Optional
rerunOnlyFailedTasks Optionally run only failing tasks boolean Optional

PlanExecutionStatus

Plan Execution info
Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
startTimestamp Start time of job execution (UTC) datetime Required
RFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
RFC3339
userId User who triggered or scheduled the execution string Required
userType Type of user who triggered or scheduled the execution string Required
planId Plan ID string Required
executionStatus Execution (detailed) status string Required
plannedExecutableCount Number of planned executables integer Optional
INT32
doneExecutableCount Number of done executables integer Optional
INT32
doneExecutableDetails array of JobExecutionStatus Optional
Datatype details
Type Description Attributes and examples
array Execution activity info
JobExecutionStatus

StepExecution

Object
Name Description Type Attributes and examples
id string Optional
executionId string Optional
startTimestamp datetime Optional
RFC3339
finishTimestamp datetime Optional
RFC3339
executionStatus string Optional

Execute Plan

POST /processing/executions/plans

Allows to run a Plan.

This endpoint also allows for re-running some plan execution from some failed step : you should then provide execution id and step from which you wish to restart execution.

Request

Body
PlanExecutable

Response

201Status 201
Execution started
Body
Executionidentifier
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
ErrorResponse
404Status 404
Not found - resource not found
Body
ErrorResponse
405Status 405
Plan is paused, it can’t be executed
422Status 422
Entity exists but cannot be processed, because too old or plan exec didn’t fail
Body
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
ErrorResponse
501Status 501
Plan cannot be deployed because it is not configured
Body
ErrorResponse

Get available Plans executions

GET /processing/executables/plans/executions
List all plan executions the user is granted visibility on. For each plan execution returns execution and plan identifiers, start and finish timestamps, execution status and counters on done and planned executables within plan. This query is paginated and accepts filters as defined in parameters documented above.

Request

Query parameters
Name Description Type Attributes and examples
workspaceId workspace ID the plans should belong to string Optional
limit the number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
INT32
status execution status string Optional
lastDays lastDays integer Optional
INT32
from from date time (milliseconds) integer Optional
INT64
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
INT32
environmentId environment ID the plans should belong to string Optional
to to date time (milliseconds) integer Optional
INT64

Response

200Status 200
OK
Body
Page
Status 200 application/json
{
  "items": [
    {
      "executionId": "7fb48c7a-5ea5-4cb9-9333-e345f28d9dd5",
      "startTimestamp": "2022-10-20T14:20:28.582Z",
      "finishTimestamp": "2022-10-20T14:20:39.830Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "dc57c82f-7f18-4228-9f82-a81e02bae617",
      "startTimestamp": "2022-10-20T14:17:56.390Z",
      "finishTimestamp": "2022-10-20T14:18:07.865Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "4555c45f-9548-41e9-98b4-a9bfa2ee4e5c",
      "startTimestamp": "2022-10-20T13:47:37.611Z",
      "finishTimestamp": "2022-10-20T13:47:48.864Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "f3cfb077-2ea6-4557-bbb2-fd027f3cae4e",
      "startTimestamp": "2022-10-20T13:45:04.027Z",
      "finishTimestamp": "2022-10-20T13:45:15.929Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "b668cd50-6421-4e57-b15a-79b39ac153a8",
      "startTimestamp": "2022-10-20T13:18:01.235Z",
      "finishTimestamp": "2022-10-20T13:18:16.093Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_FAILED",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_failed"
    },
    {
      "executionId": "226dc12c-a288-4fe0-96f6-f6d51fa2b9ff",
      "startTimestamp": "2022-10-20T12:08:48.823Z",
      "finishTimestamp": "2022-10-20T12:18:52.729Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    }
  ],
  "limit": 100,
  "offset": 0,
  "total": 6
}
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get all steps for a plan execution (order by designed execution, only steps, without error handlers)

GET /processing/executions/plans/{planExecutionId}/steps
Get Steps executions status

Request

Path variables
Name Description Type Attributes and examples
planExecutionId Plan execution ID string Required

Response

200Status 200
Successful response
Body
details
Name Description Type Attributes and examples
array of StepExecution
Datatype details
Type Description Attributes and examples
array
StepExecution
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
ErrorResponse
404Status 404
Not found - resource not found
Body
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
ErrorResponse

Get Plan executions

GET /processing/executables/plans/{planId}/executions
Get Plan executions

Request

Path variables
Name Description Type Attributes and examples
planId plan ID string Required
Query parameters
Name Description Type Attributes and examples
limit the number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
INT32
status execution status string Optional
lastDays lastDays integer Optional
INT32
from from date time (milliseconds) integer Optional
INT64
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
INT32
to to date time (milliseconds) integer Optional
INT64

Response

200Status 200
OK
Body
Page
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get Plan Execution status

GET /processing/executions/plans/{id}
Get detailed status of one plan execution

Request

Path variables
Name Description Type Attributes and examples
id Execution id string Required

Response

200Status 200
Detailed status on one Platform execution
Body
PlanExecutionStatus
400Status 400
Status 400
Body
ErrorResponse
401Status 401
Status 401
Body
ErrorResponse
403Status 403
Permissions required
Body
ErrorResponse
404Status 404
Execution not found
Body
ErrorResponse
429Status 429
Status 429
Body
ErrorResponse
500Status 500
Something went wrong on server side
Body
ErrorResponse

Get the step for plan execution without error handlers

GET /processing/executions/plans/{planExecutionId}/steps/{stepExecutionId}
Get Step execution status

Request

Path variables
Name Description Type Attributes and examples
planExecutionId Plan execution ID string Required
stepExecutionId Step execution ID string Required

Response

200Status 200
Successful response
Body
PlanExecutionStatus
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
ErrorResponse
404Status 404
Not found - resource not found
Body
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
ErrorResponse

Get execution status of the handler for the specified step

GET /processing/executions/plans/{planExecutionId}/steps/{stepExecutionId}/error-handler
Get Step handler execution status

Request

Path variables
Name Description Type Attributes and examples
planExecutionId Plan execution ID string Required
stepExecutionId Step execution ID string Required

Response

200Status 200
Successful response
Body
PlanExecutionStatus
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
ErrorResponse
404Status 404
Not found - resource not found
Body
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
ErrorResponse

Promotion Executions

AdvancedPromotionSpec

Object
Name Description Type Attributes and examples
artifactId Advanced Promotion Artifact Id string Optional
artifactType Advanced Promotion Artifact Type. In the following list of allowed values, ‘ACTION’ refers to Artifacts, ‘FLOW’ refers to Tasks, ‘PLAN’ refers to Plans and ‘WORKSPACE’ refers to Workspaces string Optional

ArtifactPromotionResult

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
analyzeReport Report Optional
promotionReport Report Optional
usedBy array of PromotionResultInfo Optional
Datatype details
Type Description Attributes and examples
array Used by Artifacts
PromotionResultInfo
targetVersion Artifact Version In Target Environment string Optional
version Artifact Version In Source Environment string Optional
artifactId Flow artifact Id string Optional
jobType Action Job Type string Optional
versions array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array List of Action Versions
ArtifactPromotionResult

Execution

Object
Name Description Type Attributes and examples
executionId string Optional
executionReport PromotionExecutionStatusV26 Optional

FlowPromotionResult

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
analyzeReport Promotion Analysis Stage Report Report Optional
promotionReport Promotion Execution Stage Report Report Optional
usedBy Actions that use this flow FlowPromotionResult Optional
targetVersion Promoted Flow Version In Target Environment string Optional
version Source Flow Version integer Optional
INT32
versions array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Action Versions
ArtifactPromotionResult
jobType Flow Job Type string Optional
workspaceId Promoted Flow Workspace ID string Optional
artifactId Flow Artifact ID string Optional

PromotionExecutableTaskV26

Object
Name Description Type Attributes and examples
executable Executable identifier string Required
keepTargetResources Allows to keep target resources untouched. boolean Optional
keepTargetRunProfiles Allows to keep target run profiles untouched. boolean Optional
advanced Advance option to narrow the scope of the promotion execution : for instance limit to a workspace, etc. When not provided, promotion will apply on the whole environment scope. AdvancedPromotionSpec Optional
context Execution context message string Optional
255

PromotionExecutionStatus

Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
startTimestamp Start time of job execution (UTC) datetime Required
RFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
RFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
RFC3339
userId User who triggered or scheduled the execution string Required
promotionId Promotion ID string Optional
pipelineId Deprecated! Use Promotion ID instead string Optional
keepTargetResources Keep Target Resources flag boolean Optional
keepTargetRunProfiles Keep Target Run Profiles flag (not returned for API versions earlier V2.2) boolean Optional
advanced AdvancedPromotionSpec Optional
defective Defective Promotion flag boolean Optional
status Execution status string Optional
statusMessage Execution status message string Optional
workspaces array of WorkspacePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Workspace Promotion Execution Info
WorkspacePromotionResult
engines array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engines Promotion Execution Info
RuntimePromotionResult
clusters array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engine Clusters Promotion Execution Info
RuntimePromotionResult

PromotionExecutionStatusV26

Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
startTimestamp Start time of job execution (UTC) datetime Required
RFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
RFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
RFC3339
userId User who triggered or scheduled the execution string Required
promotionId Promotion ID string Optional
pipelineId Deprecated! Use Promotion ID instead string Optional
keepTargetResources Keep Target Resources flag boolean Optional
keepTargetRunProfiles Keep Target Run Profiles flag (not returned for API versions earlier V2.2) boolean Optional
advanced AdvancedPromotionSpec Optional
defective Defective Promotion flag boolean Optional
status Execution status string Optional
statusMessage Execution status message string Optional
workspaces array of WorkspacePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Workspace Promotion Execution Info
WorkspacePromotionResult
engines array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engines Promotion Execution Info
RuntimePromotionResult
clusters array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engine Clusters Promotion Execution Info
RuntimePromotionResult
context Execution context message string Optional
255
userType Promotion executor type, either HUMAN or SERVICE string Optional

PromotionResultInfo

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
analyzeReport Report Optional
promotionReport Report Optional
targetVersion Applicable to flows only, displays version of flow in target environment after promotion execution string Optional
version Applicable to flows only, displays version of flow in source environment integer Optional
INT32

Report

Object
Name Description Type Attributes and examples
status string Optional
messages array of Message Optional
Datatype details
Type Description Attributes and examples
array
Message

RuntimePromotionResult

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
analyzeReport Report Optional
promotionReport Report Optional
usedBy array of PromotionResultInfo Optional
Datatype details
Type Description Attributes and examples
array Used by Artifacts
PromotionResultInfo
runProfiles array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Run profiles
ArtifactPromotionResult

WorkspacePromotionResult

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
workspaceType Promoted workspace type, can be either Shared or Custom string Optional
analyzeReport Report Optional
promotionReport Report Optional
flows array of FlowPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Flows Promotion Execution Info
FlowPromotionResult
plans array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Plans Promotion Execution Info
ArtifactPromotionResult
actions array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Actions Promotion Execution Info
ArtifactPromotionResult
connections array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Connections Promotion Execution Info
ArtifactPromotionResult
resources array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Resources Promotion Execution Info
ArtifactPromotionResult
engines array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Engines Promotion Execution Info
RuntimePromotionResult
clusters array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Clusters Promotion Execution Info
RuntimePromotionResult

Execute a Promotion

POST /processing/executions/promotions

Execute a Promotion from source Environment to target one, according to all the default configuration set on Tasks.

Detailed rules on promotion executions are available here : https://help.talend.com/r/en-US/Cloud/management-console-user-guide/promotion-rules

In addition to these rules, current execution can be adapted as such :

  • Default scope of Promotion is the whole environment. When narrower scope (Workspace, Task, etc.) please provide advanced option parameter accordingly.
  • Run Profiles and Resources are defaulty updated on target. You can also choose to keep them untouched using parameters.

Lastly, promotion simulation and analysis is also available: helpful when check before exec is needed. Automatic-undo of a promotion execution is not provided.

Request

Body
PromotionExecutableTaskV26

Response

201Status 201
Execution success
Body
Execution
202Status 202
Execution started
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse
501Status 501
Promotions can’t be executed because it has errors
Body
ErrorResponse

Get Promotion execution status

GET /processing/executions/promotions/{executionId}
Get Promotion execution status

Request

Path variables
Name Description Type Attributes and examples
executionId Execution ID string Required

Response

200Status 200
OK
Body
PromotionExecutionStatusV26
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Remote Engines

Engine

Object
Name Description Type Attributes and examples
id Engine id string Required
name Engine name string Required
description Engine description string Optional
workspace WorkspaceInfo Optional
createDate Date of creation of the engine datetime Required
RFC3339
updateDate Date of updating of the engine datetime Optional
RFC3339
runtimeId Engine runtime id string Required
availability Availability status of engine|cluster string Optional
managed Indicates whether target runtime (engine/cluster) is managed or not boolean Optional
status Engine status string Required
runProfiles array of string Required
Datatype details
Type Description Attributes and examples
array Run profiles of engine
string
debug EngineDebug Optional
clusterId cluster that this engine is part of string Optional
preAuthorizedKey Preauthorized key (only for not paired engine) string Optional
esbCompatibilityVersion ESB compatibility version string Optional

EngineDebug

Object
Name Description Type Attributes and examples
host Debug host of engine string Required

EngineRequest

Object
Name Description Type Attributes and examples
name Engine name string Required
environmentId Id of engine environment string Required
workspaceId Id of engine workspace string Optional
runProfiles array of string Optional
Datatype details
Type Description Attributes and examples
array Run profiles of engine
string
debug EngineDebug Optional
description Engine|cluster description string Optional
esbCompatibilityVersion Data Service Runner/Talend Runtime version. Mandatory if TALEND_RUNTIME or MICROSERVICE profile selected string Optional

RunProfileRequest

Object
Name Description Type Attributes and examples
name Run profile name string Required
description Description string Optional
jvmArguments array of string Required
Datatype details
Type Description Attributes and examples
array Jvm arguments
string
type Run profile type string Required
runtimeId Runtime id string Required

RunProfileUpdateRequest

Object
Name Description Type Attributes and examples
name Run profile name string Required
description Description string Optional
jvmArguments array of string Required
Datatype details
Type Description Attributes and examples
array Jvm arguments
string

RuntimeRunProfile

Object
Name Description Type Attributes and examples
id string Optional
name string Optional
description string Optional
createDate datetime Optional
RFC3339
updateDate datetime Optional
RFC3339
type Run profile type string Optional
jvmArguments array of string Optional
Datatype details
Type Description Attributes and examples
array Jvm arguments
string
runtimeId string Optional
version Run profile version integer Optional
INT32

Create new Remote Engine

POST /processing/runtimes/remote-engines
Create new Remote Engine

Request

Body
EngineRequest

Response

201Status 201
Created
Body
Engine
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get all (available) Remote Engines

GET /processing/runtimes/remote-engines
Get all (available) Remote Engines

Request

Query parameters
Name Description Type Attributes and examples
query search query (FIQL format) string Optional

Response

200Status 200
OK
Body
details
Name Description Type Attributes and examples
array of Engine
Datatype details
Type Description Attributes and examples
array
Engine
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Delete Remote Engine by id

DELETE /processing/runtimes/remote-engines/{id}
Delete Remote Engine by id

Request

Path variables
Name Description Type Attributes and examples
id remote engine id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
409Status 409
Conflict
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get Remote Engine by id

GET /processing/runtimes/remote-engines/{id}
Get Remote Engine by id

Request

Path variables
Name Description Type Attributes and examples
id remote engine id string Required

Response

200Status 200
OK
Body
Engine
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Unpair Remote Engine

DELETE /processing/runtimes/remote-engines/{id}/pairing

Unpair Remote Engine.

If data service tasks and Route tasks are available in your license, you can maintain OSGi Data Services as deployed while upgrading your Remote Engine, without need to undeploy and redeploy those services. See Upgrading Remote Engine v2.11.8 onwards for data service Job or Route tasks for more details about this specific upgrade process.

Request

Path variables
Name Description Type Attributes and examples
id remote engine id string Required
Query parameters
Name Description Type Attributes and examples
keepOsgiDeployments Boolean to indicate if OSGi Routes or data services should be kept as deployed while unpairing - used for Remote Engine upgrade with data service tasks and Route tasks only boolean Optional
keepMicroserviceDeployments Boolean to indicate if microservice Routes or data services should be kept as deployed while unpairing - used for Remote Engine upgrade with data service tasks and Route tasks only boolean Optional

Response

200Status 200
OK
Body
string
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Create engine run profile

POST /processing/runtimes/remote-engines/{engineId}/run-profiles
Create engine run profile

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
Body
RunProfileRequest

Response

201Status 201
Created
Body
RuntimeRunProfile
400Status 400
Bad run profile definition
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
No engine can be found by provided
Body
ErrorResponse
409Status 409
Run profile with corresponding name already exists for given runtime
Body
ErrorResponse
500Status 500
Internal server error
Body
ErrorResponse

Get all engine run profiles

GET /processing/runtimes/remote-engines/{engineId}/run-profiles
Get all engine run profiles

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required

Response

200Status 200
OK
Body
details
Name Description Type Attributes and examples
array of RuntimeRunProfile
Datatype details
Type Description Attributes and examples
array
RuntimeRunProfile
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Engine not found
Body
ErrorResponse

Get engine run profile by id

GET /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}
Get engine run profile by id

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
runProfileId run profile id string Required

Response

200Status 200
OK
Body
RuntimeRunProfile
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Run profile not found
Body
ErrorResponse

Update engine run profile

PUT /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}
Update engine run profile

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
runProfileId run profile id string Required
Body
RunProfileUpdateRequest

Response

200Status 200
OK
Body
RuntimeRunProfile
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Run profile not found
Body
ErrorResponse
500Status 500
Internal server error
Body
ErrorResponse

Delete engine run profile

DELETE /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}

Delete engine run profile.

When in use in some task configuration, deletion is aborted unless you provided some new Run Profile id to use. It that latter case, tasks will be using this other Run Profile and the original Run Profile would be deleted.

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
runProfileId run profile id string Required
Query parameters
Name Description Type Attributes and examples
newRunProfileId new run profile id to use in tasks string Optional

Response

202Status 202
Run Profile deleted.
Body
string
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
No run profile can be found
Body
ErrorResponse
409Status 409
Run Profile is in use in some task configuration
Body
List of Task ID that are using the Run Profile.
string
500Status 500
Internal server error
Body
ErrorResponse

Remote Engine Clusters

ClusterRequest

Object
Name Description Type Attributes and examples
name Cluster name string Required
workspaceId Workspace identifier string Required
environmentId Environment identifier string Required
description Cluster description string Required
remoteEngines array of string Required
Datatype details
Type Description Attributes and examples
array List of remote engine ids
string

EngineCluster

Object
Name Description Type Attributes and examples
id Resource id string Required
name Resource name string Required
description Resource description string Optional
workspace WorkspaceInfo Optional
createDate Date of creation of the resource datetime Required
RFC3339
updateDate Date of updating of the resource datetime Optional
RFC3339
runtimeId Resource runtime id string Required
availability Availability status of engine|cluster string Optional
managed Indicates whether target runtime (engine/cluster) is managed or not boolean Optional
engines array of string Optional
Datatype details
Type Description Attributes and examples
array Engines in cluster
string

Create Remote Engine Cluster

POST /processing/runtimes/remote-engine-clusters
Create Remote Engine Cluster

Request

Body
ClusterRequest

Response

201Status 201
Created
Body
EngineCluster
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get all (available) Remote Engine Clusters

GET /processing/runtimes/remote-engine-clusters
Get all (available) Remote Engine Clusters

Request

Query parameters
Name Description Type Attributes and examples
_s search query (FIQL format), e.g. “workspace.environment.id==5cb47ca4b1b5247f6006529e”,“name==NewCluster” string Optional

Response

200Status 200
OK
Body
details
Name Description Type Attributes and examples
array of BaseEngine
Datatype details
Type Description Attributes and examples
array
BaseEngine
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get Remote Engine Cluster by id

GET /processing/runtimes/remote-engine-clusters/{clusterId}
Get Remote Engine Cluster by id

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required

Response

200Status 200
OK
Body
EngineCluster
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Delete Remote Engine Cluster by id

DELETE /processing/runtimes/remote-engine-clusters/{clusterId}
Delete Remote Engine Cluster by id

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Add Remote Engine to Remote Engine Cluster

PUT /processing/runtimes/remote-engine-clusters/{clusterId}/engines/{engineId}
Add Remote Engine to Remote Engine Cluster

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required
engineId remote engine id string Required
Body
Optional reschedule option for the run profile
RunProfileRescheduleOptions

Response

204Status 204
No Content
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
409Status 409
Conflict
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Remove Remote Engine from Remote Engine Cluster

DELETE /processing/runtimes/remote-engine-clusters/{clusterId}/engines/{engineId}
Remove Remote Engine from Remote Engine Cluster

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required
engineId remote engine id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get cluster run profiles

GET /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles
Get cluster run profiles

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required

Response

200Status 200
OK
Body
details
Name Description Type Attributes and examples
array of RuntimeRunProfile
Datatype details
Type Description Attributes and examples
array
RuntimeRunProfile
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Cluster not found
Body
ErrorResponse

Create cluster run profile

POST /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles
Create cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
Body
RunProfileRequest

Response

201Status 201
Body
RuntimeRunProfile
400Status 400
Bad run profile definition
Body
RuntimeRunProfile
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
No cluster can be found by provided
Body
ErrorResponse
409Status 409
Run profile name already used
Body
ErrorResponse
500Status 500
Internal server error
Body
ErrorResponse

Update cluster run profile

PUT /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}
Update cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
runProfileId run profile id string Required
Body
RunProfileUpdateRequest

Response

200Status 200
OK
Body
RuntimeRunProfile
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
No run profile can be found
Body
ErrorResponse
500Status 500
Internal server error
Body
ErrorResponse

Delete cluster run profile

DELETE /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}
Delete cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
runProfileId run profile id string Required
Query parameters
Name Description Type Attributes and examples
runProfileId new run profile id string Optional

Response

204Status 204
No Content
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
No run profile can be found
Body
ErrorResponse
500Status 500
Internal server error
Body
ErrorResponse

Get cluster run profile

GET /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}
Get cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
runProfileId run profile id string Required

Response

200Status 200
OK
Body
RuntimeRunProfile
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Run profle not found
Body
ErrorResponse

RunProfileRescheduleOptions

Object
Name Description Type Attributes and examples
method Run profile reschedule method string Required
DEFAULT
runProfileId Run profile to be used from cluster string Required

Remote Engine Gen2

AdvancedRunProfile

Pipeline run profile of type ‘Advanced’
Object
Name Description Type Attributes and examples
name Name string Required
description Description string Optional
engineId Pipeline engine id string Optional
definition Json with definition string Optional
pushdownPreferenceEnabled Prefer executions in SQL Pushdown mode when possible (if pipelines are compatible, they are sent to the data warehouse for execution). boolean Optional
warehouseName Optional warehouse name you can provide to overwrite the default warehouse parameter used to connect to the data warehouse at runtime. string Optional
createDate Creation date of the run profile datetime Optional
RFC3339
updateDate Update date of the run profile datetime Optional
RFC3339
id run profile id string Optional
type string Optional

BigDataRunProfile

Pipeline run profile of type ‘Big Data’
Object
Name Description Type Attributes and examples
name Name string Required
description Description string Optional
engineId Pipeline engine id string Optional
distribution Distribution string Optional
executionFramework Execution framework string Optional
microBatchInterval Micro-batch interval (in ms) integer Required
INT32
streamingTimeout Streaming timeout (in ms) integer Required
INT32
yarnQueue Yarn queue string Optional
driverCores Driver cores integer Optional
INT32
driverMemory Driver cores (in MB) integer Optional
INT32
temporaryStorage Temporary storage string Required
yarnStrategy YarnStrategy Required
executorMemoryOverhead Execution memory overhead (MB) integer Required
INT32
isCheckpointingEnabled Checkpointing boolean Optional
checkPointFilePath Checkpoint file path. Required if checkpointing enabled string Optional
parameters Parameters ObjectOptional
id Run profile id string Optional
type Type string Optional
createDate Created on datetime Optional
RFC3339
updateDate Updated on datetime Optional
RFC3339

Databrick

Object
Name Description Type Attributes and examples
endpoint Endpoint string Required
token Token (can also be provided in the Remote Engine configuration) string Optional
stagingLocation DBFS library staging directory string Required
target DatabricksCluster Required
cloudProvider Cloud provider string Required

DatabricksCluster

Object
Name Description Type Attributes and examples
cluster New or existing cluster string Required
id ID of existing cluster. Required for ‘old’ cluster string Optional
sparkVersion Databricks runtime version. Required for ‘new’ cluster string Optional
nodeTypeId Node type ID. Required for ‘new’ cluster string Required
logsDestination Logs destination. Required for ‘new’ cluster string Required
workersNumber Number of workers. Required for ‘new’ cluster integer Required
INT32

DatabricksRunProfile

Pipeline run profile of type ‘Databricks’
Object
Name Description Type Attributes and examples
name Name string Required
description Description string Optional
engineId Pipeline engine id string Optional
microBatchInterval Micro-batch interval (in ms) integer Required
INT32
databricks Databrick Optional
parameters Parameters ObjectOptional
id Run profile id string Optional
type Type string Optional
createDate Created on datetime Optional
RFC3339
updateDate Updated on datetime Optional
RFC3339

DataIntegrationRunProfile

Object
Name Description Type Attributes and examples
name string Required
description string Optional
jvmArguments array of string Optional
Datatype details
Type Description Attributes and examples
array jvm arguments applied on job execution
string

DataIntegrationRunProfileResponse

Object
Name Description Type Attributes and examples
id string Optional
name string Required
description string Optional
engineId string Optional
runtimeId string Optional
jvmArguments string Optional
type string Optional
createDate date Optional
updateDate string Optional

PipelineEngine

Object
Name Description Type Attributes and examples
id Resource id string Required
name Resource name string Required
description Resource description string Optional
workspace WorkspaceInfo Optional
createDate Date of creation of the resource datetime Required
RFC3339
updateDate Date of updating of the resource datetime Optional
RFC3339
runtimeId Resource runtime id string Required
availability Availability status of engine|cluster string Optional
managed Indicates whether target runtime (engine/cluster) is managed or not boolean Optional
status string Optional
preAuthorizedKey string Optional
cloudRunner boolean Optional
runStatus Engine status for Cloud Runner string Optional

PipelineEngineRequest

Object
Name Description Type Attributes and examples
name Engine name string Required
environmentId Id of engine environment string Required
workspaceId Id of engine workspace string Optional
description Engine|cluster description string Optional

PipelineRunProfileResponse

Pipeline run profile response object.
Object
Name Description Type Attributes and examples
name Name string Required
description Description string Optional
engineId Pipeline engine id string Optional
id Run profile id string Optional
type Type string Optional
createDate Created on datetime Optional
RFC3339
updateDate Updated on datetime Optional
RFC3339

StandardRunProfile

Pipeline run profile of type ‘Standard’
Object
Name Description Type Attributes and examples
name Name string Required
description Description string Optional
engineId Pipeline engine id string Optional
configuration Configuration Required
executionFramework Execution framework string Required
pushdownPreferenceEnabled Prefer executions in SQL Pushdown mode when possible (if pipelines are compatible, they are sent to the data warehouse for execution). boolean Optional
warehouseName Optional warehouse name you can provide to overwrite the default warehouse parameter used to connect to the data warehouse at runtime. string Optional

StandardRunProfileResponse

Pipeline run profile of type ‘Standard’
Object
Name Description Type Attributes and examples
name Name string Required
description Description string Optional
engineId Pipeline engine id string Optional
configuration Configuration Required
executionFramework Execution framework string Required
id Run profile id string Optional
type Type string Optional
createDate Created on datetime Optional
RFC3339
updateDate Updated on datetime Optional
RFC3339
pushdownPreferenceEnabled Prefer executions in SQL Pushdown mode when possible (if pipelines are compatible, they are sent to the data warehouse for execution). boolean Optional
warehouseName Optional warehouse name you can provide to overwrite the default warehouse parameter used to connect to the data warehouse at runtime. string Optional

YarnStrategy

Object
Name Description Type Attributes and examples
type Yarn strategy type string Required
initialExecutors Initial number of executors (for Dynamic strategy) integer Optional
INT32
minExecutors Min. number of executors (for Dynamic strategy) integer Optional
INT32
maxExecutors Max. number of executors (for Dynamic strategy) integer Optional
INT32
coresPerExecutor Cores per executor integer Optional
INT32
executorMemory Executor memory (in MB) integer Optional
INT32
executorsNumber Number of executors (for Fixed strategy) integer Optional
INT32

Create new Pipeline Engine

POST /processing/runtimes/pipeline-engines
Create new Pipeline Engine

Request

Body
PipelineEngineRequest

Response

201Status 201
Created
Body
PipelineEngine
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get Pipeline Engines

GET /processing/runtimes/pipeline-engines
Get Pipeline Engines

Request

Query parameters
Name Description Type Attributes and examples
query search query (FIQL format), e.g. “workspace.environment.id==5cb47ca4b1b5247f6006529e”,“status==PAIRED” string Optional

Response

200Status 200
OK
Body
details
Name Description Type Attributes and examples
array of PipelineEngine
Datatype details
Type Description Attributes and examples
array
PipelineEngine
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Delete Pipeline Engine by id

DELETE /processing/runtimes/pipeline-engines/{engineId}
Delete Pipeline Engine by id

Request

Path variables
Name Description Type Attributes and examples
engineId pipeline engine id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
409Status 409
Conflict
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get Pipeline Engine by id

GET /processing/runtimes/pipeline-engines/{engineId}
Get Pipeline Engine by id

Request

Path variables
Name Description Type Attributes and examples
engineId pipeline engine id string Required

Response

200Status 200
OK
Body
PipelineEngine
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Unpair Pipeline Engine

DELETE /processing/runtimes/pipeline-engines/{engineId}/pairing
Unpair Pipeline Engine

Request

Path variables
Name Description Type Attributes and examples
engineId pipeline engine id string Required

Response

200Status 200
OK
Body
string
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get run profile for engine

GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles

Request

Path variables
Name Description Type Attributes and examples
engineId run profile id string Required

Response

200Status 200
successful operation
Body
details
Name Description Type Attributes and examples
array of PipelineRunProfileResponse
Datatype details
Type Description Attributes and examples
array
PipelineRunProfileResponse
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal server error
Body
ErrorResponse

Create new advanced run profile

POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
Body
AdvancedRunProfile

Response

201Status 201
Created
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Update advanced run profile

PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced/{runProfileId}

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
runProfileId run profile id string Required
Body
AdvancedRunProfile

Response

204Status 204
No Content
400Status 400
Bad Request
Body
ErrorResponse
401Status 401
Unauthorized
Body
ErrorResponse
403Status 403
Forbidden
Body
ErrorResponse
404Status 404
Not Found
Body
ErrorResponse
500Status 500
Internal Server Error
Body
ErrorResponse

Get advanced run profile

GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced/{runProfileId}

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
runProfileId run profile id string Required

Response

200Status 200
OK
Body