Processing

Configure Engines and Clusters, manage and browse executions of your Tasks, Plans and Promotions.

Endpoints

Global security

These security schemes apply to the entire API

Security scheme

This scheme can be referenced across the API

Public
This Bearer authentication can be set in the Authorization header of your requests. Authentication tokens and Personal Access Tokens are supported. Personal Access Tokens can be generated in the Profile Preferences page of the Talend Cloud Portal.
Bearer authentication
Name Description
Format Bearer <token>

Task Executions

Executabletask

Object
Name Description Type Attributes and examples
executable Executable identifier string Required
Example
57f64991e4b0b689a64feed0
parameters Executable parameters ObjectOptional
logLevel Execution log level string Optional
Enum
  • OFF
  • ERROR
  • WARN
  • INFO
Example
WARN
timeout Execution timeout in minutes. Minimal value is 1. integer Optional
FormatINT32

JobExecutionStatus

Execution info
Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
Example
7b2b122e-d6b8-42de-b0ba-fa2f0d36306e
startTimestamp Start time of job execution (UTC) datetime Required
FormatRFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
FormatRFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
FormatRFC3339
userId User who triggered or scheduled the execution string Required
Example
fupton
userType Type of user who triggered or scheduled the execution string Required
Enum
  • HUMAN
  • SERVICE
jobId Job ID string Required
Example
57f64991e4b0b689a64feed0
jobVersion The version of the executed job string Required
Example
5.2
environmentVersion Execution environment version string Required
Enum
  • 1.2
  • 1.3
Example
1.3
executionStatus Execution status string Required
Enum
  • EXECUTION_EVENT_RECEIVED
  • DISPATCHING_FLOW
  • STARTING_FLOW_EXECUTION
  • STOPPING_FLOW_EXECUTION
  • EXECUTION_TERMINATED
  • DEPLOY_FAILED
  • EXECUTION_FAILED
  • EXECUTION_REJECTED
  • EXECUTION_SUCCESS
executionType Execution type string Required
Enum
  • MANUAL
  • SCHEDULED
  • WEBHOOK
  • PLAN
Example
SCHEDULED
executionDestination Destination of execution string Required
Enum
  • CLOUD
  • CLOUD_EXCLUSIVE
  • REMOTE_ENGINE
  • REMOTE_ENGINE_CLUSTER
Example
REMOTE_ENGINE
containerId Container ID string Optional
runProfileId Run profile ID string Optional
Example
157f818f-a901-4425-b592-0f9282687784
remoteEngineId Remote engine ID string Optional
Example
157f818f-a901-4425-b592-0f9282687784
remoteEngineClusterId Remote engine cluster ID string Optional
numberOfProcessedRows Number of processed rows integer Optional
FormatINT64
Example
1234567890
numberOfRejectedRows Number of rejected rows integer Optional
FormatINT64
Example
0
accountId Account ID string Required
Example
8494b016-b5ef-4b9c-b16d-8b1f824d7616
workspaceId Workspace ID string Required
Example
57ce63d3e4b0681c36d1a1c4
planId Plan execution ID string Optional
errorType Error type if error occurs string Optional
errorMessage Error message if error occurs string Optional
timeout Execution timeout if set (in minutes) integer Optional
FormatINT32

JobExecutionStatusV21

Execution info
Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
Example
7b2b122e-d6b8-42de-b0ba-fa2f0d36306e
startTimestamp Start time of job execution (UTC) datetime Required
FormatRFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
FormatRFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
FormatRFC3339
userId User who triggered or scheduled the execution string Required
Example
fupton
userType Type of user who triggered or scheduled the execution string Required
Enum
  • HUMAN
  • SERVICE
jobId Job ID string Required
Example
57f64991e4b0b689a64feed0
jobVersion The version of the executed job string Required
Example
5.2
environmentVersion Execution environment version string Required
Enum
  • 1.2
  • 1.3
Example
1.3
executionStatus Execution status string Required
Enum
  • EXECUTION_EVENT_RECEIVED
  • DISPATCHING_FLOW
  • STARTING_FLOW_EXECUTION
  • STOPPING_FLOW_EXECUTION
  • EXECUTION_TERMINATED
  • DEPLOY_FAILED
  • EXECUTION_FAILED
  • EXECUTION_REJECTED
  • EXECUTION_SUCCESS
  • EXECUTION_TERMINATED_TIMEOUT
executionType Execution type string Required
Enum
  • MANUAL
  • SCHEDULED
  • WEBHOOK
  • PLAN
Example
SCHEDULED
executionDestination Destination of execution string Required
Enum
  • CLOUD
  • CLOUD_EXCLUSIVE
  • REMOTE_ENGINE
  • REMOTE_ENGINE_CLUSTER
Example
REMOTE_ENGINE
containerId Container ID string Optional
runProfileId Run profile ID string Optional
Example
157f818f-a901-4425-b592-0f9282687784
remoteEngineId Remote engine ID string Optional
Example
157f818f-a901-4425-b592-0f9282687784
remoteEngineClusterId Remote engine cluster ID string Optional
numberOfProcessedRows Number of processed rows integer Optional
FormatINT64
Example
1234567890
numberOfRejectedRows Number of rejected rows integer Optional
FormatINT64
Example
0
accountId Account ID string Required
Example
8494b016-b5ef-4b9c-b16d-8b1f824d7616
workspaceId Workspace ID string Required
Example
57ce63d3e4b0681c36d1a1c4
planId Plan execution ID string Optional
errorType Error type if error occurs string Optional
errorMessage Error message if error occurs string Optional
status Task execution status string Required
Enum
  • dispatching
  • deploy_failed
  • executing
  • execution_rejected
  • execution_successful
  • execution_failed
  • terminated
  • terminated_timeout
Example
execution_successful

Workspaceinfo

Object
Name Description Type Attributes and examples
id Workspace identifier string Required
Example
57f64991e4b0b689a64feed2
name Workspace name string Required
Example
Personal
description Workspace description string Optional
Example
workspace detail description
owner Workspace owner string Optional
Example
admin
type Workspace type string Required
Enum
  • shared
  • personal
  • custom
Example
custom
environment Environmentinfo Optional

TaskExecutionsFilters

Object
Name Description Type Attributes and examples
limit the number of items to return. Must be in range [1, 100]. Default value - 100 number Optional
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 number Optional
status filter on status of execution : ‘dispatching’, ‘deploy_failed’, ‘executing’, ‘execution_successful’, ‘execution_rejected’, ‘execution_failed’, ‘terminated’, ‘terminated_timeout string Optional
Enum
  • dispatching
  • deploy_failed
  • executing
  • execution_successful
  • execution_rejected
  • execution_failed
  • terminated
  • terminated_timeout
from Time period filter start date time (milliseconds) number Optional
to Time period filter end date time (milliseconds) number Optional
lastDays Number of days in the past number Optional
Example
1
environmentId Environment filter string Optional
Example
6148511d6ccc6d41563574e5
workspaceId Workspace filter string Optional
Example
6148511d6ccc6d41563574e5
tags array of string Optional
Datatype details
Type Description Attributes and examples
array Task multiple tags filter
Example
critical + finance-eu
string One elementary tag to filter execution
Example
critical
TaskExecutionsFilters
{
  "limit": 100,
  "offset": 10,
  "status": "execution_rejected",
  "from": 1646159400000,
  "to": 1649183400000,
  "lastDays": 5,
  "environmentId": "57f64991e4b0b689a64feed3",
  "workspaceId": "57f64991e4b0b689a64feed2",
  "tags": [
    "critical",
    "supply-asia"
  ]
}

Get available Tasks Executions

POST /processing/executables/tasks/executions

Get available Tasks Executions. Available filters are :

  • limit to a number of days in recent history (1 : yesterday, 5 : past five days)
  • DateTime period in the past
  • Status of execution
  • some specific Environment : alternate filter to Workspace
  • some specific Workspace : alternate filter to Environment
  • tags associated with tasks : when multiple tags provided, each one of them will be used as filter (ie logical AND applied)

All those parameters are optional and can be combined at your convenience, apart from Environment and Workspace which are exclusive (ie you should not provide both in a same call)

Use cases

Request

Body
application/json
TaskExecutionsFilters
application/json 
{
  "limit": 100,
  "offset": 10,
  "status": "executing",
  "from": 1646159400000,
  "to": 1649183400000,
  "lastDays": 20,
  "environmentId": "57f64991e4b0b689a64feed3",
  "workspaceId": "57f64991e4b0b689a64feed2",
  "tags": [
    "critical",
    "finance-eu"
  ]
}

Response

200Status 200
Ok
Body
application/json
Page
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get Tasks Executions

GET /processing/executables/tasks/executions
Get available Tasks Executions

Request

Query parameters
Name Description Type Attributes and examples
workspaceId workspace ID the tasks should belong to string Optional
limit the number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
FormatINT32
status execution status string Optional
Enum
  • dispatching
  • deploy_failed
  • executing
  • execution_successful
  • execution_rejected
  • execution_failed
  • terminated
from from date time (milliseconds) integer Optional
FormatINT64
lastDays lastDays integer Optional
FormatINT32
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
FormatINT32
to to date time (milliseconds) integer Optional
FormatINT64
environmentId environment ID the tasks should belong to string Optional

Response

200Status 200
OK
Body
application/json
Page
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get Task Executions

GET /processing/executables/tasks/{taskId}/executions
Get task executions

Use cases

Request

Path variables
Name Description Type Attributes and examples
taskId task ID string Required
Query parameters
Name Description Type Attributes and examples
limit the number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
FormatINT32
status execution status string Optional
Enum
  • dispatching
  • deploy_failed
  • executing
  • execution_successful
  • execution_rejected
  • execution_failed
  • terminated
lastDays lastDays integer Optional
FormatINT32
from from date time (milliseconds) integer Optional
FormatINT64
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
FormatINT32
to to date time (milliseconds) integer Optional
FormatINT64

Response

200Status 200
OK
Body
application/json
Page
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Execute Task

POST /processing/executions
Execute Task

Use cases

Request

Body
application/json
Executabletask

Response

201Status 201
Execution started
Body
application/json
Executionidentifier
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
application/json
ErrorResponse
404Status 404
Not found - resource not found
Body
application/json
ErrorResponse
405Status 405
Current Task is paused, it can’t be executed
500Status 500
Server error - something went wrong on server side
Body
application/json
ErrorResponse
501Status 501
Task cannot be deployed because it is not configured
Body
application/json
ErrorResponse

Get Task execution status

GET /processing/executions/{executionId}
Get Task execution status

Request

Path variables
Name Description Type Attributes and examples
executionId execution ID string Required

Response

200Status 200
Successful response
Body
application/json
JobExecutionStatusV21
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
application/json
ErrorResponse
404Status 404
Not found - resource not found
Body
application/json
ErrorResponse
429Status 429
Too many requests
Body
application/json
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
application/json
ErrorResponse

Terminate Task execution

DELETE /processing/executions/{executionId}
Terminate Task execution

Request

Path variables
Name Description Type Attributes and examples
executionId execution ID string Required

Response

200Status 200
Execution already finished
Body
text/plain
string
204Status 204
Execution stopped succesfully
400Status 400
Bad Request
Body
text/plain
ErrorResponse
401Status 401
Unauthorized
Body
text/plain
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
text/plain
ErrorResponse
404Status 404
Not found - resource not found
Body
text/plain
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
text/plain
ErrorResponse

Plan Executions

PlanExecutable

Object
Name Description Type Attributes and examples
executable Executable identifier string Required
Example
b91cf8b2-5dd1-4b18-915b-4c447cee5267
executionPlanId The plan execution identifier in case of re-execution string Optional
Example
0798b8d1-0e12-472f-be02-a0f04e792daa
stepId The step identifier to start with string Optional
Example
09043c9f-02d0-41f6-b3cb-0ea53ffde377
rerunOnlyFailedTasks Optionally run only failing tasks boolean Optional

PlanExecutionStatus

Plan Execution info
Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
Example
7b2b122e-d6b8-42de-b0ba-fa2f0d36306e
startTimestamp Start time of job execution (UTC) datetime Required
FormatRFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
FormatRFC3339
userId User who triggered or scheduled the execution string Required
Example
fupton
userType Type of user who triggered or scheduled the execution string Required
Enum
  • HUMAN
  • SERVICE
planId Plan ID string Required
Example
b91cf8b2-5dd1-4b18-915b-4c447cee5267
executionStatus Execution (detailed) status string Required
Enum
  • UNDEFINED
  • STARTED
  • FINISHED
  • EXECUTION_FAILED
  • EXECUTION_SUCCESS
Example
FINISHED
plannedExecutableCount Number of planned executables integer Optional
FormatINT32
Example
2
doneExecutableCount Number of done executables integer Optional
FormatINT32
Example
1
doneExecutableDetails array of JobExecutionStatus Optional
Datatype details
Type Description Attributes and examples
array Execution activity info
JobExecutionStatus

StepExecution

Object
Name Description Type Attributes and examples
id string Optional
executionId string Optional
startTimestamp datetime Optional
FormatRFC3339
finishTimestamp datetime Optional
FormatRFC3339
executionStatus string Optional

Execute Plan

POST /processing/executions/plans

Allows to run a Plan.

This endpoint also allows for re-running some plan execution from some failed step : you should then provide execution id and step from which you wish to restart execution.

Request

Body
application/json
PlanExecutable

Response

201Status 201
Execution started
Body
application/json
Executionidentifier
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
application/json
ErrorResponse
404Status 404
Not found - resource not found
Body
application/json
ErrorResponse
405Status 405
Plan is paused, it can’t be executed
422Status 422
Entity exists but cannot be processed, because too old or plan exec didn’t fail
Body
application/json
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
application/json
ErrorResponse
501Status 501
Plan cannot be deployed because it is not configured
Body
application/json
ErrorResponse

Get available Plans executions

GET /processing/executables/plans/executions
List all plan executions the user is granted visibility on. For each plan execution returns execution and plan identifiers, start and finish timestamps, execution status and counters on done and planned executables within plan. This query is paginated and accepts filters as defined in parameters documented above.

Request

Query parameters
Name Description Type Attributes and examples
workspaceId workspace ID the plans should belong to string Optional
limit the number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
FormatINT32
status execution status string Optional
Enum
  • executing
  • execution_successful
  • execution_failed
lastDays lastDays integer Optional
FormatINT32
from from date time (milliseconds) integer Optional
FormatINT64
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
FormatINT32
environmentId environment ID the plans should belong to string Optional
to to date time (milliseconds) integer Optional
FormatINT64

Response

200Status 200
OK
Body
application/json
Page
Status 200 application/json 
{
  "items": [
    {
      "executionId": "7fb48c7a-5ea5-4cb9-9333-e345f28d9dd5",
      "startTimestamp": "2022-10-20T14:20:28.582Z",
      "finishTimestamp": "2022-10-20T14:20:39.830Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "dc57c82f-7f18-4228-9f82-a81e02bae617",
      "startTimestamp": "2022-10-20T14:17:56.390Z",
      "finishTimestamp": "2022-10-20T14:18:07.865Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "4555c45f-9548-41e9-98b4-a9bfa2ee4e5c",
      "startTimestamp": "2022-10-20T13:47:37.611Z",
      "finishTimestamp": "2022-10-20T13:47:48.864Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "f3cfb077-2ea6-4557-bbb2-fd027f3cae4e",
      "startTimestamp": "2022-10-20T13:45:04.027Z",
      "finishTimestamp": "2022-10-20T13:45:15.929Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    },
    {
      "executionId": "b668cd50-6421-4e57-b15a-79b39ac153a8",
      "startTimestamp": "2022-10-20T13:18:01.235Z",
      "finishTimestamp": "2022-10-20T13:18:16.093Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_FAILED",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_failed"
    },
    {
      "executionId": "226dc12c-a288-4fe0-96f6-f6d51fa2b9ff",
      "startTimestamp": "2022-10-20T12:08:48.823Z",
      "finishTimestamp": "2022-10-20T12:18:52.729Z",
      "planId": "e7734729-cf3d-4578-939c-30381e95cd10",
      "executionStatus": "EXECUTION_SUCCESS",
      "plannedExecutableCount": 1,
      "doneExecutableCount": 1,
      "status": "execution_successful"
    }
  ],
  "limit": 100,
  "offset": 0,
  "total": 6
}
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get all steps for a plan execution (order by designed execution, only steps, without error handlers)

GET /processing/executions/plans/{planExecutionId}/steps
Get Steps executions status

Request

Path variables
Name Description Type Attributes and examples
planExecutionId Plan execution ID string Required

Response

200Status 200
Successful response
Body
application/json
details
Name Description Type Attributes and examples
array of StepExecution
Datatype details
Type Description Attributes and examples
array
StepExecution
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
application/json
ErrorResponse
404Status 404
Not found - resource not found
Body
application/json
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
application/json
ErrorResponse

Get Plan executions

GET /processing/executables/plans/{planId}/executions
Get Plan executions

Request

Path variables
Name Description Type Attributes and examples
planId plan ID string Required
Query parameters
Name Description Type Attributes and examples
limit the number of items to return. Must be in range [1, 100]. Default value - 100 integer Optional
FormatINT32
status execution status string Optional
Enum
  • executing
  • execution_successful
  • execution_failed
lastDays lastDays integer Optional
FormatINT32
from from date time (milliseconds) integer Optional
FormatINT64
offset the position of the first item. Must be greater than or equal to 0. Default value - 0 integer Optional
FormatINT32
to to date time (milliseconds) integer Optional
FormatINT64

Response

200Status 200
OK
Body
application/json
Page
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get Plan Execution status

GET /processing/executions/plans/{id}
Get detailed status of one plan execution

Request

Path variables
Name Description Type Attributes and examples
id Execution id string Required

Response

200Status 200
Detailed status on one Platform execution
Body
application/json
PlanExecutionStatus
400Status 400
Status 400
Body
*/*
ErrorResponse
401Status 401
Status 401
Body
application/json
ErrorResponse
403Status 403
Permissions required
Body
*/*
ErrorResponse
404Status 404
Execution not found
Body
*/*
ErrorResponse
429Status 429
Status 429
Body
*/*
ErrorResponse
500Status 500
Something went wrong on server side
Body
*/*
ErrorResponse

Get the step for plan execution without error handlers

GET /processing/executions/plans/{planExecutionId}/steps/{stepExecutionId}
Get Step execution status

Request

Path variables
Name Description Type Attributes and examples
planExecutionId Plan execution ID string Required
stepExecutionId Step execution ID string Required

Response

200Status 200
Successful response
Body
application/json
PlanExecutionStatus
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
application/json
ErrorResponse
404Status 404
Not found - resource not found
Body
application/json
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
application/json
ErrorResponse

Get execution status of the handler for the specified step

GET /processing/executions/plans/{planExecutionId}/steps/{stepExecutionId}/error-handler
Get Step handler execution status

Request

Path variables
Name Description Type Attributes and examples
planExecutionId Plan execution ID string Required
stepExecutionId Step execution ID string Required

Response

200Status 200
Successful response
Body
application/json
PlanExecutionStatus
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden - no permissions to access resource
Body
application/json
ErrorResponse
404Status 404
Not found - resource not found
Body
application/json
ErrorResponse
500Status 500
Server error - something went wrong on server side
Body
application/json
ErrorResponse

Promotion Executions

AdvancedPromotionSpec

Object
Name Description Type Attributes and examples
artifactId Advanced Promotion Artifact Id string Optional
Example
57f64991e4b0b689a64feed0
artifactType Advanced Promotion Artifact Type. In the following list of allowed values, ‘ACTION’ refers to Artifacts, ‘FLOW’ refers to Tasks, ‘PLAN’ refers to Plans and ‘WORKSPACE’ refers to Workspaces string Optional
Enum
  • WORKSPACE
  • PLAN
  • FLOW
  • ACTION
Example
WORKSPACE

ArtifactPromotionResult

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
Enum
  • WORKSPACE
  • PLAN
  • FLOW
  • ACTION
  • CONNECTION
  • RESOURCE
  • ENGINE
  • CLUSTER
  • FOLDER
Example
WORKSPACE
analyzeReport Report Optional
promotionReport Report Optional
usedBy array of PromotionResultInfo Optional
Datatype details
Type Description Attributes and examples
array Used by Artifacts
PromotionResultInfo
targetVersion applicable to artifacts with type FLOW only, version of flow in target environment after promotion execution string Optional
Example
1.1
version applicable to artifacts with type FLOW only, version of flow in source environment integer Optional
FormatINT32
Example
1

Execution

Object
Name Description Type Attributes and examples
executionId string Optional
executionReport PromotionExecutionStatusV26 Optional

PromotionExecutableTaskV26

Object
Name Description Type Attributes and examples
executable Executable identifier string Required
Example
57f64991e4b0b689a64feed0
keepTargetResources Keep Target Resources flag boolean Optional
keepTargetRunProfiles Keep Target Run Profiles flag (works starting from V2.2) boolean Optional
advanced AdvancedPromotionSpec Optional
context Execution context message string Optional
Max length255
Example
Free text describing the context for promotion execution

PromotionExecutionStatus

Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
Example
7b2b122e-d6b8-42de-b0ba-fa2f0d36306e
startTimestamp Start time of job execution (UTC) datetime Required
FormatRFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
FormatRFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
FormatRFC3339
userId User who triggered or scheduled the execution string Required
Example
fupton
promotionId Promotion ID string Optional
pipelineId Deprecated! Use Promotion ID instead string Optional
keepTargetResources Keep Target Resources flag boolean Optional
keepTargetRunProfiles Keep Target Run Profiles flag (not returned for API versions earlier V2.2) boolean Optional
advanced AdvancedPromotionSpec Optional
defective Defective Promotion flag boolean Optional
status Execution status string Optional
Enum
  • ANALIZED
  • PROMOTION_STARTED
  • PROMOTION_WARNING
  • PROMOTION_FAILED
  • PROMOTED
Example
PROMOTED
statusMessage Execution status message string Optional
workspaces array of WorkspacePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Workspace Promotion Execution Info
WorkspacePromotionResult
engines array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engines Promotion Execution Info
RuntimePromotionResult
clusters array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engine Clusters Promotion Execution Info
RuntimePromotionResult

PromotionExecutionStatusV26

Object
Name Description Type Attributes and examples
executionId Job execution ID string Required
Example
7b2b122e-d6b8-42de-b0ba-fa2f0d36306e
startTimestamp Start time of job execution (UTC) datetime Required
FormatRFC3339
triggerTimestamp Trigger time of job execution (UTC) datetime Required
FormatRFC3339
finishTimestamp End time of job execution (UTC) datetime Optional
FormatRFC3339
userId User who triggered or scheduled the execution string Required
Example
fupton
promotionId Promotion ID string Optional
pipelineId Deprecated! Use Promotion ID instead string Optional
keepTargetResources Keep Target Resources flag boolean Optional
keepTargetRunProfiles Keep Target Run Profiles flag (not returned for API versions earlier V2.2) boolean Optional
advanced AdvancedPromotionSpec Optional
defective Defective Promotion flag boolean Optional
status Execution status string Optional
Enum
  • ANALIZED
  • PROMOTION_STARTED
  • PROMOTION_WARNING
  • PROMOTION_FAILED
  • PROMOTED
Example
PROMOTED
statusMessage Execution status message string Optional
workspaces array of WorkspacePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Workspace Promotion Execution Info
WorkspacePromotionResult
engines array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engines Promotion Execution Info
RuntimePromotionResult
clusters array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Remote Engine Clusters Promotion Execution Info
RuntimePromotionResult
context Execution context message string Optional
Max length255
Example
Free text describing the context for promotion execution

PromotionResultInfo

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
Enum
  • WORKSPACE
  • PLAN
  • FLOW
  • ACTION
  • CONNECTION
  • RESOURCE
  • ENGINE
  • CLUSTER
  • FOLDER
Example
WORKSPACE
analyzeReport Report Optional
promotionReport Report Optional
targetVersion Applicable to flows only, displays version of flow in target environment after promotion execution string Optional
Example
1.1
version Applicable to flows only, displays version of flow in source environment integer Optional
FormatINT32

Report

Object
Name Description Type Attributes and examples
status string Optional
Enum
  • CREATED
  • REPLACED
  • REUSED
  • FAILED
  • SKIPPED
messages array of Message Optional
Datatype details
Type Description Attributes and examples
array
Message

RuntimePromotionResult

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
Enum
  • WORKSPACE
  • PLAN
  • FLOW
  • ACTION
  • CONNECTION
  • RESOURCE
  • ENGINE
  • CLUSTER
  • FOLDER
Example
WORKSPACE
analyzeReport Report Optional
promotionReport Report Optional
usedBy array of PromotionResultInfo Optional
Datatype details
Type Description Attributes and examples
array Used by Artifacts
PromotionResultInfo
runProfiles array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Run profiles
ArtifactPromotionResult

WorkspacePromotionResult

Object
Name Description Type Attributes and examples
id Artifact ID string Optional
targetId Target Artifact ID string Optional
name Artifact Name string Optional
type Artifact Type string Optional
Enum
  • WORKSPACE
  • PLAN
  • FLOW
  • ACTION
  • CONNECTION
  • RESOURCE
  • ENGINE
  • CLUSTER
  • FOLDER
Example
WORKSPACE
analyzeReport Report Optional
promotionReport Report Optional
flows array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Flows Promotion Execution Info
ArtifactPromotionResult
plans array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Plans Promotion Execution Info
ArtifactPromotionResult
actions array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Actions Promotion Execution Info
ArtifactPromotionResult
connections array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Connections Promotion Execution Info
ArtifactPromotionResult
resources array of ArtifactPromotionResult Optional
Datatype details
Type Description Attributes and examples
array Resources Promotion Execution Info
ArtifactPromotionResult
engines array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Engines Promotion Execution Info
RuntimePromotionResult
clusters array of RuntimePromotionResult Optional
Datatype details
Type Description Attributes and examples
array Clusters Promotion Execution Info
RuntimePromotionResult

Execute a Promotion

POST /processing/executions/promotions
Execute a Promotion

Request

Body
application/json
PromotionExecutableTaskV26

Response

201Status 201
Execution success
Body
application/json
Execution
202Status 202
Execution started
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse
501Status 501
Promotions can’t be executed because it has errors
Body
application/json
ErrorResponse

Get Promotion execution status

GET /processing/executions/promotions/{executionId}
Get Promotion execution status

Request

Path variables
Name Description Type Attributes and examples
executionId Execution ID string Required

Response

200Status 200
OK
Body
application/json
PromotionExecutionStatus
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Remote Engines

Engine

Object
Name Description Type Attributes and examples
id Resource id string Required
Example
5cdd2ce2c737973f9d581b98
name Resource name string Required
Example
Remote Engine
description Resource description string Optional
Example
Some description
workspace Workspaceinfo Optional
createDate Date of creation of the resource datetime Required
FormatRFC3339
updateDate Date of updating of the resource datetime Optional
FormatRFC3339
runtimeId Resource runtime id string Required
Example
9e8acf72-00d5-451b-9175-97bd44cd6b13
availability Availability status of engine|cluster string Optional
Enum
  • AVAILABLE
  • NOT_AVAILABLE
  • RETIRED
Example
NOT_AVAILABLE
managed Indicates whether target runtime (engine/cluster) is managed or not boolean Optional
status Engine status string Required
Enum
  • PAIRED
  • NOT_PAIRED
Example
NOT_PAIRED
runProfiles array of string Required
Datatype details
Type Description Attributes and examples
array Run profiles of engine
Example
["MICROSERVICE"]
string
debug EngineDebug Optional
clusterId cluster that this engine is part of string Optional
preAuthorizedKey Preauthorized key (only for not paired engine) string Optional
Example
123A822BF26DF49045C90D94CD269F3850BFB308B2606019CC63EDADE55B4391
esbCompatibilityVersion ESB compatibility version string Optional
Example
7.3.1

EngineDebug

Object
Name Description Type Attributes and examples
host Debug host of engine string Required
Example
192.168.0.1

EngineRequest

Object
Name Description Type Attributes and examples
name Engine name string Required
Example
Remote Engine
environmentId Id of engine environment string Required
Example
5c1111d7a4186a4eafed0587
workspaceId Id of engine workspace string Optional
Example
5c2222d7a4186a4eafed0587
runProfiles array of string Optional
Datatype details
Type Description Attributes and examples
array Run profiles of engine
Example
["MICROSERVICE"]
string
debug EngineDebug Optional
description Engine|cluster description string Optional
Example
Some description
esbCompatibilityVersion Data Service Runner/Talend Runtime version. Mandatory if TALEND_RUNTIME or MICROSERVICE profile selected string Optional
Example
7.2.1

RunProfileRequest

Object
Name Description Type Attributes and examples
name Run profile name string Required
Example
Run profile
description Description string Optional
Example
Run profile with default options
jvmArguments array of string Required
Datatype details
Type Description Attributes and examples
array Jvm arguments
Example
["-Xms128m","-Xmx256m"]
string
type Run profile type string Required
Enum
  • JOB_SERVER
  • MICROSERVICE
  • TALEND_RUNTIME
Example
JOB_SERVER
runtimeId Runtime id string Required
Example
5e713618de3c75a450447262

RunProfileUpdateRequest

Object
Name Description Type Attributes and examples
name Run profile name string Required
Example
Run profile
description Description string Optional
Example
Run profile with default options
jvmArguments array of string Required
Datatype details
Type Description Attributes and examples
array Jvm arguments
Example
["-Xms128m","-Xmx256m"]
string

RuntimeRunProfile

Object
Name Description Type Attributes and examples
id string Optional
name string Optional
description string Optional
createDate datetime Optional
FormatRFC3339
updateDate datetime Optional
FormatRFC3339
type Run profile type string Optional
Enum
  • JOB_SERVER
  • MICROSERVICE
  • TALEND_RUNTIME
jvmArguments array of string Optional
Datatype details
Type Description Attributes and examples
array Jvm arguments
string
runtimeId string Optional
version Run profile version integer Optional
FormatINT32

Create new Remote Engine

POST /processing/runtimes/remote-engines
Create new Remote Engine

Request

Body
*/*
EngineRequest

Response

201Status 201
Created
Body
application/json
Engine
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get all (available) Remote Engines

GET /processing/runtimes/remote-engines
Get all (available) Remote Engines

Request

Query parameters
Name Description Type Attributes and examples
query search query (FIQL format) string Optional
Example
workspace.environment.id==5cb47ca4b1b5247f6006529e

Response

200Status 200
OK
Body
application/json
details
Name Description Type Attributes and examples
array of Engine
Datatype details
Type Description Attributes and examples
array
Engine
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Delete Remote Engine by id

DELETE /processing/runtimes/remote-engines/{id}
Delete Remote Engine by id

Request

Path variables
Name Description Type Attributes and examples
id remote engine id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
409Status 409
Conflict
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get Remote Engine by id

GET /processing/runtimes/remote-engines/{id}
Get Remote Engine by id

Request

Path variables
Name Description Type Attributes and examples
id remote engine id string Required

Response

200Status 200
OK
Body
application/json
Engine
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Unpair Remote Engine

DELETE /processing/runtimes/remote-engines/{id}/pairing
Unpair Remote Engine

Request

Path variables
Name Description Type Attributes and examples
id remote engine id string Required

Response

200Status 200
OK
Body
text/plain
string
400Status 400
Bad Request
Body
text/plain
ErrorResponse
401Status 401
Unauthorized
Body
text/plain
ErrorResponse
403Status 403
Forbidden
Body
text/plain
ErrorResponse
404Status 404
Not Found
Body
text/plain
ErrorResponse
500Status 500
Internal Server Error
Body
text/plain
ErrorResponse

Create engine run profile

POST /processing/runtimes/remote-engines/{engineId}/run-profiles
Create engine run profile

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
Body
application/json
RunProfileRequest

Response

201Status 201
Created
Body
application/json
RuntimeRunProfile
400Status 400
Bad run profile definition
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
No engine can be found by provided
Body
application/json
ErrorResponse
500Status 500
Internal server error
Body
application/json
ErrorResponse

Get all engine run profiles

GET /processing/runtimes/remote-engines/{engineId}/run-profiles
Get all engine run profiles

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required

Response

200Status 200
OK
Body
application/json
details
Name Description Type Attributes and examples
array of RuntimeRunProfile
Datatype details
Type Description Attributes and examples
array
RuntimeRunProfile
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Engine not found
Body
application/json
ErrorResponse

Get engine run profile by id

GET /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}
Get engine run profile by id

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
runProfileId run profile id string Required

Response

200Status 200
OK
Body
application/json
RuntimeRunProfile
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Run profile not found
Body
application/json
ErrorResponse

Update engine run profile

PUT /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}
Update engine run profile

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
runProfileId run profile id string Required
Body
application/json
RunProfileUpdateRequest

Response

200Status 200
OK
Body
application/json
RuntimeRunProfile
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Run profile not found
Body
application/json
ErrorResponse
500Status 500
Internal server error
Body
application/json
ErrorResponse

Delete engine run profile

DELETE /processing/runtimes/remote-engines/{engineId}/run-profiles/{runProfileId}
Delete engine run profile

Request

Path variables
Name Description Type Attributes and examples
engineId remote engine id string Required
runProfileId run profile id string Required
Query parameters
Name Description Type Attributes and examples
runProfileId new run profile id string Optional

Response

204Status 204
No Content
403Status 403
Forbidden
Body
*/*
ErrorResponse
404Status 404
No run profile can be found
Body
*/*
ErrorResponse
500Status 500
Internal server error
Body
*/*
ErrorResponse

Remote Engine Clusters

ClusterRequest

Object
Name Description Type Attributes and examples
name Cluster name string Required
Example
Some cluster name
workspaceId Workspace identifier string Required
Example
5c935c8ba4186a4eafed0592
environmentId Environment identifier string Required
Example
5c935c8ba4186a4eafed0592
description Cluster description string Required
Example
some description
remoteEngines array of string Required
Datatype details
Type Description Attributes and examples
array List of remote engine ids
Example
["5d3039de65ff7d3583840bc0","5d3039de65ff7d3583840bk1"]
string

EngineCluster

Object
Name Description Type Attributes and examples
id Resource id string Required
Example
5cdd2ce2c737973f9d581b98
name Resource name string Required
Example
Remote Engine
description Resource description string Optional
Example
Some description
workspace Workspaceinfo Optional
createDate Date of creation of the resource datetime Required
FormatRFC3339
updateDate Date of updating of the resource datetime Optional
FormatRFC3339
runtimeId Resource runtime id string Required
Example
9e8acf72-00d5-451b-9175-97bd44cd6b13
availability Availability status of engine|cluster string Optional
Enum
  • AVAILABLE
  • NOT_AVAILABLE
  • RETIRED
Example
NOT_AVAILABLE
managed Indicates whether target runtime (engine/cluster) is managed or not boolean Optional
engines array of string Optional
Datatype details
Type Description Attributes and examples
array Engines in cluster
Example
["5d3039de65ff7d3583840bc0","5d3039de65ff7d3583840bk1"]
string

Create Remote Engine Cluster

POST /processing/runtimes/remote-engine-clusters
Create Remote Engine Cluster

Request

Body
application/json
ClusterRequest

Response

201Status 201
Created
Body
application/json
EngineCluster
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get all (available) Remote Engine Clusters

GET /processing/runtimes/remote-engine-clusters
Get all (available) Remote Engine Clusters

Request

Query parameters
Name Description Type Attributes and examples
_s search query (FIQL format), e.g. “workspace.environment.id==5cb47ca4b1b5247f6006529e”,“name==NewCluster” string Optional

Response

200Status 200
OK
Body
application/json
details
Name Description Type Attributes and examples
array of BaseEngine
Datatype details
Type Description Attributes and examples
array
BaseEngine
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get Remote Engine Cluster by id

GET /processing/runtimes/remote-engine-clusters/{clusterId}
Get Remote Engine Cluster by id

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required

Response

200Status 200
OK
Body
application/json
EngineCluster
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Delete Remote Engine Cluster by id

DELETE /processing/runtimes/remote-engine-clusters/{clusterId}
Delete Remote Engine Cluster by id

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
*/*
ErrorResponse
401Status 401
Unauthorized
Body
*/*
ErrorResponse
403Status 403
Forbidden
Body
*/*
ErrorResponse
404Status 404
Not Found
Body
*/*
ErrorResponse
500Status 500
Internal Server Error
Body
*/*
ErrorResponse

Add Remote Engine to Remote Engine Cluster

PUT /processing/runtimes/remote-engine-clusters/{clusterId}/engines/{engineId}
Add Remote Engine to Remote Engine Cluster

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required
engineId remote engine id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
*/*
ErrorResponse
401Status 401
Unauthorized
Body
*/*
ErrorResponse
403Status 403
Forbidden
Body
*/*
ErrorResponse
404Status 404
Not Found
Body
*/*
ErrorResponse
409Status 409
Conflict
Body
*/*
ErrorResponse
500Status 500
Internal Server Error
Body
*/*
ErrorResponse

Remove Remote Engine from Remote Engine Cluster

DELETE /processing/runtimes/remote-engine-clusters/{clusterId}/engines/{engineId}
Remove Remote Engine from Remote Engine Cluster

Request

Path variables
Name Description Type Attributes and examples
clusterId remote engine cluster id string Required
engineId remote engine id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get cluster run profiles

GET /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles
Get cluster run profiles

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required

Response

200Status 200
OK
Body
application/json
details
Name Description Type Attributes and examples
array of RuntimeRunProfile
Datatype details
Type Description Attributes and examples
array
RuntimeRunProfile
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Cluster not found
Body
application/json
ErrorResponse

Create cluster run profile

POST /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles
Create cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
Body
application/json
RunProfileRequest

Response

400Status 400
Bad run profile definition
Body
application/json
RuntimeRunProfile
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
No cluster can be found by provided
Body
application/json
ErrorResponse
500Status 500
Internal server error
Body
application/json
ErrorResponse

Update cluster run profile

PUT /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}
Update cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
runProfileId run profile id string Required
Body
application/json
RunProfileUpdateRequest

Response

200Status 200
OK
Body
application/json
RuntimeRunProfile
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
No run profile can be found
Body
application/json
ErrorResponse
500Status 500
Internal server error
Body
application/json
ErrorResponse

Delete cluster run profile

DELETE /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}
Delete cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
runProfileId run profile id string Required
Query parameters
Name Description Type Attributes and examples
runProfileId new run profile id string Optional

Response

204Status 204
No Content
403Status 403
Forbidden
Body
*/*
ErrorResponse
404Status 404
No run profile can be found
Body
*/*
ErrorResponse
500Status 500
Internal server error
Body
*/*
ErrorResponse

Get cluster run profile

GET /processing/runtimes/remote-engine-clusters/{clusterId}/run-profiles/{runProfileId}
Get cluster run profile

Request

Path variables
Name Description Type Attributes and examples
clusterId cluster id string Required
runProfileId run profile id string Required

Response

200Status 200
OK
Body
application/json
details
Name Description Type Attributes and examples
array of RuntimeRunProfile
Datatype details
Type Description Attributes and examples
array
RuntimeRunProfile
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Run profle not found
Body
application/json
ErrorResponse

Remote Engine Gen2

AdvancedRunProfile

Pipeline run profile of type ‘Advanced’
Object
Name Description Type Attributes and examples
name Name string Required
Example
Default run profile
description Description string Optional
Example
Run profile used by default
engineId Pipeline engine id string Optional
Example
5f7eff3245ecf259a1c56daa
definition Json with definition string Optional
Example
{"id": 1}

BigDataRunProfile

Pipeline run profile of type ‘Big Data’
Object
Name Description Type Attributes and examples
name Name string Required
Example
Default run profile
description Description string Optional
Example
Run profile used by default
engineId Pipeline engine id string Optional
Example
5f7eff3245ecf259a1c56daa
distribution Distribution string Optional
Example
EMR 5.x (Hadoop 2.7)
executionFramework Execution framework string Optional
Example
Spark on Yarn
microBatchInterval Micro-batch interval (in ms) integer Required
FormatINT32
Example
1000
streamingTimeout Streaming timeout (in ms) integer Required
FormatINT32
Example
10
yarnQueue Yarn queue string Optional
Example
q1
driverCores Driver cores integer Optional
FormatINT32
Example
1
driverMemory Driver cores (in MB) integer Optional
FormatINT32
Example
512
temporaryStorage Temporary storage string Required
Example
/tmp-storage
yarnStrategy YarnStrategy Required
executorMemoryOverhead Execution memory overhead (MB) integer Required
FormatINT32
Example
384
isCheckpointingEnabled Checkpointing boolean Optional
checkPointFilePath Checkpoint file path. Required if checkpointing enabled string Optional
Example
/file-path
parameters Parameters ObjectOptional
id Run profile id string Optional
Example
57f64991e4b0b689a64feed0
type Type string Optional
Enum
  • standard
  • big-data
  • databricks
  • advanced
createDate Created on datetime Optional
FormatRFC3339
updateDate Updated on datetime Optional
FormatRFC3339

Databrick

Object
Name Description Type Attributes and examples
endpoint Endpoint string Required
Example
https://endpoint
token Token (can also be provided in the Remote Engine configuration) string Optional
Example
1
stagingLocation DBFS library staging directory string Required
Example
dbfs:/staging/
target DatabricksCluster Required
cloudProvider Cloud provider string Required
Enum
  • AWS
  • Azure
Example
AWS

DatabricksCluster

Object
Name Description Type Attributes and examples
cluster New or existing cluster string Required
Enum
  • new
  • old
Example
new
id ID of existing cluster. Required for ‘old’ cluster string Optional
Example
132553
sparkVersion Databricks runtime version. Required for ‘new’ cluster string Optional
Example
5.5.x-scala2.11
nodeTypeId Node type ID. Required for ‘new’ cluster string Required
Example
m4.large
logsDestination Logs destination. Required for ‘new’ cluster string Required
Example
dbfs:/cluster-logs
workersNumber Number of workers. Required for ‘new’ cluster integer Required
FormatINT32
Example
2

DatabricksRunProfile

Pipeline run profile of type ‘Databricks’
Object
Name Description Type Attributes and examples
name Name string Required
Example
Default run profile
description Description string Optional
Example
Run profile used by default
engineId Pipeline engine id string Optional
Example
5f7eff3245ecf259a1c56daa
microBatchInterval Micro-batch interval (in ms) integer Required
FormatINT32
Example
1000
databricks Databrick Optional
parameters Parameters ObjectOptional
id Run profile id string Optional
Example
57f64991e4b0b689a64feed0
type Type string Optional
Enum
  • standard
  • big-data
  • databricks
  • advanced
createDate Created on datetime Optional
FormatRFC3339
updateDate Updated on datetime Optional
FormatRFC3339

DataIntegrationRunProfile

Object
Name Description Type Attributes and examples
name string Required
description string Optional
jvmArguments array of string Optional
Datatype details
Type Description Attributes and examples
array jvm arguments applied on job execution
Example
["-XX:PermSize=128m", "-XX:MaxHeapFreeRatio=70"]
string

DataIntegrationRunProfileResponse

Object
Name Description Type Attributes and examples
id string Optional
Example
"62444fbce9f05d51f8da4031"
name string Required
Example
"Default run profile"
description string Optional
Example
"run profile description"
engineId string Optional
Example
"6246ea2275ca6f0e05bc4a87"
runtimeId string Optional
Example
"f4422f10-4152-4a3f-adae-3b97ba3ba918"
jvmArguments string Optional
Example
["-XX:MaxHeapFreeRatio=70", "-XX:PermSize=128m"]
type string Optional
Enum
  • data-integration
createDate date Optional
updateDate string Optional

PipelineEngine

Object
Name Description Type Attributes and examples
id Resource id string Required
Example
5cdd2ce2c737973f9d581b98
name Resource name string Required
Example
Remote Engine
description Resource description string Optional
Example
Some description
workspace Workspaceinfo Optional
createDate Date of creation of the resource datetime Required
FormatRFC3339
updateDate Date of updating of the resource datetime Optional
FormatRFC3339
runtimeId Resource runtime id string Required
Example
9e8acf72-00d5-451b-9175-97bd44cd6b13
availability Availability status of engine|cluster string Optional
Enum
  • AVAILABLE
  • NOT_AVAILABLE
  • RETIRED
Example
NOT_AVAILABLE
managed Indicates whether target runtime (engine/cluster) is managed or not boolean Optional
status string Optional
Enum
  • PAIRED
  • NOT_PAIRED
preAuthorizedKey string Optional
cloudRunner boolean Optional

PipelineEngineRequest

Object
Name Description Type Attributes and examples
name Engine name string Required
Example
Remote Engine
environmentId Id of engine environment string Required
Example
5c1111d7a4186a4eafed0587
workspaceId Id of engine workspace string Optional
Example
5c2222d7a4186a4eafed0587
description Engine|cluster description string Optional
Example
Some description

PipelineRunProfileResponse

Pipeline run profile response object.
Object
Name Description Type Attributes and examples
name Name string Required
Example
Default run profile
description Description string Optional
Example
Run profile used by default
engineId Pipeline engine id string Optional
Example
5f7eff3245ecf259a1c56daa
id Run profile id string Optional
Example
57f64991e4b0b689a64feed0
type Type string Optional
Enum
  • standard
  • big-data
  • databricks
  • advanced
  • data-integration
createDate Created on datetime Optional
FormatRFC3339
updateDate Updated on datetime Optional
FormatRFC3339

StandardRunProfile

Pipeline run profile of type ‘Standard’
Object
Name Description Type Attributes and examples
name Name string Required
Example
Default run profile
description Description string Optional
Example
Run profile used by default
engineId Pipeline engine id string Optional
Example
5f7eff3245ecf259a1c56daa
configuration Configuration Required
executionFramework Execution framework string Required
Example
Spark local

StandardRunProfileResponse

Pipeline run profile of type ‘Standard’
Object
Name Description Type Attributes and examples
name Name string Required
Example
Default run profile
description Description string Optional
Example
Run profile used by default
engineId Pipeline engine id string Optional
Example
5f7eff3245ecf259a1c56daa
configuration Configuration Required
executionFramework Execution framework string Required
Example
Spark local
id Run profile id string Optional
Example
57f64991e4b0b689a64feed0
type Type string Optional
Enum
  • standard
  • big-data
  • databricks
  • advanced
createDate Created on datetime Optional
FormatRFC3339
updateDate Updated on datetime Optional
FormatRFC3339

YarnStrategy

Object
Name Description Type Attributes and examples
type Yarn strategy type string Required
Enum
  • Fixed
  • Dynamic
initialExecutors Initial number of executors (for Dynamic strategy) integer Optional
FormatINT32
minExecutors Min. number of executors (for Dynamic strategy) integer Optional
FormatINT32
maxExecutors Max. number of executors (for Dynamic strategy) integer Optional
FormatINT32
coresPerExecutor Cores per executor integer Optional
FormatINT32
executorMemory Executor memory (in MB) integer Optional
FormatINT32
executorsNumber Number of executors (for Fixed strategy) integer Optional
FormatINT32

Create new Pipeline Engine

POST /processing/runtimes/pipeline-engines
Create new Pipeline Engine

Request

Body
application/json
PipelineEngineRequest

Response

201Status 201
Created
Body
application/json
PipelineEngine
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Get Pipeline Engines

GET /processing/runtimes/pipeline-engines
Get Pipeline Engines

Request

Query parameters
Name Description Type Attributes and examples
query search query (FIQL format), e.g. “workspace.environment.id==5cb47ca4b1b5247f6006529e”,“status==PAIRED” string Optional

Response

200Status 200
OK
Body
application/json
details
Name Description Type Attributes and examples
array of PipelineEngine
Datatype details
Type Description Attributes and examples
array
PipelineEngine
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Delete Pipeline Engine by id

DELETE /processing/runtimes/pipeline-engines/{engineId}
Delete Pipeline Engine by id

Request

Path variables
Name Description Type Attributes and examples
engineId pipeline engine id string Required

Response

204Status 204
No Content
400Status 400
Bad Request
Body
*/*
ErrorResponse
401Status 401
Unauthorized
Body
*/*
ErrorResponse
403Status 403
Forbidden
Body
*/*
ErrorResponse
404Status 404
Not Found
Body
*/*
ErrorResponse
409Status 409
Conflict
Body
*/*
ErrorResponse
500Status 500
Internal Server Error
Body
*/*
ErrorResponse

Get Pipeline Engine by id

GET /processing/runtimes/pipeline-engines/{engineId}
Get Pipeline Engine by id

Request

Path variables
Name Description Type Attributes and examples
engineId pipeline engine id string Required

Response

200Status 200
OK
Body
application/json
PipelineEngine
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Unpair Pipeline Engine

DELETE /processing/runtimes/pipeline-engines/{engineId}/pairing
Unpair Pipeline Engine

Request

Path variables
Name Description Type Attributes and examples
engineId pipeline engine id string Required

Response

200Status 200
OK
Body
text/plain
string
400Status 400
Bad Request
Body
text/plain
ErrorResponse
401Status 401
Unauthorized
Body
text/plain
ErrorResponse
403Status 403
Forbidden
Body
text/plain
ErrorResponse
404Status 404
Not Found
Body
text/plain
ErrorResponse
500Status 500
Internal Server Error
Body
text/plain
ErrorResponse

Get run profile for engine

GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles

Request

Path variables
Name Description Type Attributes and examples
engineId run profile id string Required

Response

200Status 200
successful operation
Body
application/json
details
Name Description Type Attributes and examples
array of PipelineRunProfileResponse
Datatype details
Type Description Attributes and examples
array
PipelineRunProfileResponse
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal server error
Body
application/json
ErrorResponse

Create new advanced run profile

POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
Body
application/json
AdvancedRunProfile

Response

201Status 201
Created
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Update advanced run profile

PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced/{runProfileId}

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
runProfileId run profile id string Required
Body
application/json
AdvancedRunProfile

Response

204Status 204
No Content
400Status 400
Bad Request
Body
*/*
ErrorResponse
401Status 401
Unauthorized
Body
*/*
ErrorResponse
403Status 403
Forbidden
Body
*/*
ErrorResponse
404Status 404
Not Found
Body
*/*
ErrorResponse
500Status 500
Internal Server Error
Body
*/*
ErrorResponse

Get advanced run profile

GET /processing/runtimes/pipeline-engines/{engineId}/run-profiles/advanced/{runProfileId}

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
runProfileId run profile id string Required

Response

200Status 200
OK
Body
application/json
AdvancedRunProfile
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Create new Big Data run profile

POST /processing/runtimes/pipeline-engines/{engineId}/run-profiles/big-data

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
Body
application/json
BigDataRunProfile

Response

201Status 201
Created
400Status 400
Bad Request
Body
application/json
ErrorResponse
401Status 401
Unauthorized
Body
application/json
ErrorResponse
403Status 403
Forbidden
Body
application/json
ErrorResponse
404Status 404
Not Found
Body
application/json
ErrorResponse
500Status 500
Internal Server Error
Body
application/json
ErrorResponse

Update Big Data run profile

PUT /processing/runtimes/pipeline-engines/{engineId}/run-profiles/big-data/{runProfileId}

Request

Path variables
Name Description Type Attributes and examples
engineId engine id string Required
runProfileId run profile id string Required
Body
application/json
BigDataRunProfile

Response

204Status 204
No Content
400Status 400
Bad Request
Body
*/*
ErrorResponse
401Status 401
Unauthorized
Body
*/*
ErrorResponse
403Status 403
Forbidden
Body
*/*
ErrorResponse
404Status 404
Not Found
Body
*/*
ErrorResponse
500Status 500
Internal Server Error
Body
*/*
ErrorResponse