Execution History Search

Retrive execution history by filtering on the fields such as time period, workspaces while exluding plan execution tasks.

Media type

application/json

Endpoints

Global security

These security schemes apply to the entire API

Security scheme

This scheme can be referenced across the API

BearerAuthentication
This Bearer authentication can be set in the Authorization header of your requests. Authentication tokens and Personal Access Tokens are supported. Personal Access Tokens can be generated in the Profile Preferences page of the Talend Cloud Portal.
Bearer authentication
Name Description

Get execution records from history

POST /monitoring/observability/executions/search

Use this endpoint to filter historical executions by filtering various execution fields, including workspaceId, environmentId, name, destination, or tags.
This endpoing also supports partial match for faster searches.

In addition, you can use the exclude parameter to exclude plan executions or executions triggered in a plan.

Request

Body
application/json
SearchRequest

Response

200Status 200
Body
application/json
ExecutionPage
400Status 400
Body
application/json
Invalid parameter values
ErrorResponse
Status 400 application/json 

{
    "status": 400,
    "message": "Workspace ID is not valid id.",
    "requestId": "f1f6d388-b114-4d02-8d90-da2c8d1af60f"
}
401Status 401
Body
application/json
Unauthorized - (missing or invalid) authorization token
ErrorResponse
Status 401 application/json 

{
    "status": 401, 
    "message": "Unauthorized"
}
403Status 403
Body
application/json
Access denied (no permissions to access resource)
ErrorResponse
Status 403 application/json 

{
    "status": 403, 
    "message": "Forbidden",
    "requestId": "8c7c7de7-0a9a-45be-9e28-f0a34e6007fd"
}
500Status 500
Body
application/json
Internal server error
ErrorResponse
Status 500 application/json 

{
    "status": 500,
    "message": "Internal Server Error",
    "requestId": "2b6e98ae-66dd-4895-8f04-dc2f85617855"
}

FieldFilter

Object
Name Description Type Attributes and examples
field field names to be used as filtering criteria string Required
Enum
  • status
  • workspaceId
  • name
  • taskId
  • planId
  • artifactId
  • artifactVersion
  • tags
  • runtime ( including the following subfields: engineId, clusterId and destination )
  • period
  • executeDurationMs
  • operator
  • author
  • version (requires taskId filter to be used together with it)
  • logLevel (requires taskId filter to be used together with it)
  • id (Run ID)
  • runProfileId
operator

Filter operators. They are used to specify criteria for filtering data based on field values, allowing you to narrow down your searches.

  1. Operators match and not_match are used when a search must include or contain the value in the given filter, with the following limitations:

    • Limited to be used with the name name filter field only
    • The value cannot be less than 3 characters

  2. Operator in and not_in is for the exact match of a given character sequence (recommended to be used to filer on IDs)

  3. Operators or and and are used in the runtime field filter only:

    • Operator or is used when the sub-filters of the runtime filter uses the inner operator in.
    • Operator and is used when the sub-filters of the runtime filter uses inner operator not_in

  4. Operator between is used only with the period field filter

 string Required
Enum
  • in
  • not_in
  • match
  • not_match
  • or
  • and
  • between
  • greater_than
  • less_than
value
  1. Array of values for a given filter that is using operators in and not_in
  2. Single value for operators match and not_match
  3. Array of FieldFilter objects. These objects is used in the runtime field and only support operator or or and.
  4. When the value is used by a period filter, you need to give an array of two ISO-formatted dates:
  • first value: starting date of the period
  • second value: ending date of the period
 ObjectRequired
FieldFilter
{
    "field":"name",
    "operator": "match",
     "value": "sample_"
 }
{
    "field":"workspaceId",
    "operator": "in",
     "value": [
         "629620a561390f636b98884c", 
         "629620a561390f636b9888re", 
         "629620a561390f636bgsh538"
      ]
}
{
    "field":"tags",
    "operator": "in",
     "value": ["tag1", "tag2"]
}
{
    "field": "runtime",
    "operator": "or",
    "value": [
        {
            "field": "engineId",
            "operator": "in",
            "value": [
                "7c733c6e-986d-4060-b2a6-1f0edbe632c2"
            ]
        },
        {
            "field": "clusterId",
            "operator": "in",
            "value": [
                "f067538e-a19d-4241-a5d3-9c2ca72bb64e",
                "60b36d1a-7b88-42ac-b27e-33feca416c6a"
            ]
        },
        {
            "field": "destination",
            "operator": "in",
            "value": [
                "CLOUD",
                "CLOUD_SANDBOX"
            ]
        }
    ]
}
{
    "field": "runtime",
    "operator": "and",
    "value": [
        {
            "field": "engineId",
            "operator": "not_in",
            "value": [
                "7c733c6e-986d-4060-b2a6-1f0edbe632c2"
            ]
        },
        {
            "field": "destination",
            "operator": "not_in",
            "value": [
                "CLOUD",
                "CLOUD_SANDBOX"
            ]
        }
    ]
}
{
    "field": "period",
    "operator": "between",
    "value": [
        "2023-01-01T10:06:47.069Z",
        "2023-05-01T10:06:47.069Z"
    ]
}
{
    "field": "executeDurationMs",
    "operator": "greeter_than",
    "value": 10000 
}
{
    "field": "executeDurationMs",
    "operator": "less_than",
    "value": 10000 
}
{
  "field": "author",
  "operator": "in",
  "value": [
    "user1",
    "fA3FFnkkMmJAdDC5ybc6jalMY5imLmxgymwkBSQMztY"
  ]
}
{
  "field": "operator",
  "operator": "not_in",
  "value": [
    "user1",
    "fA3FFnkkMmJAdDC5ybc6jalMY5imLmxgymwkBSQMztY"
  ]
}
{
    "field": "version",
    "operator": "in",
    "value": ["1.1"]
}
{
    "field": "logLevel",
    "operator": "in",
    "value": ["ERROR"]
}
{
   "field": "id",
   "operator": "in",
   "value": [
      "82bb724f-9327-4e92-a02e-4d8cd2a8b114",
      "474db242-b7f0-4a94-b13e-721ec86bbaad"
   ]
}
{
    "field": "runProfileId",
    "operator": "in",
    "value": [
       "658bc937086a403a938b60b4"
    ]
}

SearchRequest

Object
Name Description Type Attributes and examples
environmentId the ID that identifies the environment where the execution targeted by the search took place string Required
Example
5da03d41ce4f7e14862192be
category

group of execution types

ETL includes types:

  • PLAN
  • STANDARD
  • PIPELINE
  • BIG_DATA_BATCH
  • BIG_DATA_STREAMING

ESB includes types:

  • ROUTE
  • DATA_SERVICE
 string Required
Enum
  • ETL
  • ESB
filters array of FieldFilter Optional
Datatype details
Type Description Attributes and examples
array Defined filters for execution query
FieldFilter
limit This is the number of records to be returned on a single page (Page size) number Optional
Default50
Minimum1Maximum200
offset Used for fetching next page in paginated response number Optional
exclude

Excluding certain execution types from the returned results

TASK_EXECUTIONS_TRIGGERED_BY_PLAN: all task executions triggered by plans are excluded, that is to say, the returned executions include all plans and all individual execution tasks of the ETL category that are not triggered by any plan.

PLAN_EXECUTIONS: plans are excluded, in other words, the returned executions include all execution tasks of category ETL, triggered or not triggered by plans.

 string Optional
DefaultTASK_EXECUTIONS_TRIGGERED_BY_PLAN
Enum
  • PLAN_EXECUTIONS
  • TASK_EXECUTIONS_TRIGGERED_BY_PLAN
SearchRequest
{
    "environmentId": "5da03d41ce4f7e14862192be",
    "category": "ETL",
    "filters": [
        {
            "field": "workspaceId",
            "operator": "in",
            "value": [
                "6269036efbc8e217c8a39e2f"
            ]
        },
        {
            "field": "status",
            "operator": "not_in",
            "value": [
                "STARTING_FLOW_EXECUTION",
                "DISPATCHING_FLOW",
                "EXECUTION_SUCCESS",
                "EXECUTION_REJECTED",
                "EXECUTION_FAILED",
                "EXECUTION_MISFIRED",
                "EXECUTION_TERMINATED"
            ]
        },
        {
            "field": "name",
            "operator": "match",
            "value": "sample_"
        },
        {
            "field": "runtime",
            "operator": "or",
            "value": [
                {
                    "field": "engineId",
                    "operator": "in",
                    "value": [
                        "7c733c6e-986d-4060-b2a6-1f0edbe632c2"
                    ]
                },
                {
                    "field": "clusterId",
                    "operator": "not_in",
                    "value": [
                        "f067538e-a19d-4241-a5d3-9c2ca72bb64e",
                        "60b36d1a-7b88-42ac-b27e-33feca416c6a"
                    ]
                },
                {
                    "field": "destination",
                    "operator": "in",
                    "value": [
                        "CLOUD",
                        "CLOUD_SANDBOX"
                    ]
                }
            ]
        },
        {
            "field": "executeDurationMs",
            "operator": "greater_than",
            "value": 3600000
         },
        {
            "field": "operator",
            "operator": "in",
            "value": [
              "john",
              "CKBh2zRgh82BvoqrtgMVG3VIkRoXhkWh"
            ]
        },
        {
            "field": "author",
            "operator": "not_in",
            "value": [
              "john"
            ]
        },
        {
            "field": "period",
            "operator": "between",
            "value": [
                "2023-01-01T10:06:47.069Z",
                "2023-05-01T10:06:47.069Z"
              ]
         }
    ],
    "limit": 50,
    "offset": 0,
    "exclude": "TASK_EXECUTIONS_TRIGGERED_BY_PLAN"
}

Execution

Object
Name Description Type Attributes and examples
id Execution ID string Required
PatternUUID
Example
caf50cbf-d6f8-47c6-a5e0-b31e0d11102a
name Task or plan name string Required
Examples
  • MyDailyTask
  • George's plan
status Execution status string Required
Enum
  • DEPLOY_FAILED
  • EXECUTION_SUCCESS
  • EXECUTION_FAILED
  • EXECUTION_TERMINATED
  • EXECUTION_TERMINATED_TIMEOUT
  • EXECUTION_REJECTED
  • EXECUTION_EVENT_RECEIVED
  • DISPATCHING_FLOW
  • STARTING_FLOW_EXECUTION
  • STOPPING_FLOW_EXECUTION
  • UNKNOWN
  • EXECUTION_MISFIRED
subStatus

Sub-status of the work flow

  • This sub-status, except EXECUTION_TERMINATED_TIMEOUT, is available only when the execution type is ROUTE or DATA_SERVICE
  • the subStatus EXECUTION_TERMINATED_TIMEOUT is also available also when the execution type is STANDARD
 string Optional
Enum
  • DOWNLOADING_ARTIFACT
  • STARTING_ARTIFACT
  • TASK_ACTIVE
  • TASK_INACTIVE
  • TASK_DETACHED
  • SENDING_REQUEST
  • UNDEPLOYING_TASK
  • EXECUTION_TERMINATED_TIMEOUT
type Execution type string Required
Enum
  • PLAN
  • STANDARD
  • PIPELINE
  • BIG_DATA_BATCH
  • BIG_DATA_STREAMING
  • ROUTE
  • DATA_SERVICE
version

Task version

  • available only for execution type!=PLAN, that is to say, the execution type is not a plan.
 string Required
Example
1.1
triggerType Trigger type string Required
Enum
  • MANUAL
  • SCHEDULED
  • WEBHOOK
  • PLAN
triggerTime

Trigger time of this execution in milliseconds.

  • useful only for execution type!=PLAN, meaning the execution type is not a plan.
  • when the execution type is a plan (type=PLAN), this triggerTime equals runStartTime
 number Optional
FormatINT32
Example
1663766366142
runStartTime Start time of the execution in milliseconds number Required
FormatINT32
Example
1663766366460
runEndTime Time when the execution finishes in milliseconds number Required
FormatINT32
Example
1663766368765
operator

The operator of a task or plan

  • Login name of user when authorType=HUMAN
  • ID of service account when authorType=SERVICE
 string Optional
Examples
  • john
  • CKBh2zRgh82BvoqrtgMVG3VIkRoXhkWh
operatorName

Actual name of the user or service as task or plan operator.

Warning! If the user/service account has been deleted, this field value will be returned with an empty string

 string Required
Defaultempty string
Examples
  • John Doe
  • My_Service_Name
operatorType Operator type string Optional
Enum
  • HUMAN
  • SERVICE
author

The author of a task or plan

  • Login name of user when authorType=HUMAN
  • ID of service account when authorType=SERVICE
 string Required
Examples
  • john
  • CKBh2zRgh82BvoqrtgMVG3VIkRoXhkWh
authorName

Actual name of the user or service that is the author

Warning! If the user/service account has been deleted, this field value will be returned with an empty string

 string Required
Defaultempty string
Examples
  • John Doe
  • My_Service_Name
authorType string Required
Enum
  • HUMAN
  • SERVICE
attempts

Scheduler may try to start task in parallel several times

  • available only for execution type!=PLAN
 number Optional
FormatINT32
Example
3
environmentId Environment ID string Required
Example
5da03d41ce4f7e14862192be
workspaceId Workspace ID string Required
Example
6269036efbc8e217c8a39e2f
taskId

Task ID

  • available only for execution type!=PLAN
 string Optional
Example
629620a561390f636b98884e
planExecutionId

Execution ID of the plan from which a task was triggered.

  • available only for execution type=STANDARD/PIPELINE/BIG_DATA_BATCH/BIG_DATA_STREAMING, and triggerType=PLAN
  • if user has no permission over the plan’s workspace, this field will not be included in the response
 string Optional
PatternUUID
Example
6e8fb590-6898-418f-9618-7c0228804e9b
planName

Name of the plan from which a task was triggered.

  • available only for execution type=STANDARD/PIPELINE/BIG_DATA_BATCH/BIG_DATA_STREAMING, and triggerType=PLAN
  • if user has no permission over the plan’s workspace, this field will be masked as ******
 string Optional
Examples
  • myPlan
  • ******
artifactId

Artifact ID

  • available only for execution type!=PLAN
 string Optional
Example
629620a561390f636b98884c
artifactName

Artifact name

  • available only for execution type!=PLAN
 string Optional
Examples
  • dataCollect
  • processFromAccountData
artifactVersion

Artifact version

  • available only for execution type!=PLAN
 string Optional
Examples
  • 1.0
  • 1.0.0
  • 0.1.0.20223105020524
artifactWorkspaceId

Artifact workspace ID

  • available only for execution type!=PLAN
 string Optional
Example
6269036efbc8e217c8a39e2f
destination

Where the job runs

  • available only for execution type!=PLAN
 string Optional
Enum
  • CLOUD
  • CLOUD_EXCLUSIVE
  • CLOUD_SANDBOX
  • REMOTE_ENGINE
  • REMOTE_ENGINE_CLUSTER
  • ELASTIC_ENGINE
clusterId

Cluster ID

  • available only for execution type!=PLAN
 string Optional
Example
62ea2c1d29a7271db937c266
clusterName

Cluster name

  • available only for execution type!=PLAN
 string Optional
Example
cluster1
engineId

Engine ID

  • available only for execution type!=PLAN
 string Optional
Example
62ea2c1d29a7271db937c265
engineName

Engine name

  • available only for execution type!=PLAN
 string Optional
Example
engine1
runProfileId

Run profile ID

  • available only for execution type!=PLAN
 string Optional
Example
62ea2e44dbfa6440ccff61c6
runProfileName

Run profile name

  • available only for execution type!=PLAN
 string Optional
Defaultprofile1
errorMessage

Message raised by an error during execution process.

  • it may not present if none produced
 string Optional
Example
This engine does not support Databricks 7.3 LTS run profiles. Change your target engine or adapt your run profile accordingly.
misfiredReason Misfired type string Optional
Enum
  • PARALLEL_EXECUTION_FORBIDDEN
  • UNABLE_TO_RETRIEVE_ARTIFACT_OR_TASK_MAVEN_URL
  • TASK_IS_PAUSED
  • DE_ENV_NOT_CONNECTED
  • REMOTE_ENGINE_UNAVAILABLE_IN_CLUSTER
  • REMOTE_ENGINE_UNAVAILABLE
  • DYNAMIC_ENGINE_UNAVAILABLE
  • REMOTE_ENGINE_CONTAINER_NOT_FOUND
  • MAX_DEPLOYMENT_ATTEMPTS_REACHED
  • MAX_PARALLEL_TASKS_LIMIT_REACHED
  • PIPELINE_RUN_PROFILE_NOT_FOUND
  • PIPELINE_ARTIFACT_NOT_FOUND
  • DATA_FABRIC_ARTIFACT_NOT_FOUND
  • ELASTIC_ENGINE_CONTAINER_NOT_FOUND
  • INCOMPATIBILITY_BETWEEN_ACTIONS_VERSIONS_AND_ENGINE
  • UNKNOWN
  • SIMULTANEOUS_EXECUTIONS_LIMIT_REACHED
  • PLAN_EXECUTIONS_PAUSED
  • PARALLEL_EXECUTIONS_FORBIDDEN
logLevel

Logging level set for the execution

  • available only for execution type!=PLAN
 string Optional
Enum
  • INFO
  • DEBUG
  • ERROR
  • WARN
tags array of string Optional
Datatype details
Type Description Attributes and examples
array

Tags

  • available only for execution type!=PLAN
Example
["tag1", "tag2"]
string
Examples
  • tag1
  • tag2
numberOfRejectedRows

Number of rows that have been rejected

  • available in executions of type!=PLAN and status=EXECUTION_REJECTED
 number Optional
Example
3
deploymentStrategy

Deployment strategy of artifacts

  • available in executions of type!=PLAN
 string Optional
Enum
  • PARALLEL
  • ROLLING
planId

Plan ID

  • available in executions of type=PLAN or triggerType=PLAN
  • if triggerType=PLAN, and user has no permission of the plan’s workspace, this field will not included in the response.
 string Optional
PatternUUID
Example
6e8fb590-6898-418f-9618-7c0228804e9f
failedStepName

Failed step name

  • available in executions of type=PLAN
 string Optional
Example
Step 3
rerunPlanExecutionId

Rerun plan execution ID

  • available in executions of type=PLAN
 string Optional
PatternUUID
Example
caf50cbf-d6f8-47c6-a5e0-b31e0d11103b
parentPlanExecutionId

Parent plan execution ID

  • available in executions of type=PLAN
 string Optional
PatternUUID
Example
caf50cbf-d6f8-47c6-a5e0-b31e0d11103a
executeDurationMs Execute duration with millisecond unit number Optional
Example
60000
engineVersion

Engine version

  • available only for execution type!=PLAN
 string Optional
Example
2.13.2
javaVersion

Java version

  • available only for execution type!=PLAN
 string Optional
Example
1.8.0_392
javaVendor

Java vendor

  • available only for execution type!=PLAN
 string Optional
Example
Red Hat, Inc.
jvmName

JVM name

  • available only for execution type!=PLAN
 string Optional
Example
OpenJDK 64-Bit Server VM
Execution
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestRejectedJob",
    "status": "EXECUTION_REJECTED",
    "type": "STANDARD",
    "version": "1.1",
    "triggerType": "PLAN",
    "triggerTime": 1663766366142,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
    "executeDurationMs": 2305,
    "operator": "georgi",
    "operatorName": "George Rock",
    "operatorType": "HUMAN",
    "author": "georgi",
    "authorName": "George Rock",
    "authorType": "HUMAN",
    "attempts": 3,
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "taskId": "629620a561390f636b98884e",
    "planExecutionId": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102b",
    "planName": "George's plan",
    "artifactId": "629620a561390f636b98884c",
    "artifactName": "TestRejectedJob",
    "artifactVersion": "0.1.0.20223105020524",
    "artifactWorkspaceId": "6269036efbc8e217c8a39e2f",
    "destination": "CLOUD",
    "clusterId": "62ea2c1d29a7271db937c266",
    "clusterName": "cluster1",
    "engineId": "62ea2c1d29a7271db937c265",
    "engineName": "engine1",
    "engineVersion": "2.13.2",
    "runProfileId": "62ea2e44dbfa6440ccff61c6",
    "runProfileName": "profile1",
    "logLevel": "INFO",
    "tags": [
        "tag1",
        "tag2"
    ],
    "numberOfRejectedRows": 3, 
    "deploymentStrategy" : "PARALLEL",
    "planId": "6e8fb590-6898-418f-9618-7c0228804e9f",
    "javaVersion": "1.8.0_392",
    "javaVendor": "Red Hat, Inc.",
    "jvmName": "OpenJDK 64-Bit Server VM"
}
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestPlan",
    "status": "EXECUTION_FAILED",
    "type": "PLAN",
    "triggerType": "SCHEDULED",
    "triggerTime": 1663766366460,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
    "executeDurationMs": 2305,
    "operator": "georgi",
    "operatorName": "George Rock",
    "operatorType": "HUMAN",
    "author": "georgi",
    "authorName": "George Rock",
    "authorType": "HUMAN",
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "errorMessage": "this is some error message",
    "planId": "6e8fb590-6898-418f-9618-7c0228804e9f",
    "failedStepName" : "Step 3",
    "rerunPlanExecutionId": "caf50cbf-d6f8-47c6-a5e0-b31e0d11103b",
    "parentPlanExecutionId": "caf50cbf-d6f8-47c6-a5e0-b31e0d11103a"
}
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestFailedJob",
    "status": "EXECUTION_FAILED",
    "type": "STANDARD",
    "version": "1.1",
    "triggerType": "PLAN",
    "triggerTime": 1663766366142,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
    "executeDurationMs": 2305,
    "operator": "4sI2AK6HHPKUGklLdNIM8dYZqEWQKD56",
    "operatorName": "SA-TEST",
    "operatorType": "SERVICE",
    "author": "4sI2AK6HHPKUGklLdNIM8dYZqEWQKD56",
    "authorName": "SA-TEST",
    "authorType": "SERVICE",
    "attempts": 3,
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "taskId": "629620a561390f636b98884e",
    "planExecutionId": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102b",
    "planName": "George's plan",
    "artifactId": "629620a561390f636b98884c",
    "artifactName": "TestFailedJob",
    "artifactVersion": "0.1.0.20223105020524",
    "artifactWorkspaceId": "6269036efbc8e217c8a39e2f",
    "destination": "CLOUD",
    "clusterId": "62ea2c1d29a7271db937c266",
    "clusterName": "cluster1",
    "engineId": "62ea2c1d29a7271db937c265",
    "engineName": "engine1",
    "runProfileId": "62ea2e44dbfa6440ccff61c6",
    "runProfileName": "profile1",
    "errorMessage": "some error message.",
    "logLevel": "INFO",
    "tags": [
        "tag1",
        "tag2"
    ],
    "planId": "6e8fb590-6898-418f-9618-7c0228804e9f"
}
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestPipelineJob",
    "status": "EXECUTION_SUCCESS",
    "type": "PIPELINE",
    "version": "1.1",
    "triggerType": "MANUAL",
    "triggerTime": 1663766366142,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
    "executeDurationMs": 2305,
    "operator": "john",
    "operatorName": "John Doe",
    "operatorType": "HUMAN",
    "author": "john",
    "authorName": "John Doe",
    "authorType": "HUMAN",
    "attempts": 3,
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "taskId": "629620a561390f636b98884e",
    "artifactId": "629620a561390f636b98884c",
    "artifactName": "TestPipelineJob",
    "artifactVersion": "0.1.0.20223105020524",
    "artifactWorkspaceId": "6269036efbc8e217c8a39e2f",
    "destination": "ELASTIC_ENGINE",
    "clusterId": "62ea2c1d29a7271db937c266",
    "clusterName": "cluster1",
    "engineId": "62ea2c1d29a7271db937c265",
    "engineName": "engine1",
    "runProfileId": "62ea2e44dbfa6440ccff61c6",
    "runProfileName": "profile1",
    "logLevel": "INFO",
    "tags": [
        "tagA",
        "tagB"
    ]
}
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestBigDataBatchJob",
    "status": "EXECUTION_FAILED",
    "type": "BIG_DATA_BATCH",
    "version": "1.1",
    "triggerType": "MANUAL",
    "triggerTime": 1663766366142,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
    "executeDurationMs": 2305,
    "operator": "pwlin",
    "operatorName": "Pinwen Strong",
    "operatorType": "HUMAN",
    "author": "pwlin",
    "authorName": "Pingwen Strong",
    "authorType": "HUMAN",
    "attempts": 3,
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "taskId": "629620a561390f636b98884e",
    "artifactId": "629620a561390f636b98884c",
    "artifactName": "TestBigDataBatchJob",
    "artifactVersion": "0.1.0.20223105020524",
    "artifactWorkspaceId": "6269036efbc8e217c8a39e2f",
    "destination": "CLOUD",
    "clusterId": "62ea2c1d29a7271db937c266",
    "clusterName": "cluster1",
    "engineId": "62ea2c1d29a7271db937c265",
    "engineName": "engine1",
    "runProfileId": "62ea2e44dbfa6440ccff61c6",
    "runProfileName": "profile1",
    "errorMessage": "some error message.",
    "logLevel": "INFO",
    "tags": [
        "tagA",
        "tabB"
    ]
}
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestBigDataStreamingJob",
    "status": "EXECUTION_FAILED",
    "type": "BIG_DATA_STREAMING",
    "version": "1.1",
    "triggerType": "MANUAL",
    "triggerTime": 1663766366142,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
   "executeDurationMs": 2305,
    "operator": "lucy",
    "operatorName": "Lucy Brown",
    "operatorType": "HUMAN",
    "author": "lucy",
    "authorName": "Lucy Brown",
    "authorType": "HUMAN",
    "attempts": 3,
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "taskId": "629620a561390f636b98884e",
    "artifactId": "629620a561390f636b98884c",
    "artifactName": "TestBigDataStreamingJob",
    "artifactVersion": "0.1.0.20223105020524",
    "artifactWorkspaceId": "6269036efbc8e217c8a39e2f",
    "destination": "CLOUD",
    "clusterId": "62ea2c1d29a7271db937c266",
    "clusterName": "cluster1",
    "engineId": "62ea2c1d29a7271db937c265",
    "engineName": "engine1",
    "runProfileId": "62ea2e44dbfa6440ccff61c6",
    "runProfileName": "profile1",
    "errorMessage": "some error message.",
    "logLevel": "WARN",
    "tags": [
        "tag1",
        "tab2"
    ]
}
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestRouteJob",
    "status": "STARTING_FLOW_EXECUTION",
    "subStatus": "TASK_ACTIVE",
    "type": "ROUTE",
    "version": "1.1",
    "triggerType": "MANUAL",
    "triggerTime": 1663766366142,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
    "executeDurationMs": 2305,
    "operator": "georgi",
    "operatorName": "George Rock",
    "operatorType": "HUMAN",
    "author": "georgi",
    "authorName": "George Rock",
    "authorType": "HUMAN",
    "attempts": 3,
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "taskId": "629620a561390f636b98884e",
    "artifactId": "629620a561390f636b98884c",
    "artifactName": "TestRouteJob",
    "artifactVersion": "0.1.0.20223105020524",
    "artifactWorkspaceId": "6269036efbc8e217c8a39e2f",
    "destination": "REMOTE_ENGINE",
    "clusterId": "62ea2c1d29a7271db937c266",
    "clusterName": "cluster1",
    "engineId": "62ea2c1d29a7271db937c265",
    "engineName": "engine1",
    "runProfileId": "62ea2e44dbfa6440ccff61c6",
    "runProfileName": "profile1",
    "logLevel": "WARN",
    "deploymentStrategy" : "PARALLEL"
}
{
    "id": "caf50cbf-d6f8-47c6-a5e0-b31e0d11102a",
    "name": "TestDataServiceJob",
    "status": "STARTING_FLOW_EXECUTION",
    "subStatus": "TASK_ACTIVE",
    "type": "DATA_SERVICE",
    "version": "1.1",
    "triggerType": "MANUAL",
    "triggerTime": 1663766366142,
    "runStartTime": 1663766366460,
    "runEndTime": 1663766368765,
    "executeDurationMs": 2305,
    "operator": "peter",
    "operatorName": "Peter Chang",
    "operatorType": "HUMAN",
    "author": "peter",
    "authtorName": "Peter Chang",
    "authorType": "HUMAN",
    "attempts": 3,
    "environmentId": "5da03d41ce4f7e14862192be",
    "workspaceId": "6269036efbc8e217c8a39e2f",
    "taskId": "629620a561390f636b98884e",
    "artifactId": "629620a561390f636b98884c",
    "artifactName": "TestDataServiceJob",
    "artifactVersion": "0.1.0.20223105020524",
    "artifactWorkspaceId": "6269036efbc8e217c8a39e2f",
    "destination": "REMOTE_ENGINE",
    "clusterId": "62ea2c1d29a7271db937c266",
    "clusterName": "cluster1",
    "engineId": "62ea2c1d29a7271db937c265",
    "engineName": "engine1",
    "runProfileId": "62ea2e44dbfa6440ccff61c6",
    "runProfileName": "profile1",
    "logLevel": "INFO",
    "deploymentStrategy" : "PARALLEL"
}

ExecutionPage

Object
Name Description Type Attributes and examples
items array of Execution Optional
Datatype details
Type Description Attributes and examples
array

NOTE: Records are sorted by runEndTime in desc order and then by triggerTime in asc order.

Sorting by trigger time allows you to get the oldest runs on top.
Execution
limit Maximum number of records to be shown on each page number Optional
Default50
Minimum5Maximum200
offset Pagination pointer index, from which to start returning “n” number of records number Optional
nextPageExists Boolean flag indicating if there is a next page boolean Required
ExecutionPage
{
   "items":
   [
       {
           "id": "cce4d0c5-866e-45c2-854d-f77483263394",
           "name": "PipelinePlan",
           "status": "STARTED",
           "type": "PLAN",
           "triggerType": "SCHEDULE",
           "triggerTime": 1663682400069,
           "runStartTime": 1663682447069,
           "executeDurationMs": 253000,
           "operator": "CKBh2zRgh82BvoqrtgMVG3VIkRoXhkWh",
           "operatorName": "SA-TEST",
           "operatorType": "SERVICE",
           "author": "CKBh2zRgh82BvoqrtgMVG3VIkRoXhkWh",
           "authorName": "SA-TEST",
           "authorType": "SERVICE",
           "accountId": "02c47147-0aa4-47a9-af91-0a6d2ab71bb1",
           "environmentId": "5da03d41ce4f7e14862192be",
           "workspaceId": "6093f6a36f01d639680d31cd",
           "planId": "3d0509b8-2d98-4acd-ab1e-d9180e77638a"
       },
       {
           "id": "eb6305d0-de97-4037-bb69-2f04b03655c0",
           "name": "MyPlanHasPermission",
           "status": "EXECUTION_FAILED",
           "type": "PLAN",
           "triggerType": "SCHEDULE",
           "triggerTime": 1663768800069,
           "runStartTime": 1663768847069,
           "runEndTime": 1663769100069,
           "executeDurationMs": 253000,
           "operator": "john",
           "operatorName": "John Doe",
           "operatorType": "HUMAN",
           "author": "john",
           "authorName": "John Doe",
           "authorType": "HUMAN",
           "accountId": "02c47147-0aa4-47a9-af91-0a6d2ab71bb1",
           "environmentId": "5da03d41ce4f7e14862192be",
           "workspaceId": "6093f6a36f01d639680d31cd",
           "errorMessage": "An error has occured during execution.",
           "planId": "0cacc5cd-60c0-4c2b-a940-2e01d0e72c0b",
           "failedStepName": "Step 3",
           "rerunPlanExecutionId": "eb6305d0-de97-4037-bb69-2f04b03655d0"
       },
       {
           "id": "958ccb44-0a3f-498e-8c6f-6130725a259b",
           "name": "sample_anagrammatize",
           "status": "EXECUTION_SUCCESS",
           "type": "STANDARD",
           "version": "1.0",
           "triggerType": "WEBHOOK",
           "triggerTime": 1663668407069,
           "runStartTime": 1663668407069,
           "runEndTime": 1663668407069,
           "executeDurationMs": 253000,
           "operator": "4sI2AK6HHPKUGklLdNIM8dYZqEWQKD56",
           "operatorName": "MY_SERVICE",
           "operatorType": "SERVICE",
           "author": "4sI2AK6HHPKUGklLdNIM8dYZqEWQKD56",
           "authorName": "MY_SERVICE",
           "authorType": "SERVICE",
           "attempts": 2,
           "accountId": "02c47147-0aa4-47a9-af91-0a6d2ab71bb1",
           "environmentId": "5da03d41ce4f7e14862192be",
           "workspaceId": "6093f6a36f01d639680d31cd",
           "taskId": "SIs6gwOENbw6KqVH2pZlr5xd",
           "artifactId": "FVL4vVqSVApbuJXGdRjPLHiT",
           "artifactName": "sample_anagrammatize",
           "artifactVersion": "sVKv7QNYhW1id25tX5CR",
           "artifactWorkspaceId": "sVKv7QNYhW1id25tX5CR",
           "destination": "CLOUD_SANDBOX",
           "engineId": "7c733c6e-986d-4060-b2a6-1f0edbe632c2",
           "engineName": "tOF5C410eHfQ5aO74vPE",
           "errorMessage": "Unexpected exception",
           "logLevel": "WARN",
           "tags":
           [
               "nailing",
               "pisgah",
               "anaqua",
               "graphemes",
               "pubble",
               "floscular"
           ],
           "deploymentStrategy": "ROLLING"
       },
       {
           "id": "958ccb44-0a3f-498e-8c6f-6130725a259c",
           "name": "TaskWithNoTags",
           "status": "EXECUTION_SUCCESS",
           "type": "STANDARD",
           "version": "1.0",
           "triggerType": "MANUAL",
           "triggerTime": 1663668407069,
           "runStartTime": 1663668407069,
           "runEndTime": 1663668407069,
           "executeDurationMs": 253000,
           "operator": "4sI2AK6HHPKUGklLdNIM8dYZqEWQKD56",
           "operatorName": "MY_SERVICE",
           "operatorType": "SERVICE",
           "author": "4sI2AK6HHPKUGklLdNIM8dYZqEWQKD56",
           "authorName": "MY_SERVICE",
           "authorType": "SERVICE",
           "attempts": 2,
           "accountId": "02c47147-0aa4-47a9-af91-0a6d2ab71bb1",
           "environmentId": "5da03d41ce4f7e14862192be",
           "workspaceId": "6093f6a36f01d639680d31cd",
           "taskId": "SIs6gwOENbw6KqVH2pZlr5xd",
           "artifactId": "FVL4vVqSVApbuJXGdRjPLHiT",
           "artifactName": "TaskWithNoTags",
           "artifactVersion": "sVKv7QNYhW1id25tX5CR",
           "artifactWorkspaceId": "sVKv7QNYhW1id25tX5CR",
           "destination": "CLOUD_SANDBOX",
           "engineId": "7c733c6e-986d-4060-b2a6-1f0edbe632c2",
           "engineName": "tOF5C410eHfQ5aO74vPE",
           "errorMessage": "Unexpected exception",
           "logLevel": "WARN",
           "deploymentStrategy": "ROLLING"
       },
       {
           "id": "d30b2ec0-b7ac-44be-aae2-8e30cbc95b89",
           "name": "standaloneTask",
           "status": "STARTING_FLOW_EXECUTION",
           "subStatus": "TASK_ACTIVE",
           "type": "STANDARD",
           "version": "1.1",
           "triggerType": "WEBHOOK",
           "triggerTime": 1663668000069,
           "runStartTime": 1663668047069,
           "runEndTime": 1663668300069,
           "executeDurationMs": 253000,
           "operator": "john",
           "operatorName": "John Doe",
           "operatorType": "HUMAN",
           "author": "john",
           "operatorName": "John Doe",
           "authorType": "HUMAN",
           "attempts": 3,
           "accountId": "02c47147-0aa4-47a9-af91-0a6d2ab71bb1",
           "environmentId": "5da03d41ce4f7e14862192be",
           "workspaceId": "6093f6a36f01d639680d31cd",
           "taskId": "SIs6gwOENbw6KqVH2pZlr5xd",
           "artifactId": "629620a561390f636b98884c",
           "artifactName": "standaloneTask",
           "artifactVersion": "0.1.0.20223105020524",
           "artifactWorkspaceId": "sVKv7QNYhW1id25tX5CR",
           "destination": "CLOUD_SANDBOX",
           "clusterId": "05b9e490-5cb5-473b-838d-370e052a264b",
           "clusterName": "cluster7803",
           "engineId": "33e1ab19-c7a6-4ae1-b935-b803dec480fe",
           "engineName": "engine27",
           "runProfileId": "63a2db6eee8065c0f77bf641",
           "runProfileName": "myProfile",
           "logLevel": "WARN",
           "tags":
           [
               "tag1",
               "tag2"
           ],
           "deploymentStrategy": "PARALLEL"
       }
   ],
   "limit": 50,
   "offset": 0,
   "nextPageExists": false
}

ErrorResponse

Object
Name Description Type Attributes and examples
status number Required
FormatINT32
Examples
  • 400
  • 401
  • 403
  • 500
message string Required
Example
Parameter 'invalid_field' is not valid field.
requestId string Optional
PatternUUID
Example
026367a8-8967-47f8-aa18-bc379a8ec528
ErrorResponse
{
    "status": 400,
    "message": "Workspace ID is not valid id.",
    "requestId": "f1f6d388-b114-4d02-8d90-da2c8d1af60f"
}

Download execution records from history

POST /monitoring/observability/executions/download

Use this endpoint to download historical executions by filtering various execution fields, includingincluding workspaceId, environmentId, name, destination, or tags.

This endpoing also supports partial match for faster searches.

In addition, you can use the exclude parameter to exclude plan executions or executions triggered in a plan. Partial match is also supported for a quicker search performance.

This endpoint makes use of a keyset pagination.

  • limit is used to set the page size.
  • lastId contains the last element of the previous page. You can get it from the downloaded CSV file.

The elements are sorted according to the following criteria

  • runEndTime in desc order
  • triggerTime in asc order
  • id in desc

Request

Body
application/json
DownloadRequest
application/json 

{
    "environmentId": "5da03d41ce4f7e14862192be",
    "category" : "ETL",
    "filters": [
     {
         "field":"workspaceId",
         "operator": "in",
         "value": ["6269036efbc8e217c8a39e2f"]
     },
      {
         "field":"status",
         "operator": "in",
         "value": [
           "STARTING_FLOW_EXECUTION",
           "DISPATCHING_FLOW", 
           "EXECUTION_SUCCESS", 
           "EXECUTION_REJECTED",
           "EXECUTION_FAILED", 
           "EXECUTION_TERMINATED"]
      },
      {
         "field":"name",
         "operator": "match",
         "value": "sample_"
       }
    ],
    "exclude": "TASK_EXECUTIONS_TRIGGERED_BY_PLAN",
    "triggerTimeFrom": "2022-09-19T10:06:47.069Z",
    "triggerTimeTo": "2022-09-22T10:06:47.069Z",
    "lastId": "0ccc5546-708d-4fb7-a524-6c47e8867d0b",
    "limit": 1000
}

Response

200Status 200
Body
application/octet-stream

This API returns a CSV file which complies with the following rules

  • The delimiter is a comma
  • The first line of the file is a header. It contains the following column names : id, name, status, subStatus, type, version, triggerType, triggerTime, runStartTime, runEndTime, operator, operatorType, author, authorType, attempts, accountId, environmentId, workspaceId, taskId, planExecutionId, planName, artifactId, artifactName, artifactVersion, artifactWorkspaceId, destination, clusterId, clusterName, engineId, engineName, runProfileId, runProfileName, errorMessage, misfiredReason, logLevel, tags, numberOfRejectedRows, deploymentStrategy, planId, failedStepName,rerunPlanExecutionId, parentPlanExecutionId, executeDurationMs, engineVersion, javaVersion, javaVendor, jvmName
  • The double quotation marks are used as quoting character
  • When data contains double quotations, is it escaped by doubling the double quotations;
  • Encoding : UTF8 is used

The following table describes the format of each column

Column name Column type Comment
id UUID Execution ID of plan/task
name String name of plan/task
status Enum execution status of plan/task
subStatus Enum available only for execution type=ROUTE/DATA_SERVICE; the subStatus EXECUTION_TERMINATED_TIMEOUT available also in type=STANDARD
type Enum execution type
version String available only for execution type!=PLAN
triggerType String trigger type of the execution
triggerTime timestamp with time zone trigger time of the execution
runStartTime timestamp with time zone run start time of the execution
runEndTime timestamp with time zone run end time of the execution
executeDurationMs Long duration of the execution
operator String operator of the execution; login of user or ID of service account
operatorType String HUMAN or SERVICE
author String author of the plan/task; login of user or ID of service account
authorType String
attempts int attempt number of running the task; only for execution type!=PLAN
accountId UUID
environmentId String environment ID of the plan/task
workspaceId String workspace ID of the plan/task
taskId String ID of the task; only for execution type!=PLAN
planExecutionId UUID available only for execution type=STANDARD/PIPELINE/BIG_DATA_BATCH/BIG_DATA_STREAMING, and triggerType=PLAN
planName String available only for execution type=STANDARD/PIPELINE/BIG_DATA_BATCH/BIG_DATA_STREAMING, and triggerType=PLAN
artifactId String available only for execution type!=PLAN
artifactName String available only for execution type!=PLAN
artifactVersion String available only for execution type!=PLAN
artifactWorkspaceId String available only for execution type!=PLAN
destination Enum CLOUD OR CLOUD_EXCLUSIVE OR CLOUD_SANDBOX OR REMOTE_ENGINE OR REMOTE_ENGINE_CLUSTER OR ELASTIC_ENGINE; available only for execution type!=PLAN
clusterId String available only for execution type!=PLAN
clusterName String available only for execution type!=PLAN
engineId String available only for execution type!=PLAN
engineName available only for execution type!=PLAN
runProfileId String available only for execution type!=PLAN
runProfileName String available only for execution type!=PLAN
errorMessage String available only for execution type!=PLAN
misfiredReason String misfired type
logLevel String INFO, DEBUG, ERROR, WARN; available only for execution type!=PLAN
tags String available only for execution type!=PLAN
numberOfRejectedRows int available only for execution type!=PLAN
deploymentStrategy Enum PARALLEL OR ROLLING; available only for type!=PLAN
planId UUID available only for execution of type=PLAN OR triggerType=PLAN
failedStepName String available only for execution of type=PLAN
rerunPlanExecutionId String available only for execution of type=PLAN
parentPlanExecutionId String available only for execution of type=PLAN
executeDurationMs Number execute duration with millisecond unit
engineVersion String available only for execution type!=PLAN
javaVersion String available only for execution type!=PLAN
javaVendor String available only for execution type!=PLAN
jvmName String available only for execution type!=PLAN
file
400Status 400
Body
application/json
Invalid parameter values
ErrorResponse
Status 400 application/json 

{
    "status": 400,
    "message": "Workspace ID is not valid id.",
    "requestId": "f1f6d388-b114-4d02-8d90-da2c8d1af60f"
}
401Status 401
Body
application/json
Unauthorized - (missing or invalid) authorization token
ErrorResponse
Status 401 application/json 

{
    "status": 401, 
    "message": "Unauthorized"
}
403Status 403
Body
application/json
Access denied (no permissions to access resource)
ErrorResponse
Status 403 application/json 

{
    "status": 403, 
    "message": "Forbidden",
    "requestId": "8c7c7de7-0a9a-45be-9e28-f0a34e6007fd"
}
500Status 500
Body
application/json
Internal server error
ErrorResponse
Status 500 application/json 

{
    "status": 500,
    "message": "Internal Server Error",
    "requestId": "2b6e98ae-66dd-4895-8f04-dc2f85617855"
}

DownloadRequest

Object
Name Description Type Attributes and examples
environmentId the ID that identifies the environment in which the execution targeted by the search took place string Required
Example
5da03d41ce4f7e14862192be
category

group of execution types

ETL includes types:

  • PLAN
  • STANDARD
  • PIPELINE
  • BIG_DATA_BATCH
  • BIG_DATA_STREAMING

ESB includes types:

  • ROUTE
  • DATA_SERVICE
 string Required
Enum
  • ETL
  • ESB
filters array of FieldFilter Optional
Datatype details
Type Description Attributes and examples
array Defined filters for record query
FieldFilter
exclude

Excluding certain execution types from the returned results

TASK_EXECUTIONS_TRIGGERED_BY_PLAN: all task executions triggered by plans are excluded, that is to say, the returned executions include all plans and all individual execution tasks of the ETL category that are not triggered by any plan.

PLAN_EXECUTIONS: plans are excluded, in other words, the returned executions include all execution tasks of category ETL, triggered or not triggered by plans.

 string Optional
DefaultTASK_EXECUTIONS_TRIGGERED_BY_PLAN
Enum
  • PLAN_EXECUTIONS
  • TASK_EXECUTIONS_TRIGGERED_BY_PLAN
limit This is the number of records to be returned on a single page (Page size) number Optional
Default1000
Minimum1Maximum100000
lastId the last execution id of the previous page string Optional
Example
0ccc5546-708d-4fb7-a524-6c47e8867d0b
DownloadRequest

{
    "environmentId": "5da03d41ce4f7e14862192be",
    "category" : "ETL",
    "filters": [
     {
         "field":"workspaceId",
         "operator": "in",
         "value": ["6269036efbc8e217c8a39e2f"]
     },
      {
         "field":"status",
         "operator": "in",
         "value": [
           "STARTING_FLOW_EXECUTION",
           "DISPATCHING_FLOW", 
           "EXECUTION_SUCCESS", 
           "EXECUTION_REJECTED",
           "EXECUTION_FAILED", 
           "EXECUTION_MISFIRED",
           "EXECUTION_TERMINATED"]
      },
      {
         "field":"name",
         "operator": "match",
         "value": "sample_"
       }
    ],
    "exclude": "TASK_EXECUTIONS_TRIGGERED_BY_PLAN",
    "lastId": "0ccc5546-708d-4fb7-a524-6c47e8867d0b",
    "limit": 1000
}
back to top↑