...
| Template Name | Extension Name | Extension Version |
|---|---|---|
| AWS Glue | ue-aws-glue | 1.2.01 |
Refer to Changelog for version history information.
...
- Universal Agent for Windows x64 Version 7.02.0.0 and later with python options installed.
- Universal Agent for Linux Version 7.02.0.0 and later with python options installed.
...
| Field | Input type | Default value | Type | Description | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Action | Required | Start Job Run | Choice | The action performed upon the task execution. The available actions are as follows.
| ||||||
| AWS Region Optional since version 1.1.0 | Optional | - | Text | Region for the Amazon Web Service. Find more information about the AWS Service endpoints and quotas here. When AWS Region is not populated as part of the task definition, during task execution the integration will look for credentials on the task execution environment. Refer to configuration options for more information. | ||||||
| AWS Credentials Optional since version 1.1.0 | Optional | - | Credentials | The Credentials definition should be as follows.
| ||||||
| Role Based Access | Optional | False | Boolean | Special type of authorization is provided by Role Assumption where the client sends his own credentials and the role he wants to assume from another user. If allowed, the client receives temporary credentials with limited time access to some resources. | ||||||
| Role ARN | Optional | - | Text | Role Arn: Amazon Role, which is applied for the connection. Role ARN format: Example RoleArn: arn:aws:iam::119322085622:role.Required when Role Based Access="True". | ||||||
| Job Name | Required | - | Text | Name of the Glue job that will be invoked. | ||||||
| Job Run ID | Optional | - | Text | ID of a previous Job Run to retry. | ||||||
| Security Configuration | Optional | - | Text | Name of the Security Configuration structure to be used with the Job Run. | ||||||
| Worker Type | Optional | None | Choice | Type of predefined worker that is allocated when a job runs. Available options are the following.
| ||||||
| Number Of Workers | Optional | - | Integer | Number of workers of a defined Worker Type that are allocated when a job is executed. The maximum number of workers that can be defined are as follows.
| ||||||
| Job Timeout | Optional | 2880 | Integer | Job Run timeout in minutes.
| ||||||
| Notify Delay Period | Optional | - | Integer | After a job run starts, the number of minutes to wait before sending a job run delay notification. | ||||||
Introduced in version 1.2.0 | Required | Array Field | Choice | Source of job arguments with possible choices: “Array Field” or “Script”. Job arguments replace the default arguments set in the job definition, for the current run. More info here. | ||||||
| Input Arguments Script Introduced in version 1.2.0 | Optional | - | Script | Job arguments in UAC Script in JSON format. Used to pass arguments from UAC environment variables or UAC Functions. Data Type of arguments must be string and character escaping actions to be performed where needed. Check the example for more information. Visible when Input Arguments Source is configured as "Script". | ||||||
| Input Arguments | Optional | - | Array | Job arguments in array format. Visible when Input Arguments Source is configured as "Array Field". | ||||||
| Wait for Success or Failure Introduced in version 1.2.0 | Optional | False | Boolean | If selected, the task will continue running until Job reaches the "SUCCEDED" or "FAILED" state."STOPPED", "TIMEOUT","ERROR' are considered "FAILED" states. | ||||||
| Polling Interval Introduced in version 1.2.0 | Optional | 60 | Integer | The polling interval in seconds between checking for the Job status. Required when Wait for Success or Failure ="True". | ||||||
| Proxy Type | Optional | HTTP | Choice | Type of proxy connection to be used. Available options are the following.
| ||||||
| Proxy | Optional | - | Text | Comma separated list of Proxy servers. Valid formats are the following.
| ||||||
| Proxy CA Bundle File | Optional | - | Text | The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy. Used when Proxy Type is configured for "HTTPS" or "HTTPS With Credentials". | ||||||
| Proxy Credentials | Optional | - | Credentials | Credentials to be used for the proxy communication. The credential definition should be as follows.
Required when "Proxy Type" is configured for "HTTPS With Credentials". |
...
{
"exit_code": 0,
"status_description": "SUCCESS: AWS Glue Job started successfully.",
"changed": true,
"invocation": {
"extension": "ue-aws-glue",
"version": "1.2.01",
"fields": {
"action": "Start Job Run",
"aws_credentials_user": "****",
"aws_credentials_password": "****",
"region": "us-east-1",
"role_based_access": false,
"role_arn": null,
"job_name": "AWS_Glue_pythonJob",
"job_run_id": null,
"security_configuration": null,
"worker_type": null,
"num_workers": null,
"job_timeout": 2880,
"notify_delay_period": null,
"use_proxy": false,
"proxy": null,
"proxy_type": null,
"proxy_ca_bundle_file": null,
"proxy_credentials_user": null,
"proxy_credentials_password": null,
"wait_for_success_or_failure": false,
"polling_interval": 60,
"input_arguments_source": "array_field",
"input_arguments_script": null,
"input_arguments": [
{
"--sleep": "0"
},
{
"--JOB_NAME": "Glue_Job"
}
]
}
},
"result": {
"out_job_run_id": "jr_123456789"
}
}...
{
"exit_code": 0,
"status_description": "SUCCESS: AWS Glue Job started successfully and resulted in status SUCCEEDED.",
"changed": true,
"invocation": {
"extension": "ue-aws-glue",
"version": "1.2.01",
"fields": {
"action": "Start Job Run",
"aws_credentials_user": "****",
"aws_credentials_password": "****",
"region": "us-east-1",
"role_based_access": false,
"role_arn": null,
"job_name": "AWS_Glue_pythonJob",
"job_run_id": null,
"security_configuration": null,
"worker_type": null,
"num_workers": null,
"job_timeout": 2880,
"notify_delay_period": null,
"use_proxy": false,
"proxy": null,
"proxy_type": null,
"proxy_ca_bundle_file": null,
"proxy_credentials_user": null,
"proxy_credentials_password": null,
"wait_for_success_or_failure": true,
"polling_interval": 3,
"input_arguments_source": "script_field",
"input_arguments_script": {
"--JOB_NAME": "Glue_Job"
},
"input_arguments": []
}
},
"result": {
"job_run_id": "jr_123456789",
"job_run_status": "SUCCEEDED",
"started_on": "2022-09-13 11:06:34.360000+03:00",
"last_modified_on": "2022-09-13 11:07:41.514000+03:00",
"completed_on": "2022-09-13 11:07:41.514000+03:00",
"error_message": null
}
}
...
| Document Link | Description |
|---|---|
| Universal Templates | User documentation for creating, working with and understanding Universal Templates and Integrations. |
| Universal Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. |
| Credentials | User documentation for creating and working with credentials. |
| Resolvable Credentials Permitted Property | User documentation for Resolvable Credentials Permitted Property. |
| Anchor | ||||
|---|---|---|---|---|
|
Changelog
ue-aws-glue-1.2.1 (2023-12-21)
Enhancements
Fixed: auto-renew AWS temporary credentials before expiration when using ARN based accessFixed: fixed issue with polling logic where task would get stuck with status Running (#35135)
Enhancements
Added: Support Start Glue Job and Wait until Job Reaches status "Succeeded" or "Failed" (#30157)Added: Larger set of output fields (#30157)Added: Log payload response for Job Run Status and Start Glue Job Run Action on debug mode.Added: Option to pass Input Arguments as UAC script supporting UAC environment variables and UAC Functions.
ue-aws-glue-1.2.0 (2022-11-11)
Enhancements
Added: Support Start Glue Job and Wait until Job Reaches status "Succeeded" or "Failed" (#30157)Added: Larger set of output fields (#30157)Added: Log payload response for Job Run Status and Start Glue Job Run Action on debug mode.Added: Option to pass Input Arguments as UAC script supporting UAC environment variables and UAC Functions.
ue-aws-glue-1.1.0 (2022-06-23)
Enhancements
Added: Allow AWS Credentials and AWS Region as optional fields enabling their configuration on the task execution environment. (#28312)
...