...
This Universal Extension provides tha the capability to submit a new AWS Glue Job.
...
Software Requirements for Universal Template and Universal Task
Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.
...
Network and Connectivity Requirements
Extension's Univesal Universal Agent host should be able to reach AWS Glue REST endpoints. The AWS Credentials provided in the AWS Glue Universal Task, should have sufficient permissions on AWS to invoke Glue Jobs.
...
This Universal Task requires the Resolvable Credentials feature, check that the Resolvable Credentials Permitted system property has been set to true. For more information about Resolvable Credentials click here.
Download the provided ZIP file.
In the Universal Controller UI, select Administration >Configuration > Universal Templates to display the current list of Universal Templates.
Click Import Template.
Select the template ZIP file and Import.
...
The input fields for this Universal Extension are described belowin the following table.
| Field | Input type | Default value | Type | Description | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Action | Required | Start Job Run | Choice | The action Action performed upon the task execution. Available action:
| ||||||||
| AWS Region | Required | - | Text | Region for the Amazon Web Service. Find more information about the AWS Service endpoints and quotas here. | ||||||||
| AWS Credentials | Required | - | Credentials | The AWS account credentials. They are comprised of:
| ||||||||
| Role Based Access | Optional | False | Boolean | Special type of authorization is provided by Role Assumption where the client sends his own credentials and the role he wants to assume from another user. If allowed, the client receives temporary credentials with limited time access to some resources. | ||||||||
| Role ARN | Optional | - | Text | Role Arn: Amazon Role, which is applied for the connection. Role ARN format: Required when Role Based Access="True". | ||||||||
| Job Name | Required | - | Text | The name Name of the Glue job that will be invoked. | ||||||||
| Job Run ID | Optional | - | Text | The ID of a previous Job Run to retry. | ||||||||
| Security Configuration | Optional | - | Text | The name Name of the Security Configuration structure to be used with the Job Run. | ||||||||
| Worker Type | Optional | None | Choice | The type Type of predefined worker that is allocated when a job runs. Available options are:
| ||||||||
| Number Of Workers | Optional | - | Integer | The number Number of workers of a defined Worker Type that are allocated when a job is executed. The maximum number of workers that can be defined are:
Required when Worker Type is not None. | ||||||||
| Job Timeout | Optional | 2880 | Integer | The Job Run timeout in minutes.
| ||||||||
| Notify Delay Period | Optional | - | Integer | After a job run starts, the number of minutes to wait before sending a job run delay notification. | ||||||||
| Input Arguments | Optional | - | Array | The job Job arguments specifically for this run. For this Job Run, they replace the default arguments set in the job definition itself. | ||||||||
| Proxy Type | Optional | HTTP | Choice | Type of proxy connection to be used. Available options are:
Visible only when Use Proxy = "True". | ||||||||
| Proxy | Optional | - | Text | Comma-separated list of Proxy servers. Valid formats:
Required when Use Proxy is checked. | ||||||||
| Proxy CA Bundle File | Optional | - | Text | The path Path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy. Used when Proxy Type is configured for "HTTPS" or "HTTPS With Credentials". | ||||||||
| Proxy Credentials | Optional | - | Credentials | Credentials to be used for the proxy communication. They are comprised of:
Required when "Proxy Type" is configured for "HTTPS" or "HTTPS With Credentials". |
Extension
...
Cancellation
When using a 7.0 or newer template, we must ensure that the “Always Cancel On Force Finish” is checked. This is to minimize leaving “orphan” processes on the OS without the option for the agent to see they are running.
Task Examples
Start Job Run with only required arguments
...
| Exit Code | Status Classification Code | Status Classification Description | Status Description |
|---|---|---|---|
| 0 | SUCCESS | Successful Execution | SUCCESS: Successful Task execution. |
| 1 | FAIL | Failed Execution | FAIL: < Error Description >. |
| 2 | AUTHENTICATION_ERROR | Bad credentials | AUTHENTICATION_ERROR: Account cannot be authenticated. |
| 3 | AUTHORIZATION_ERROR | Insufficient Permissions | AUTHORIZATION_ERROR: Account is not authorized to perform the requested action. |
| 10 | CONNECTION_ERROR | Bad connection data or connection timed out | CONNECTION_ERROR: < Error Description >. |
| 11 | CONNECTION_ERROR | Extension specific connection error | CONNECTION_ERROR: ProxyConnectionError: Failed to connect to proxy URL <url>. |
| 20 | DATA_VALIDATION_ERROR | Input fields validation error | DATA_VALIDATION_ERROR: Some of the input fields cannot be validated. See STDERR for more details. |
Extension Output
The Extension Output for AWS Lambda Universal Task successful execution is described below.
...
| Name | Location | Description |
|---|---|---|
| Universal Templates | https://docs.stonebranch.com/confluence/display/U70U71x/Universal+Templates | User documentation for creating Universal Templates in the Universal Controller user interface. |
| Universal Tasks | https://docs.stonebranch.com/confluence/display/UC70UC71x/Universal+Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. |
| AWS Glue | https://docs.aws.amazon.com/glue/?id=docs_gateway | Documentation for AWS Lambda. |
| IAM RBAC authorization model | https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html#introduction_attribute-based-access-control_compare-rbac | User Documentation for Comparing ABAC to the traditional RBAC model. |