Disclaimer

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/

Overview

AWS Batch is a set of batch management capabilities that enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS.

This Universal Extension provides the capability to submit new AWS Batch Jobs as well as read the status for an existing AWS Batch Job.

Version Information

Refer to ChangeLog for version history information.

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Software Requirements for Universal Template and Universal Task

Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.

Software Requirements for Universal Agent

Both Windows and Linux agents are supported:

• Universal Agent for Windows x64 Version 7.2.0.0 and later with python options installed.
• Universal Agent for Linux Version 7.2.0.0 and later with python options installed.

Software Requirements for Universal Controller

Universal Controller Version 7.2.0.0 and later.

Network and Connectivity Requirements

Extensions' Universal Agent host should be able to reach AWS Batch REST endpoints, through the configured VPC.

More information on setting up AWS Batch VPC, can be found in the AWS Batch official user guide.

Key Features

This Universal Extension provides the following key features:

• Support to submit a new Batch Job, with option to Terminate Job after a timeout period.
• Support to submit a new Batch Job and wait until it reaches state "success" or "failed".
• Support to read Batch Job status for an existing Job ID.
• Support for authorization via IAM Role-Based Access Control (RBAC) strategy.
• Support for Proxy communication via HTTP/HTTPS protocol.

Import Universal Template

To use the Universal Template, you first must perform the following steps:

1. This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.

2. To import the Universal Template into your Controller, follow the instructions here.

3. When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.

Configure Universal Task

For a new Universal Task, create a new task, and enter the required input fields.

Input Fields

The input fields for this Universal Extension are described below.

Task Examples

Submit Job

Example of AWS Batch Universal Task for submitting a new AWS Batch Job.

Submit Job with all optional input arguments

Example of AWS Batch Universal Task for submitting a new AWS Batch with the following arguments.

• Environment Variables as AWS Region

  Example of Universal Task for submitting a new AWS Batch Job providing no AWS Credentials in task definition and providing AWS Region as Environment Variable, leaving the respective input fields empty. AWS Credentials are expected in this case to be configured on the task execution environment. Please refer to AWS Credentials input field for more information.

• Job Timeout

• Container Overrides Script

• Wait for Success or Failure

• Proxy Type "HTTPS With Credentials"

Read Job Status with Role Based Access Enabled

Example of AWS Batch Universal Task for reading the status of an existing Batch Job by ID, Role Based Access and "HTTP Proxy" connection.

Task Output

Output Only Fields

The output fields for this Universal Extension are described below.

Exit Codes

The exit codes for AWS Batch Extension are described below.

Attribute changed is populated as follows

• true, in case Action "Submit Job" is executed successfully.
• false, in case Action "Read Job" is executed.

The Extension output contains attribute result. Attribute result contains the following sub-attributes:

Extension Output

The Extension Output for AWS Batch Universal Task successful execution is described below.

{
   "exit_code":0,
   "status_description":"SUCCESS: AWS Batch Task submitted successfully",
   "changed":true,
   "invocation":{
      "extension":"ue-aws-batch",
      "version":"1.3.0",
      "fields":{
         "action":"Submit Job",
         "credentials_user":"****",
         "credentials_password":"****",
         "region":"us-east-1",
         "role_based_access":false,
         "role_arn":null,
         "job_name":"ue-aws-batch-demo-job",
         "job_definition":"ue-aws-batch-job-definition",
         "job_queue":"ue-aws-batch-test-queue",
         "job_timeout":100,
         "additional_job_parameters":[
            
         ],
         "job_id":null,
         "use_proxy":false,
         "proxy_type":null,
         "proxy":null,
         "proxy_credentials_user":null,
         "proxy_credentials_password":null,
         "proxy_ca_bundle_file":null,
         "container_overrides_script":{
            "environment":[
               {
                  "name":"ENV_VAR1",
                  "value":"value 1"
               },
               {
                  "name":"ENV_VAR2",
                  "value":"value 2"
               }
            ]
         },
         "wait_for_success_or_failure":true,
         "polling_interval":3,
         "full_job_info": true

      }
   },
   "result":{
      "out_job_name":"ue-aws-batch-demo-job",
      "out_job_status":"SUCCEEDED",
      "out_job_arn":"arn:aws:batch:us-east-1:123456789:job/1234-5678-90123-456-7890",
      "out_job_id":"1111-11111-11111-11111-11111",
      "job_info: [ ... ]
   }
}

Cancelation and Re-Run

There will be no specific cancel logic. In case of Retry, the Batch job will be re-submitted.

STDOUT and STDERR

STDOUT and STDERR provide additional information to User. The populated content can be changed in future versions of this extension without notice. Backward compatibility is not guaranteed.

Document References

This document references the following documents.

Changelog

ue-aws-batch-1.3.1 (2024-02-01)

Fixes

• Fixed: Revert `certifi` as requirement. (#35294)

ue-aws-batch-1.3.0 (2023-12-11)

Enhancements

• Added: Extension Output is enhanced to provide the latest job information coming from the AWS Batch Service. (#33753)

ue-aws-batch-1.2.1 (2023-08-04)

Fixes

• Fixed: Field 'Proxy Type' raised a data validation error on Controller 7.3.0.0 and later. (#34038)

ue-aws-batch-1.2.0 (2022-07-19)

Enhancements

• Added: Support Submit Batch Job and Wait until Job Reaches status "Succeeded" or "Failed (#29278)
• Added: Output fields are updated during execution for the above scenario.
• Added: Log payload response for Read and Submit Job Action on debug mode.

ue-aws-batch-1.1.0 (2022-06-09)

Enhancements

• Added: Provide the capabilty to rely on AWS credentials set-up on the environment where the extension is running and therefore it is not mandatory to be passed on the task definition as input fields. The same applies to AWS Region. (#28286)
• Added: AWS Region can be resolved from environment variables. (#28286)
• Added: Support for "containerOverrides" parameter. (#28860)

ue-aws-batch-1.0.1 (2022-03-14)

Enhancements

• Added: Update of extension icon. (#27753)

Fixes

• Fixed Fix wrong exit code when invalid ARN is provided. (#27736)
• Fixed Fix response message when status is SUCCESS. (#27730)