Versions Compared

Old Version 1

changes.mady.by.user Former user

Saved on

compared with

New Version 2

changes.mady.by.user Kostas Papageorgiou

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/

Overview

AWS Batch is a set of batch management capabilities that enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS.

This Universal Extension provides the capability to submit new AWS Batch Jobs as well as read the status for an existing AWS Batch Job.

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Software Requirements for Universal Template and Universal Task

Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.

Software Requirements for Universal Agent

Both Windows and Linux agents are supported:

  • Universal Agent for Windows x64 Version 7.0.0.0 and later with python options installed.
  • Universal Agent for Linux Version 7.0.0.0 and later with python options installed.

Software Requirements for Universal Controller

Universal Controller Version 7.0.0.0 and later.

Network and Connectivity Requirements

Extensions' Universal Agent host should be able to reach AWS Batch REST endpoints, through the configured VPC.

More information on setting up AWS Batch VPC, can be found in the AWS Batch official user guide.

Key Features

This Universal Extension provides the following key features:

  • Support to submit a new Batch Job, with option to Terminate Job after a timeout period.
  • Support to read Batch Job status for an existing Job ID.
  • Support for authorization via IAM Role-Based Access Control (RBAC) strategy.
  • Support for Proxy communication via HTTP/HTTPS protocol.

Import Universal Template

To use the Universal Template, you first must perform the following steps:

  1. This Universal Task requires the Resolvable Credentials feature, check that the Resolvable Credentials Permitted system property has been set to true.

  2. Download the provided ZIP file.

  3. In the Universal Controller UI, select Administration >Configuration > Universal Templates to display the current list of Universal Templates.

  4. Click Import Template.

  5. Select the template ZIP file and Import.

When the template has been imported successfully, the Universal Template will appear on the list, refresh your Navigation Tree to see these tasks in the Automation Center Menu.

Configure Universal Task

For the new Universal Task type AWS Batch, create a new task, and enter the task-specific details that were created in the Universal Template.

Input Fields

The input fields for this Universal Extension are described below.

...

  • Submit Job
  • Read Job Status

...

AWS account credentials.

They are comprised of:

  • AWS access key ID
  • AWS secret access key

...

Special type of authorization is provided by Role Assumption where the client sends his own credentials and the role he wants to assume from another user.

If allowed, the client receives temporary credentials with limited time access to some resources.

...

Role Amazon Resource Name (ARN) to have access to the SQS queue.Role ARN format: arn:aws:iam::<AWS Account ID>:instance-profile/<Role name>.

Required when Role Based Access has been checked.

...

Job definition used by this job. This value can be one of name , name:revision , or the Amazon Resource Name (ARN) for the job definition.

If name is specified without a revision then the latest active revision is used.

Required when Submit Job action has been selected.

...

The job queue where the job is submitted. You can specify either the name or the Amazon Resource Name (ARN) of the queue.

Required when Submit Job action has been selected.

...

Can be filled when Action = Submit Job.

The timeout configuration (in seconds) for this Submit Job operation.

You can specify a timeout duration after which Batch terminates your jobs if they have not finished. If a job is terminated due to a timeout, it is not retried. The minimum value for the timeout is 60 seconds. This configuration overrides any timeout configuration specified in the job definition.

Optional when Submit Job action has been selected.

...

Additional parameters passed to the job that replace parameter substitution placeholders that are set in the job definition. Parameters are specified as a key and value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition.

Optional when Submit Job action has been selected.

...

The Job ID of an already submitted Batch Job.

Required when Read Job Status action has been selected.

...

  • HTTP
  • HTTPS
  • HTTPS with password

Required when Use Proxy is checked.

...

Comma separated list of Proxy servers. Valid formats: http://proxyip:port or http://proxyip:port,https://proxyip:port.

Required when Use Proxy is checked.

...

The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy.

Used when Proxy Type is configured for "HTTPS" or "HTTPS With Password".

...

  • username
  • password

Required when Proxy Type is configured for "HTTPS" or "HTTPS With Password".

Task Examples

Submit Job

Example of AWS Batch Universal Task for submitting a new AWS Batch Job.

Image Removed

Read Job Status

Example of AWS Batch Universal Task for reading the status of an existing Batch Job by id.

Image Removed

Submit Job with Role Based Access Enabled

Example of AWS Batch Universal Task for submitting a new AWS Batch Job, with Role Based Access enabled.

Image Removed

HTTP Proxy Type Task Configuration

Example of AWS Batch Universal Task for reading job status through HTTP proxy connection.

Image Removed

HTTPS Proxy Type Task Configuration

Example of AWS Batch Universal Task for reading job status through HTTPS proxy connection.

Image Removed

HTTPS with password Proxy Type Task Configuration

Example of AWS Batch Universal Task for reading job status through HTTPS with password proxy connection.

Image Removed

Task Output

Output Only Fields

The output parameters are set after every message processing. If they are 'preserved on re-run', they prevent a re-run of the task instance from restarting from scratch.
The output fields for this Universal Extension are described below.

...

Exit Codes

The exit codes for AWS Batch Extension are described below.

...

Extension Output

The Extension Output for AWS Batch Universal Task successful execution is described below.

{
	"exit_code":0,
	"status_description":"ue-aws-batch: Job submited successfuly",
	"changed":true,
	"invocation":{
		"extension":"ue-aws-batch",
		"version":"1.0.0",
		"fields":{
			"action":"Submit Job",
			"credentials_user": "****",
			"credentials_password": "****",
			"aws_default_region":"us-east-1",
			"job_name":"example",
			"job_definition":"sleep60",
			"job_queue":"HighPriority",
			"role_based_access":"False",
			"role_arn":null,
			"parameters": [
				{
					"name":"parameter1",
					"value":"value1"
				},
				{
					"name":"parameter2",
					"value":"value2"
              	}
			],
			"job_timeout":null,
			"use_proxy":false,
			"proxy_type":null,
			"proxy":null,
			"proxy_credentials_user":null,
			"proxy_ca_bundle_file":null
		},
		"result":{
			"out_job_arn":"arn:aws:batch:us-east-1:169598488829:job/912ac941-ffda-4b3e-b8ec-0f644cad650d",
			"out_job_id": "876da822-4198-45f2-a252-6cea32512ea8"
		}
	}
}

Document References

This document references the following documents:

...