Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Changes for v1.2.0

...

Template NameExtension NameExtension Version
AWS Batchue-aws-batch1.12.0

Refer to Changelog for version history information.

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

...

  • Support to submit a new Batch Job, with option to Terminate Job after a timeout period.
  • Support to submit a new Batch Job and wait until it reaches state "success" or "failed".
  • Support to read Batch Job status for an existing Job ID.
  • Support for authorization via IAM Role-Based Access Control (RBAC) strategy.
  • Support for Proxy communication via HTTP/HTTPS protocol.

...

To use the Universal Template, you first must perform the following steps:

  1. This Universal Task requires the Resolvable Credentials feature

    . Check

    , check that the Resolvable Credentials Permitted system property has been set to true.

  2. Download the provided ZIP file.

  3. In the Universal Controller UI, select Administration >Configuration > Universal Templates to display the current list of Universal Templates.

  4. Click Import Template.

  5. Select the template ZIP file and Import.

...

  1. For more information about Resolvable Credentials click here.

  2. To import the Universal Template into your Controller, follow the instructions here.

  3. When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list

...

  1. .

Configure Universal Task

For the a new Universal Task type AWS Batch, create a new task, and enter the task-specific details that were created in the Universal Templaterequired input fields.

Input Fields

The input fields for this Universal Extension are described below.

FieldInput typeDefault valueTypeDescription
ActionRequiredSubmit JobChoiceAction performed upon the task execution. Available actions:
  • Submit Job
  • Read Job Status

AWS Region

Optional since version 1.1.0

Optional-TextRegion for the Amazon Web Service. Find more information about the AWS Service endpoints and quotas here.

When AWS Region is not populated as part of the task definition, during task execution the integration will look for credentials on the task execution environment. Refer to configuration options for more information.

AWS Credentials

Optional since version 1.1.0

Optional-Credentials

AWS account credentials.

They are comprised of:

  • AWS access key ID
  • AWS secret access key
Role Based AccessOptionalFalseBoolean

Special type of authorization is provided by Role Assumption where the client sends his The Credentials definition should be as follows.

  • AWS Access Key ID as "Runtime User".
  • AWS Secret Access Key as "Runtime Password".

When AWS Credentials are not populated as part of the task definition, during task execution the integration will look for AWS Credentials on the task execution environment. Refer to configuration options for more information.

Role Based AccessOptionalFalseBoolean

Special type of authorization is provided by Role Assumption where the client sends his own credentials and the role he wants to assume from another user.

If allowed, the client receives temporary credentials with limited time access to some resources.

Role ARNOptional-Text

Role Amazon Resource Name (ARN) to have access to the SQS queue.Role ARN format: arn:aws:iam::<AWS Account ID>:instance-profile/<Role name>.

Required when Role Based Access has been checked.

Job NameOptional-TextName of the Batch Job that will be submitted.
Job DefinitionOptional-Text

Job definition used by this job. This value can be one of name , name:revision , or the Amazon Resource Name (ARN) for the job definition.

If name is specified without a revision then the latest active revision is used.

Required when Submit Job action has been selected.

Job QueueOptional-Text

The job queue where the job is submitted. You can specify either the name or the Amazon Resource Name (ARN) of the queue.

Required when Submit Job action has been selected.

Job TimeoutOptional-Integer

Can be filled when Action = Submit Job.

The timeout configuration (in seconds) for this Submit Job operation.

You can specify a timeout duration after which Batch terminates your jobs if they have not finished. If a job is terminated due to a timeout, it is not retried. The minimum value for the timeout is 60 seconds. This configuration overrides any timeout configuration specified in the job definition.

Optional when Submit Job action has been selected.

Container Overrides Script

Introduced in version 1.1.0

Optional-Script Field

The overrides that user want to apply on the container.

The script payload should be in JSON format.

Additional Job ParametersOptional-Array

Additional parameters passed to the job that replace parameter substitution placeholders that are set in the job definition. Parameters are specified as a key and value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition.

Optional when Submit Job action has been selected.

Job IDOptional-Text

The Job ID of an already submitted Batch Job.

Required when Read Job Status action has been selected.

Use ProxyOptionalFalseBooleanFlag to indicate whether Proxy shall be used in the communication with AWS.
Proxy TypeOptionalHTTPChoiceType of proxy connection to be used. Available options are:
  • HTTP
  • HTTPS
  • HTTPS with password

Required when Use Proxy is checked.

ProxyOptional-Text

Comma separated list of Proxy servers. Valid formats: http://proxyip:port or http://proxyip:port,https://proxyip:port.

Required when Use Proxy is checked.

Proxy CA Bundle FileOptional-Text

The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy.

Used when Proxy Type is configured for "HTTPS" or "HTTPS With PasswordCredentials".

Proxy CredentialsOptional-CredentialsCredentials to be used for the proxy communication. They are comprised of:
  • username
  • password

Required when Proxy Type is configured for "HTTPS" or "HTTPS With PasswordCredentials".

Task Examples

Submit Job

Example of AWS Batch Universal Task for submitting a new AWS Batch Job.

Image Removed

Read Job Status

Example of AWS Batch Universal Task for reading the status of an existing Batch Job by id.

Image Removed

Submit Job with Role Based Access Enabled

...

Wait for Success or Failure

Introduced in version 1.2.0
OptionalFalseBoolean

If selected, the task will continue running until Job reaches the "SUCCEDED" or "FAILED" state.

Required for Action "Submit Job".

Polling Interval

Introduced in version 1.2.0
Optional30Integer

The polling interval in seconds between checking for the Job status.

Required when Wait for Success or Failure ="True".

Task Examples

Note

There will be no specific cancel logic. In case of Retry, the Batch job will be re-submitted.

Submit Job

Example of AWS Batch Universal Task for submitting a new AWS Batch Job, with Role Based Access enabled.

Image Removed

...

Image Added

Submit Job with all optional input arguments

Example of AWS Batch Universal Task for reading job status through HTTP proxy connection.

Image Removed

...

submitting a new AWS Batch with the following arguments.

  • Environment Variables as AWS Region

    Example of

...

  • Universal Task for

...

Image Removed

HTTPS with password Proxy Type Task Configuration

Example of AWS Batch Universal Task for reading job status through HTTPS with password proxy connection.

Image Removed

Submit Job with Container Overrides payload

Example of AWS Batch Universal Task for submitting a new AWS Batch Job and setting a payload for containerOverrides parameter.

...

  • submitting a new AWS Batch Job providing no AWS Credentials in task definition and providing AWS Region as Environment Variable, leaving the respective input fields empty. AWS Credentials are expected in this case to be configured on the task execution environment. Please refer to AWS Credentials input field for more information.

  • Job Timeout

  • Container Overrides Script

  • Wait for Success or Failure

  • Proxy Type "HTTPS With Credentials"

Image Added

Read Job Status with Role Based Access Enabled

Example of AWS Batch Universal Task for reading the status of an existing Batch Job by ID, Role Based Access and "HTTP Proxy" connection.

Image Added

Task Output

Output Only Fields

The output parameters are set after every message processing. If they are 'preserved on re-run', they prevent a re-run of the task instance from restarting from scratch.
The output fields for this Universal Extension are fields for this Universal Extension are described below.

Status
FieldTypePreserved on re-runDescription
Exit CodeStatus Classification CodeStatus Classification Description
Job ARNTextFalseARN value of the newly submitted Batch Job.
Generated in Submit Job action.
Job IDTextFalseId of the newly submitted Batch Job.
Generated in Submit Job action.
Job NameTextFalse

Name of the Batch Job.
Generated in Read Job Status action.

Job StatusTextFalseStatus of the Batch Job.
Generated in Read Job Status action.

Exit Codes

The exit codes for AWS Batch Extension are described below.

Since 1.2.0: Generated for Action "Submit Job" with Wait for Success or Failure = "True", updating live during execution.

Job StatusTextFalse

Status of the Batch Job.
Generated in Read Job Status action.

Since 1.2.0: Generated for Action "Submit Job" with Wait for Success or Failure = "True", updating live during execution.

Exit Codes

The exit codes for AWS Batch Extension are described below.

Exit CodeStatus Classification CodeStatus Classification DescriptionStatus Description
0SUCCESSSuccessful ExecutionSUCCESS: Successful Task execution
1FAILFailed ExecutionFAIL: < Error Description >
2AUTHENTICATION_ERRORBad credentialsAUTHENTICATION_ERROR: Account cannot be authenticated.
3AUTHORIZATION_ERRORInsufficient PermissionsAUTHORIZATION_ERROR: Account is not authorized to perform the requested action.
10CONNECTION_ERRORBad connection data or connection timed outCONNECTION_ERROR: < Error Description >
11CONNECTION+ERRORExtension specific connection errorCONNECTION_ERROR: ProxyConnectionError: Failed to connect to proxy URL <url>
21SUCCESSSuccessful ExecutionSUCCESS: Job is in FAILED state.
22SUCCESSSuccessful ExecutionSUCCESS: Job is in RUNNING state.
23SUCCESSSuccessful ExecutionSUCCESS: Job is in STARTING state.
24SUCCESSSuccessful ExecutionSUCCESS: Job is in RUNNABLE state.
25SUCCESSSuccessful ExecutionSUCCESS: Job is in SUBMITTED state.
25SUCCESSSuccessful ExecutionSUCCESS: Job is in SUCCEEDED state>
11CONNECTION_ERRORExtension specific connection errorCONNECTION_ERROR: ProxyConnectionError: Failed to connect to proxy URL <url>
22SUCCESSSuccessful Execution

SUCCESS: Job is in RUNNING state.

Valid only for Action = "Read Job"

23SUCCESSSuccessful Execution

SUCCESS: Job is in STARTING state.

Valid only for Action = "Read Job"

24SUCCESSSuccessful Execution

SUCCESS: Job is in RUNNABLE state.

Valid only for Action = "Read Job"

25SUCCESSSuccessful Execution

SUCCESS: Job is in PENDING state.

Valid only for Action = "Read Job"

26SUCCESSSuccessful Execution

SUCCESS: Job is in SUBMITTED state.

Valid only for Action = "Read Job"

27SUCCESSSuccessful Execution

SUCCESS: Job is in SUCCEEDED state.

Valid only for Action = "Read Job"

Attribute changed is populated as follows

  • true, in case Action "Submit Job" is executed successfully.
  • false, in case Action "Read Job" is executed.

The Extension output contains attribute result. Attribute result contains the following sub-attributes:

AttributeTypeDescription
out_job_arnstringARN value of the newly submitted Batch Job.

Generated in "Submit Job" Action.
out_job_idstringID of the newly submitted Batch Job.

Generated in "Submit Job" Action.
out_job_namestringName of the Batch Job.
Generated in "Read Job" Action

Since 1.2.0: Generated for Action "Submit Job" with Wait for Success or Failure = "True", updating live during execution.
out_job_statusstringStatus of the Batch Job.
Generated in "Read Job" Action

Since 1.2.0: Generated for Action "Submit Job" with Wait for Success or Failure = "True", updating live during execution.

Extension Output

The Extension Output for AWS Batch Universal Task successful execution is described below.

{
    "exit_code": 0,

   "status_description": "SUCCESS: AWS Batch Task submitted successfully",
    "changed": true,
    "invocation": {
 
      "extension": "ue-aws-batch",
 
      "version": "1.12.0",
        "fields": {
            "action": "Submit Job",
  
         "credentials_user": "****",
  
         "credentials_password": "****",
            "region": "us-east-1",
  
         "role_based_access": false,
  
         "role_arn": null,
 
          "job_name": "ue-aws-batch-demo-job",
 
          "job_definition": "ue-aws-batch-job-definition",
  
         "job_queue": "ue-aws-batch-test-queue",
  
         "job_timeout": 100,

           "additional_job_parameters": [],parameters":[
            
         ],
         "job_id": null,
            "use_proxy": false,
  
         "proxy_type": null,
  
         "proxy": null,
            "proxy_credentials_user": null,
  
         "proxy_credentials_password": null,
  
         "proxy_ca_bundle_file": null,
 
          "container_overrides_script": {
            
   "environment": [           "environment":[
         {      {
                  "name": "ENV_VAR1",
                        "value": "value 1"
    
               },
 
                  {
     
                  "name": "ENV_VAR2",
    
                   "value": "value 2"
                    }
               }
]            ]
}         },
    },     "resultwait_for_success_or_failure":true,
{         "outpolling_job_arninterval": "arn:aws:batch:us-east-1:123456789:job/3c50a3f7-02de-46bc-bd67-490d5ceb2bc2"3
      }
   },
   "result":{
      "out_job_idname": "3c50a3f7ue-02deaws-46bcbatch-bd67demo-490d5ceb2bc2job",
    }
}

Attribute result contains the following sub-attributes.

...

  "out_job_status":"SUCCEEDED",
      "out_job_arn

...

":"arn:aws:batch:us-east-1:123456789:job/1234-5678-90123-456-7890",
      "out_job_id":"1111-11111-11111-11111-11111"
   }
}

STDOUT and STDERR

STDOUT and STDERR provide additional information to User. The populated content can be changed in future versions of this extension without notice. Backward compatibility is not guaranteed.

Document References

This document references the following documents:.

Universal Templates
NameDocument LinkLocationDescription
https://docs.stonebranch.com/confluence/display/UC72x/Universal + TemplatesUser documentation for creating, working with and understanding Universal Templates in the Universal Controller user interface.Universal Taskshttps://docs.stonebranch.com/confluence/display/UC72x/Universal+and Integrations.
Universal TasksUser documentation for creating Universal Tasks in the Universal Controller user interface.
AWS Batchhttps://docs.aws.amazon.com/batch/latest/userguide/what-is-batch.htmlUser guide for AWS Batch.
IAM RBAC authorization modelhttps://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html#introduction_attribute-based-access-control_compare-rbacUser Documentation for Comparing ABAC to the traditional RBAC model.

...

RBAC authorization modelUser Documentation for Comparing ABAC to the traditional RBAC model.

Anchor
Changelog
Changelog
Changelog

ue-aws-batch-1.2.0 (2022-07-19)

Enhancements

  • Added: Support Submit Batch Job and Wait until Job Reaches status "Succeeded" or "Failed (#29278)
  • Added: Output fields are updated during execution for the above scenario.
  • Added: Log payload response for Read and Submit Job Action on debug mode.

ue-aws-batch-1.1.0 (2022-06-09)

Enhancements

  • Added: Provide the capability capabilty to rely on AWS credentials set-up on the environment where the extension is running and therefore it is not mandatory to be passed on the task definition as input fields. The same applies to AWS Region. (#28286)
  • Added: AWS Region can be resolved from environment variables. (#28286)
  • Added: Support for "containerOverrides" parameter. (#28860)

ue-aws-batch-1.0.1 (2022-03-14)

Enhancements

  • Added: Update of extension icon. (#27753)

Fixes

  • Fixed Fix wrong exit code when invalid ARN is provided. (#27736)
  • Fixed Fix response message when status is SUCCESS. (#27730)

...