Disclaimer

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at Stonebranch Integration Hub - Terms of Use.

Overview

Azure Batch is a set of batch management capabilities that enables developers, scientists, and engineers to easily and efficiently create Azure Batch jobs and tasks. This Universal Extension provides the capability to add Azure Jobs, add Azure Batch Tasks to existing Azure Jobs, and optionally monitor Azure Task execution for completion (successful or failed).

Version Information

Template Name	Extension Name	Extension Version
Azure Batch	ue-azure-batch	1.0.0

Refer to Changelog for version history information.

Key Features

Actions:

Add an Azure Batch Job to a specific pool of nodes by providing the Azure Batch Job configuration in JSON format.
Add a task as part of a specific job & optionally monitor task for completion.

Authentication/Authorization:

Ability to connect to Azure using Client Credentials Grant Type.
Ability to connect to Azure using User Credentials Grant Type.

Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Area	Details
Python Version	Requires Python 3.7.6 or higher. Tested with the Universal Agent bundled Python distribution.
Python Libraries	Microsoft Authentication Library (MSAL)
Universal Agent	Both Windows and Linux agents are supported: Universal Agent for Windows x64 Version 7.1.0.0 and later with python options installed. Universal Agent for Linux Version 7.1.0.0 and later with python options installed.
Universal Controller	Universal Controller Version 7.1.0.0 and later.
Network and Connectivity	Needed outbound internet connectivity.
Southbound Systems	Microsoft Azure

Supported Actions

Action: “Job | Add”

Upload a Job to Azure Batch, assigned it the specified pool and verify its proper initialization. If Azure fails to initialize the task after 30 seconds, the initialization is considered a failure. After a successful execution, the job's status is "Active" and is ready for use.

Configuration examples

User Scenario: Upload a Job definition using User Credentials.

Action Output

Output Type

Description (Successful Execution)

Example (Successful Execution)

EXTENSION

Standard UC format

The extension output follows the standard UC output format, providing:

“exit_code“, “status“ , “status_description“: General info regarding the task execution.
“invocation” > “fields”: The task configuration used for this task execution.

Result fields

“result“: The result of querying the newly added job.

{
"exit_code" : 0,
"status" : "SUCCESS",
"status_description" : "Task executed successfully.",
"invocation": {
"fields" : { "..." }
},
"result" : {
"odata.metadata" : "https://testbatch.westeurope.batch.azure.com/$metadata#jobs/@Element",
"id" : "job100",
"url" : "https://testbatch.westeurope.batch.azure.com/jobs/job01",
"eTag" : "0x8DB1A5FAECB1F94",
"lastModified" : "2023-03-01T14:17:25.9819924Z",
"creationTime" : "2023-03-01T14:17:25.9539136Z",
"state" : "active",
"stateTransitionTime" : "2023-03-01T14:17:25.9819924Z",
"priority" : 0,
"maxParallelTasks" : -1,
"usesTaskDependencies" : false,
"allowTaskPreemption" : false,
"constraints" : {
"maxWallClockTime" : "P10675199DT2H48M5.4775807S",
"maxTaskRetryCount" : 0
},
"poolInfo" : {
"poolId" : "pool1"
},
"executionInfo" : {
"startTime" : "2023-03-01T14:17:25.9819924Z",
"poolId" : "pool1"
},
"onAllTasksComplete" : "noaction",
"onTaskFailure" : "noaction"
}
}

Description (Failed Execution)

Example (Failed Execution)

Standard UC format

“exit_code“, “status“ , “status_description“: Will be set according to the specific error happened during the execution. A list of possible values is provided in the extension exit codes table in this document.

Result fields

“result“ > “error“: If additional error details can be retrieved when the error occurred, they will be provided in this JSON element.
“result“ > “http_code“: If the http error from the error response can be retrieved, it will be provided in this integer element.

{
"exit_code" : 1,
"status" : "FAILED",
"status_description" : "Task execution failed with error: ...",
"invocation" : {
"fields" : {
"..."
}
},
"result" : {
"error" : { "..." },
"http_code" : 500
}
}

Action: “Task | Add”

Add a task as part of a specific job and verify its proper initialization. After a successful initialization, the task's state is "Active" and will be automatically run by Azure Batch.

The task can optionally be monitored for completion. In that case, Azure is polled every one second by default (can be parameterized by the user) for the task’s execution status.

Configuration examples

User scenario: Add a task and do not wait for its completion.

User scenario: Add a task and wait for its completion.

Action Output

Output Type

Description (Successful Execution)

Example (Successful Execution)

EXTENSION

Standard UC format

The extension output follows the standard UC output format, providing:

“exit_code“, “status“ , “status_description“: General info regarding the task execution.
“invocation” > “fields”: The task configuration used for this task execution.

Result fields

“result“: The result of querying the newly added task.

{
"exit_code" : 0,
"status" : "SUCCESS",
"status_description" : "Task executed successfully.",
"invocation" : {
"fields" : { "..." }
},
"result" : {
"odata.metadata" : "https://batchsimon.westeurope.batch.azure.com/$metadata#tasks/@Element",
"id" : "random02",
"url" : "https://testbatch.westeurope.batch.azure.com/jobs/job01/tasks/random01",
"eTag" : "0x8DB1A58BD1505C7",
"creationTime" : "2023-03-01T13:27:43.4772935Z",
"lastModified" : "2023-03-01T13:27:43.4772935Z",
"state" : "active",
"stateTransitionTime" : "2023-03-01T13:27:43.4772935Z",
"commandLine" : "ls -al",
"userIdentity" : {
"autoUser" : {
"scope" : "pool",
"elevationLevel" : "nonadmin"
}
},
"constraints" : {
"maxWallClockTime" : "P10675199DT2H48M5.4775807S",
"retentionTime" : "P7D",
"maxTaskRetryCount" : 0,
"requiredSlots" : 1,
"executionInfo" : {
"retryCount" : 0,
"requeueCount" : 0
}
}
}
}

Description (Failed Execution)

Example (Failed Execution)

Standard UC format

“exit_code“, “status“ , “status_description“: Will be set according to the specific error happened during the execution. A list of possible values is provided in the extension exit codes table in this document.

Result fields

“result“ > “error“: If additional error details can be retrieved when the error occurred, they will be provided in this JSON element.
“result“ > “http_code“: If the http error from the error response can be retrieved, it will be provided in this integer element.

{
"exit_code" : 1,
"status" : "FAILED",
"status_description" : "Task execution failed with error: ...",
"invocation" : {
"fields" : { "..." }
},
"result" : {
"error" : { "..." },
"http_code" : 500
}
}

Extension Parameters & Fields

Field	Type	Default Value	Mandatory	Description	Introduced in Version
Action	Choice	Job \| Add	Yes	The action performed upon the task execution. Available options: Job \| Add Task(s) \| Add	1.0.0
Authentication	Choice	Client Credentials	Yes	The authentication method for this task to be run with. Available options: User Credentials Client Credentials	1.0.0
Credentials	Credentials		Yes	The Azure AD user credentials or the Azure application client secret. For “User Credentials“ authentication method: Runtime User: the Azure AD username Runtime Password: the Azure AD user password For “Client Credentials“ authentication method: Token: Client Secret (value)	1.0.0
Application (client) ID	Text		Yes	The Application (client) ID your application belongs to. You can find it in “Overview” section of your App registration, in Azure Active Directory.	1.0.0
Directory (tenant) ID	Text		Yes	The Directory (tenant) ID your application belongs to. You can find it in “Overview” section of your App registration, in Azure Active Directory.	1.0.0
Batch Account Endpoint	Text		Yes	The Batch account endpoint. You can find it in “Overview” section of your Batch Account, in Azure Active Directory.	1.0.0
Job ID	Text		Yes, when Action “Task(s) \| Add“ is selected.	The Job ID the new task will be added to.	1.0.0
Wait for Success or Failure	Checkbox	False	No	Wait for the added task to be completed or not.	1.0.0
Polling Interval	Integer	1	Yes if “Wait for Completion“ is checked	The polling interval in seconds between checking for the task status. Minimum acceptable value is “1“.	1.0.0
JSON Payload (Job / Task)	Large Text		Yes	The JSON payload used to create a job or a task.	1.0.0

Extension Exit Codes

Extension Exit Code	Status	Status Description	Meaning
0	SUCCESS	“Task executed successfully.“	Successful Execution
1	FAIL	“Execution Failed: <error details>”	Task Add: The task was added & executed, but the task result was failure (raised only if the user selected to wait for task completion). Other cases: Any error that can not be mapped to the rest of the error codes.
3	AUTHORIZATION_ERROR	“Authorization Error: <error details>.”	Account is not authorized to perform the requested action.
10	CONNECTION_ERROR	“Connection failed: <error details>.“	Connectivity to Azure API endpoint failed.
20	DATA_VALIDATION_ERROR	“Data Validation Error: <error details>.“	User data validation error.
30	JOB_ADD_FAILED	“Job add failure: <error details>.“	Job add failed (operation not completed within a time window of 30 seconds is also considered a failure).
40	TASK_ADD_FAILED	“Task add failure: <error details>.“	Task add failed (operation not completed within a time window of 30 seconds is also considered a failure).

Import Universal Template

To use the Universal Template, you first must perform the following steps.

This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.
To import the Universal Template into your Controller, follow these instructions.
When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.

Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information refer to Integration Modifications.

Configure Universal Task

For a new Universal Task, create a new task, and enter the required input fields.

Integration Modifications

Modifications applied by users or customers, before or after import, might affect the supportability of this integration. The following modifications are discouraged to retain the support level as applied for this integration.

Python code modifications should not be done.
Template Modifications
- General Section
  - "Name", "Extension", "Variable Prefix", "Icon" should not be changed.
- Universal Template Details Section
  - "Template Type", "Agent Type", "Send Extension Variables", "Always Cancel on Force Finish" should not be changed.
- Result Processing Defaults Section
  - Success and Failure Exit codes should not be changed.
  - Success and Failure Output processing should not be changed.
- Fields Restriction Section
  The setup of the template does not impose any restrictions, However with respect to "Exit Code Processing Fields" section.
  1. Success/Failure exit codes need to be respected.
  2. In principle, as STDERR and STDOUT outputs can change in follow-up releases of this integration, they should not be considered as a reliable source for determining success or failure of a task.

Users and customers are encouraged to report defects, or feature requests at Stonebranch Support Desk.

Document References

This document references the following documents:

Document Link	Description
Universal Templates	User documentation for creating, working with and understanding Universal Templates and Integrations.
Universal Tasks	User documentation for creating Universal Tasks in the Universal Controller user interface.
Azure Batch Documentation	User Documentation for Microsoft Azure Batch service.

Changelog

ue-azure-batch-1.0.0 (2023-03-20)

Initial Version