Amazon SQS Monitor
Disclaimer
Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/
Overview
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
This Universal Extension provides the capability to monitor AWS SQS messages from an existing queue and run Universal Task or workflows accordingly.
Version Information
Template Name | Extension Name | Extension Version |
---|---|---|
Amazon SQS Monitor | ue-aws-sqs-monitor | 1.1.1 |
Refer to Changelog for version history information.
Software Requirements
This integration requires a Universal Agent and a Python runtime to execute the Universal Task.
Software Requirements for Universal Template and Universal Task
Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.
Software Requirements Universal Agent
Both Windows and Linux agents are supported:
Universal Agent for Windows x64 Version 7.1.0.0 and later with python options installed.
Universal Agent for Linux Version 7.1.0.0 and later with python options installed.
Software Requirements Universal Controller
Universal Controller Version 7.0.0.0 and later.
This Universal Task requires that Universal Controller property Web Service Default Response Content (uc.web_service.response.content.default) is set to JSON.
Network and Connectivity Requirements
Extension's Universal Agent host should be able to establish connection with:
- AWS SQS Queue REST endpoints (The AWS Credentials provided in the Amazon SQS Queue Monitor Universal Task, should have sufficient permissions on AWS to get messages from the specified queue).
- The Universal Controller host URL (The Universal Controller Credentials provided in permissions to launch a Universal Controller Task).
Key Features
This Universal Extension provides the following main features:
- Action
- Monitor AWS SQS messages from a standard or a FIFO queue.
- Launch a task in Universal Controller with variables holding the id, body, attributes, message attributes and receipt handle for each fetched message.
- Authentication
- AWS Credentials.
- IAM Role-Based Access Control (RBAC) strategy.
- Other
- Communication through Proxy with use of HTTP or HTTPS.
Import Universal Template
To use the Universal Template, you first must perform the following steps.
This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.
To import the Universal Template into your Controller, follow the instructions here.
When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.
Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information refer to Integration Modifications.
Configure Universal Task
For a new Universal Task, create a new task, and enter the required input fields.
Input Fields
The input fields for this Universal Extension are described in the following table.
Field | Input type | Default value | Type | Description |
---|---|---|---|---|
Action | Required | Monitor Messages | Choice | The action performed upon the task execution. |
Region Optional since version 1.1.0 | Optional | - | Text | Region for the Amazon Web Service. Find more information about the AWS Service endpoints and quotas here. When AWS Region is not populated as part of the task definition, during task execution the integration will look for AWS Region on the task execution environment. Refer to configuration options for more information. |
AWS Credentials | Optional | - | Credentials | The Credentials definition should be as follows.
When AWS Credentials are not populated as part of the task definition, during task execution the integration will look for AWS Credentials on the task execution environment. Refer to configuration options for more information. |
Queue Name | Required | - | Dynamic Choice | Dynamic fetched list of queue names. The user can select the required queue name from a drop-down list. |
Queue Name Prefix | Optional | - | Text | A prefix to use for filtering the Queue Names list results. Only those queues whose name begins with the specified string are returned. Queue Names are case-sensitive. |
Role Based Access | Optional | False | Boolean | Flag to configure whether the type of authorization is based on IAM Role-Based Access Control(RBAC) strategy. Find more information about the IAM RBAC authorization model here. |
Role ARN | Optional | - | Text | The Role Amazon Resource Name (ARN) to have access to the SQS queue. Role ARN format: Required when Role Based Access has been clicked. |
Use Proxy | Optional | False | Boolean | Flag to configure whether Proxy should be used in communication. |
Proxy Type | Optional | HTTP | Choice | Type of proxy connection to be used. Available options are:
Required when Use Proxy is enabled. |
Proxy | Optional | - | Text | Proxy server and port. Valid format: Required when Use Proxy is enabled. |
Proxy CA Bundle File | Optional | - | Text | The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy. Used when Proxy Type is configured for "HTTPS" or "HTTPS With Credentials". |
Proxy Credentials | Optional | - | Credentials | Credentials to be used for the proxy communication. The credential definition should be as follows.
Required when "Proxy Type" is configured for "HTTPS" or "HTTPS With Credentials". |
Attribute Names | Optional | All | Choice | A list of attributes to be returned along with each message. Available options are:
|
Message Attribute Names | Optional | - | Text | Attribute names of the message to be returned from the request, separated by a comma. For example:
Retrieve all message attributes starting with a prefix, using for example Naming rules and restrictions:
|
Maximum Number of Messages | Required | 10 | Integer | The maximum number of messages to return within each request. Valid values from 1 to 10. |
Visibility Timeout (seconds) | Required | 1 | Integer | The duration that the received messages are hidden from subsequent retrieve requests. Valid values range from 1 to 43200. |
Wait Time (seconds) | Required | 1 | Integer | The duration for which the call waits for Maximum Number of Messages to arrive in the queue before returning. If the requested number of messages is available, the messages are returned sooner than Wait Time (seconds). |
Polling Interval (seconds) | Required | 60 | Integer | The amount of time the task will wait before pooling for a new message batch when no messages are found during the last cycle. Valid values from 0. |
Delete After Processing | Required | True | Boolean | Flag to delete the received message after processing. If an error occurs during the launching task process, the received message will not be deleted regardless of the Delete after processing flag. |
Launch Task Name | Required | - | Text | The name of the Task to be launched in the Universal Controller. The task will always be launched with the Task variables:
|
Universal Controller URL | Required | - | Text | The Universal Controller URL to launch the Task when receiving a message. Valid URL format is: http(s)://controller_ip:controller_port/uc. |
Universal Controller Credentials | Required | - | Credentials | The credentials of Universal Controller URL to be used for launching the task The Credentials must have “Web Service Access” Permissions. |
Task Examples
Monitor Messages from Queue
Example of Universal Task for monitoring messages from an existing queue.
Monitor Messages with Role Based Access Enabled
Example of Universal Task for monitoring messages with Role Based Access Enabled. The credentials in this case are optional and will be stored as environment variables on the execution environment.
HTTPS With Credentials Proxy Type Task Configuration
Example of Universal Task for monitoring messages through HTTPS With Credentials proxy connection:
Task Output
Output Only Fields
The output parameters are set after every message processing. If they are 'preserved on re-run', they prevent a re-run of the task instance from restarting from scratch.
The output fields for this Universal Extension are described in the following table.
Field | Type | Preserved on re-run | Description |
---|---|---|---|
Last Monitor Datetime | Text | False | Timestamp of the last time the messages were pooled from the queue. |
Last Monitor Message Count | Integer | False | Number of fetched messages since the start of the monitoring (task instance). |
Launched Task IDs | Text | False | sysID (32 char) of the last launched task as returned by UAC REST API. |
Read Message Count | Integer | False | Number of fetched messages since the start of the monitoring (task instance). |
Launched Task Count | Integer | False | Number of launched tasks instances since the start of the monitoring (task instance). |
Exit Codes
The exit codes for this Universal Extension are described in the following table.
Exit Code | Status Classification Code | Status Classification Description | Status Description |
---|---|---|---|
0 | SUCCESS | Successful Execution | SUCCESS: Successful Task execution |
1 | FAIL | Failed Execution | FAIL: < Error Description > |
3 | AUTHORIZATION_ERROR | Insufficient Permissions | AUTHORIZATION_ERROR: The authorization credentials provided for the request are invalid. |
10 | CONNECTION_ERROR | Bad connection data or connection timed out | CONNECTION_ERROR: < Error Description > |
20 | DATA_VALIDATION_ERROR | Bad input fields validation | DATA_VALIDATION_ERROR: Some of the input fields cannot be validated. See STDERR for more details. |
STDOUT and STDERR
STDOUT and STDERR provide additional information to User. The populated content can be changed in future versions of this extension without notice. Backward compatibility is not guaranteed.
Integration Modifications
Modifications applied by users or customers, before or after import, might affect the supportability of this integration. The following modifications are discouraged to retain the support level as applied for this integration.
- Python code modifications should not be done.
- Template Modifications
- General Section
- "Name", "Extension", "Variable Prefix", "Icon" should not be changed.
- Universal Template Details Section
- "Template Type", "Agent Type", "Send Extension Variables", "Always Cancel on Force Finish" should not be changed.
- Result Processing Defaults Section
- Success and Failure Exit codes should not be changed.
- Success and Failure Output processing should not be changed.
- Fields Restriction Section
The setup of the template does not impose any restrictions, However with respect to "Exit Code Processing Fields" section.- Success/Failure exit codes need to be respected.
- In principle, as STDERR and STDOUT outputs can change in follow-up releases of this integration, they should not be considered as a reliable source for determining success or failure of a task.
- General Section
Users and customers are encouraged to report defects, or feature requests at Stonebranch Support Desk.
Document References
This document references the following documents:
Document Link | Description |
---|---|
Universal Templates | User documentation for creating, working with and understanding Universal Templates and Integrations. |
Universal Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. |
Amazon SQS Message | User documentation for Amazon SQS. |
IAM RBAC authorization model | User Documentation for Comparing ABAC to the traditional RBAC model. |
Changelog
ue-aws-sqs-monitor-1.1.1 (2023-07-28)
Fixes
- Fixed: Field 'Proxy Type' raised a data validation error on Controller 7.3.0.0 and later. (#33720)
- Fixed: Error handling when launching a UAC task in case of a successfully monitored message. (#33922)
ue-aws-sqs-monitor-1.1.0 (2022-09-15)
Enhancements
Added
: Allow AWS Credentials and AWS Region as optional fields enabling their configuration on the task execution environment. (#28271)Added
: Allow more data in the STDERR (Printing the name of the launched task, message id and task instance sys_id ). (#30006)Added
: Task status goes to fail if the proxy can not be reached. (#30066)Added
: Task status goes to fail if UAC conectivity error. (#30049)Added
: Task status goes to fail if lauch task does not exist. (#30006)
ue-aws-sqs-monitor-1.0.0 (2022-02-03)
Initial Version