...
Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/
...
Overview
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
Overview
Using SQS, you can send, store, and receive messages between software components.
The Universal Task for SQS allows you to create, send and monitor AWS SQS messages and automatically trigger a Task in Universal Controller once a message has been received.
Software Requirements
Software Requirements Universal Agents and Controller
Universal Agent for Linux or Windows Version 6.9.0.0 or later are required
Software Requirements Universal Controller
Universal Controller 6.9.0.0. or later is required
A Universal Controller license key with support for SAP connector is required
Software Requirements for the Application to be Scheduled
In order to connect to the SAP System the SAP NetWeaver RFC SDK 7.50 libraries are required from SAP.
Those can be downloaded from the SAP Software Download: SAP NetWeaver RFC SDK 7.50
Universal Task for AWS SQS Key Features
Some details about the Universal Tasks for AWS SQS:
...
The Universal Tasks for SQS allows you to create, send and monitor for new AWS SQS messages.
...
The Universal Task for SQS can trigger a Task in Universal Controller upon each arrival of a new message
...
Credentials for AWS S3 are stored in an encrypted format in the database
This Universal Extension provides the capability to send an AWS SQS message towards an existing queue.
Version Information
| Template Name | Extension Name | Extension Version |
|---|---|---|
| Amazon SQS Message | ue-aws-sqs-message | 1.1.1 |
Refer to Changelog for version history information.
Software Requirements
This integration requires a Universal Agent and a Python runtime to execute the Universal Task.
Software Requirements for Universal Template and Universal Task
- Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.
Software Requirements Universal Agent
Both Windows and Linux agents are supported:
Universal Agent for Windows x64 Version 7.1.0.0 and later.
Universal Agent for Linux Version 7.1.0.0 and later.
Software Requirements Universal Controller
- Universal Controller Version 7.0.0.0 and later.
Network and Connectivity Requirements
Extension's Universal Agent host should be able to reach AWS SQS Queue Message REST endpoints. The AWS Credentials provided in the Amazon SQS Queue Message Universal Task, should have sufficient permissions on AWS to send messages.
Key Features
- Actions
- Send an AWS SQS message towards a Standard or a FIFO queue.
- Authentication
- AWS Credentials.
- IAM Role-Based Access Control (RBAC)
...
Communication to AWS is done via the HTTPS protocol
...
A Proxy Server connection to AWS with basic authentication is supported
...
You can create and send a SQS message out of any modern application by calling the Universal Controller Remote Webservice API
...
The new SQS task can be integrated into any existing or new automation workflow like any other task.
...
The Universal template script is using the Python Boto3 Module. This allows to quickly introduce new AWS Service and to update the current SQS Task when new requirements occur.
...
You can set different log-levels for the Universal task, providing you more information in case of issues
Import AWS SQS Virtual Machine Downloadable Universal Template
To use this downloadable Universal Template, you first must perform the following steps:
This Universal Task requires the /wiki/spaces/UC71x/pages/5178443 feature. Check that the/wiki/spaces/UC71x/pages/5177877 system property has been set to true.
Copy or Transfer the Universal Template file to a directory that can be accessed by the Universal Controller Tomcat user.
In the Universal Controller UI, select Configuration > Universal Templates to display the current list of /wiki/spaces/UC71x/pages/5178054.
Right-click any column header on the list to display an Action menu.
Select Import from the menu, enter the directory containing the Universal Template file(s) that you want to import, and click OK.
When the files have been imported successfully, the Universal Template will appear on the list.
Configure AWS SQS Universal Tasks
For the new Universal Task type AWS SQS, create a new task and enter the task-specific Details that were created in the Universal Template.
Field Descriptions for AWS SQS Universal Task - Actions
The AWS SQS Task provides three different Actions.
receive-message
list-queues
send-message
For each action, the specific fields are described and an example is provided.
list-queues - Action
The Action list the available SQS queues for the given AWS Account and Role ARN ( optional ).
...
Action
...
list-queues action
...
AWS_ACCESS_KEY_ID
...
AWS Account credentials Access Key
...
AWS_DEFAULT_REGION
...
AWS Account Region
...
AWS_SECRET_ACCESS_KEY
...
AWS Account credentials Secret Access Key
...
Useproxy ( default is NO )
...
[NO | YES]
If set to YES, the fields to set-up the proxy server connections are displayed:
Proxy Server IP or hostname
Proxy Server Port
Proxy Server Credentials (optional)
...
Queue Name Prefix
...
A string to use for filtering the list results. Only those queues whose name begins with the specified string are returned.
Queue URLs and names are case-sensitive.
e.g. Queue Name Prefix = orders , will list all Queues starting with the name orders
...
Loglevel ( default is INFO )
...
Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]
...
Role Based Access (STS)
[NO | YES]
...
- strategy.
- Input/Output
- Capability to Print the JSON Response .
- Other
- Communication through Proxy with use of HTTP or HTTPS.
Import Universal Template
To use the Universal Template, you first must perform the following steps.
This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.
To import the Universal Template into your Controller, follow the instructions here.
When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.
Configure Universal Task
For a new Universal Task, create a new task, and enter the required input fields.
Input Fields
The input fields for this Universal Extension are described in the following table.
Field | Input Type | Default Value | Type | Description |
|---|---|---|---|---|
Action | Required | Send Message | Choice | The action performed upon the task execution. |
AWS Region Optional since version 1.1.0 | Optional | - | Text | Region for the Amazon Web Service. Find more information about the AWS Service endpoints and quotas here. When AWS Region is not populated as part of the task definition, during task execution the integration will look for AWS Region on the task execution environment. Refer to configuration options for more information. |
AWS Credentials Optional since version 1.1.0 | Optional | - | Credentials | The Credentials definition should be as follows.
When AWS Credentials are not populated as part of the task definition, during task execution the integration will look for AWS Credentials on the task execution environment. Refer to configuration options for more information. |
Queue Name | Required | - | Dynamic Choice | Dynamic fetched list of queue names. The user can select the required queue name from a drop-down list. |
Queue Name Prefix | Optional | - | Text | A prefix to use for filtering the Queue Names list results. Only those queues whose name begins with the specified string are returned. Queue Names are case-sensitive. |
Role Based Access | Optional | False | Boolean | Special type of authorization is provided by Role Assumption where the client sends his own credentials and the role he wants to assume from another user. If allowed, the client receives temporary credentials with limited time access to some resources. |
Role ARN | Optional | - | Text | Role Arn: Amazon Role, which is applied for the connection. Example Role ARN format: STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues. |
Endpoint URL | Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system |
Examples for AWS SQS Universal Tasks - Action: list-queues
List-Queues - Action
Send-message - Action
This Action inserts a message into the given AWS SQS queue. The message Body and Attributes are configurable.
...
Action
...
send-message action
...
AWS_ACCESS_KEY_ID
...
AWS Account credentials Access Key
...
AWS_DEFAULT_REGION
...
AWS Account Region
...
AWS_SECRET_ACCESS_KEY
...
AWS Account credentials Secret Access Key
...
Useproxy ( default is NO )
...
[NO | YES]
If set to YES, the fields to set-up the proxy server connections are displayed:
Proxy Server IP or hostname
Proxy Server Port
Proxy Server Credentials (optional)
...
SQS Queue Name
...
Name of the SQS Message queue Queue names are case-sensitive.
...
Message Body
...
SQS Message Body
The message must contain the parameter Message Body. Parameter is of Type string.
Example: {"Category":"Books"}
...
Attribute Script
...
The Attribute Script is a script in json format, which is saved in the Controller script library.
Each message attribute consists of a Name, Type, and Value.
The message attributes can be provided using JSON format.
Example of Attribute Script with 2 Attributes:
{
"CustomerFirstname": {
"DataType": "String",
"StringValue": "Nils"
},
"CustomerLastname": {
"DataType": "String",
"StringValue": "Buer"
}
}Amazon SQS lets you include structured metadata (such as timestamps, geospatial data, signatures, and identifiers) with messages using message attributes. Each message can have up to 10 attributes. Message attributes are optional and separate from the message body (however, they are sent alongside it).
...
Delay Seconds
...
The length of time, in seconds, for which to delay a specific message. Valid values: 0 to 900. Maximum: 15 minutes. Messages with a positive DelaySeconds value become available for processing after the delay period is finished. If you don't specify a value, the default value for the queue applies.
...
Loglevel ( default is INFO )
...
Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]
...
Role Based Access (STS)
...
[NO | YES]
If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
Role Arn: Amazon Role, which is applied for the connection
Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.
...
Endpoint URL
...
Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system
Examples for AWS SQS Universal Tasks - Action: send-message
Send-message - Action
Attribute Script Example
Script Example: orders.json
...
. | ||||
Message Body Source | Required | Raw | Choice | Type of message to be sent. Available options are:
|
Message Attributes | Optional | - | Script | The message attributes to send alongside the message body. Each message attribute consists of a Name, Type, and Value. It should be in JSON format and can have up to 10 attributes. Example of Attribute Script with 2 Attributes:
|
Message Body | Optional | - | Large Text | The message to send to queue. A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed:
Any characters not included in this list will be rejected. For more information, see the W3C specification for characters. Required when Message Body Source is configured for "Raw" |
Message Body Script | Optional | - | Script | The message to send to queue. The minimum message size is 1 byte (1 character). The maximum is 262,144 bytes (256 KB). Required when Message Body Source is configured for "Script". |
Is Queue Fifo | Required | False | Boolean | Flag to configure whether the message will be sent to a FIFO Queue. It should be enabled, if the message will be sent to a FIFO Queue. |
Delay Seconds | Optional | - | Integer | The length of time, in seconds, for which to delay a specific message. Messages with a positive value become available for processing after the delay period is finished. Valid values from 0 to 900. Used when Is Queue Fifo is disabled. |
Message Deduplication ID | Optional | - | Text | Specifies ID for deduplication of sent messages. If a Message Deduplication ID is not provided and the queue doesn't have Content Based Deduplication set on Queue, an error will be raised. Maximum Length is 128 characters. Used when Is Queue Fifo is enabled. |
Message Group ID | Optional | - | Text | Group ID that specifies that a message belongs to a specific message group. Maximum Length is 128 characters. Required when Is Queue Fifo is enabled. |
Use Proxy | Required | False | Boolean | Flag to configure whether Proxy should be used in communication. |
Proxy Type | Optional | HTTP | Choice | Type of proxy connection to be used. Available options are the following.
Visible only when Use Proxy = "True". |
Proxy | Optional | - | Text | Comma separated list of Proxy servers. Valid formats are the following. |
Proxy CA Bundle File | Optional | - | Text | The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy. Used when Proxy Type is configured for "HTTPS" or "HTTPS With Credentials". |
Proxy Credentials | Optional | - | Credentials | Credentials to be used for the proxy communication. The credential definition should be as follows.
Required when Proxy Type is configured for "HTTPS With Credentials". |
Task Examples
Send Message to Standard Queue
Example of Universal Task for sending a message towards a standard queue with minimum parameters.
Send Message to FIFO Queue with all optional input arguments
Example of Universal Task for sending a message via script towards a FIFO queue, with all optional input argument.
Send Message with Environment Variables as Region
Example of Universal Task for sending a message to StandardQueue providing no AWS Credentials in task definition and providing AWS Region as Environment Variable, leaving the respective input fields empty. AWS Credentials are expected in this case to be configured on the task execution environment. Please refer to AWS Credentials input field for more information.
Task Output
Exit Codes
The exit codes for this Universal Extension are described below.
Exit Code | Status Classification Code | Status Classification Description | Status Description |
|---|---|---|---|
0 | SUCCESS | Successful Execution | SUCCESS: Successful Task execution |
1 | FAIL | Failed Execution | FAIL: < Error Description > |
3 | AUTHORIZATION_ERROR | Insufficient Permissions | AUTHORIZATION_ERROR: The authorization credentials provided for the request are invalid. |
10 | CONNECTION_ERROR | Bad connection data or connection timed out | CONNECTION_ERROR: < Error Description > |
20 | DATA_VALIDATION_ERROR | Bad input fields validation | DATA_VALIDATION_ERROR: Some of the input fields cannot be validated. See STDERR for more details. |
Extension Output
Upon Task's successful completion, the extension always produces an Extension Output.
This Universal Extension sends a message towards a Standard or FIFO queue. When the message is sent successfully, related information is published as Extension Output.
Attribute changed is populated as follows.
- true in case the message was sent successfully.
An example of the Extension Output for a successful triggering job is presented below.
| Attribute | Type | Description |
|---|---|---|
| MD5OfMessageBody | string | MD5 digest of the non-URL-encoded message body string. |
| MD5OfMessageAttributes | string | MD5 digest of the non-URL-encoded message attribute string. |
| MessageId | string | Attribute containing the MessageId of the message sent to the queue. |
| SequenceNumber | string | Non-consecutive number that Amazon SQS assigns to each message of FIFO queues. |
| ResponseMetadata | json | Response Metadata of the REST API. |
The following is an example of Extension Output for this Universal Extension.
| Code Block | ||||
|---|---|---|---|---|
| ||||
{ "exit_code": 0, "status_description": "SUCCESS: Task executed successfully.", "changed": true, "invocation": { "extension": "ue-aws-sqs-message", " |
...
version": " |
...
1.1.0" , " |
...
fields": |
...
{ "action": "Send Message", |
...
Receive-message - Action
The Action polls in configurable interval the provided SQS queue. If a message is found a Task can be launched ( optional). Optionally it be configured after a message is received the task goes to success and does not continue to poll for new messages.
...
Action
...
receive-message action
...
AWS_ACCESS_KEY_ID
...
AWS Account credentials Access Key
...
AWS_DEFAULT_REGION
...
AWS Account Region
...
AWS_SECRET_ACCESS_KEY
...
AWS Account credentials Secret Access Key
...
Useproxy ( default is NO )
...
[NO | YES]
If set to YES, the fields to set-up the proxy server connections are displayed:
Proxy Server IP or hostname
Proxy Server Port
Proxy Server Credentials (optional)
...
Universal Controller URL
...
Universal Controller URL
URL has no backslash “/” at the end.
Example: http://192.168.88.10:8080/uc
...
Universal Controller Credentials
...
Universal Controller Credentials
The Credentials need to have “Web Service Access” Permissions
Attribute Names
...
[All|Policy|VisibilityTimeout|MaximumMessageSize|MessageRetentionPeriod|ApproximateNumberOfMessages|ApproximateNumberOfMessagesNotVisible|CreatedTimestamp|LastModifiedTimestamp|QueueArn|ApproximateNumberOfMessagesDelayed|DelaySeconds|ReceiveMessageWaitTimeSeconds|RedrivePolicy|FifoQueue|ContentBasedDeduplication|KmsMasterKeyId|KmsDataKeyReusePeriodSeconds]
For details on the attributes refer to: AWS-SQS-CLI-AttributeNames
...
MessageAttributeNames
...
[ALL | list of attributes in CSV format]
Message Attributes to be returned
‘ALL' returns all attributes of the message
Example: MessageAttributeNames = Author, Title will return only the attribute for Author and Title.
...
WaitTimeSeconds
...
The duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than WaitTimeSeconds . If no messages are available and the wait time expires, the call returns successfully with an empty list of messages.
...
Launch Task
...
Taskname of the Task to launch in Universal Controller.
The task will be always launched with two Task Variables:
{Body}: contains the message body in json format
{Attributes}: contains the message attributes in json format.
The Variable content can be used in further processing.
e.g. If a Linux task is launched by the AWS SQS Task than an echo {Attributes} command executed by the Linux task will print out all message attributes in json format to STDOUT.
...
Delete Messages from queue after reading
...
[ Yes | No ]
If “Yes” the received Messages is deleted from the queue after reading
...
Interval in seconds
...
Message Polling Interval in seconds
...
Run Mode
...
[Run Once | Run Forever]
Run once : if a message is found the task goes to success
Run Forever : each time a new message is found the configured task is launched
Example:
Run once can be used in Workflows
Run Forever can be used as Standalone Task, where each message triggers a task launch.
...
MaxNumberOfMessages
...
The maximum number of messages to return. Amazon SQS never returns more messages than this value (however, fewer messages might be returned). Valid values: 1 to 10. Default: 1.
...
Loglevel ( default is INFO )
...
Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]
...
Role Based Access (STS)
...
[NO | YES]
If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
Role Arn: Amazon Role, which is applied for the connection
Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.
...
Endpoint URL
...
Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system
Examples for AWS SQS Universal Tasks - Action: receive-message
Receive-message - Action
...
"credentials_user": "AWSUSER123456789ABCDE",
"credentials_password": "********",
"region": "us-east-2",
"queue_name": "StandardQueue",
"queue_name_prefix": "",
"role_based_access": true,
"role_arn": "arn:aws:iam::123456789:role/arn_test",
"message_body_source": "raw",
"message_body": "This is a SQS Message",
"message_body_script": null,
"message_attributes": {
"CustomerFirstname": {
"DataType": "String",
"StringValue": "John"
},
"CustomerLastname": {
"DataType": "String",
"StringValue": "Doe"
}
},
"is_queue_fifo": false,
"message_deduplication_id": null,
"message_group_id": null,
"delay_seconds": 0,
"use_proxy": false,
"proxy_type": null,
"proxy_credentials_user": null,
"proxy_credentials_password": null,
"proxy": "",
"proxy_ca_bundle_file": null
}
},
"result": {
"MD5OfMessageBody": "c6d58fdf00486074745c40334ab44780",
"MD5OfMessageAttributes": null,
"MessageId": "f65bc887-0376-42c3-b354-85749c6d1449",
"SequenceNumber": null,
"ResponseMetadata": {
"RequestId": "3f579b52-6995-5c56-83dd-c14b19cbc99b",
"HTTPStatusCode": 200,
"HTTPHeaders": {
"x-amzn-requestid": "3f579b52-6995-5c56-83dd-c14b19cbc99b",
"date": "Fri, 26 Nov 2021 11:06:06 GMT",
"content-type": "text/xml",
"content-length": "378"
},
"RetryAttempts": 0
}
}
} |
The result section, as displayed above, includes information of the dispatched SQS message, providing information about the message and response metadata.
Universal Controller provides Output Functions that can be used to resolve the JSON Extension Output for further use (for example, by a sibling task).
STDOUT and STDERR
STDOUT and STDERR provide additional information to User. The populated content can be changed in future versions of this extension without notice. Backward compatibility is not guaranteed.
Document References
This document references the following documents:
| Document Link | Description |
|---|---|
| Universal Templates | User documentation for creating, working with and understanding Universal Templates and Integrations. |
| Universal Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. |
| Amazon SQS Message | User documentation for Amazon SQS. |
| IAM RBAC authorization model | User Documentation for Comparing ABAC to the traditional RBAC model. |
| Anchor | ||||
|---|---|---|---|---|
|
Changelog
ue-aws-sqs-message-1.1.1 (2023-08-09)
Fixes
- Fixed: Field 'Proxy Type' raised a data validation error on Controller 7.3.0.0 and later. (#33871)
ue-aws-sqs-message-1.1.0 (2022-06-30)
Enhancements
Added: Allow AWS Credentials and AWS Region as optional fields enabling their configuration on the task execution environment. (#28251)
ue-aws-sqs-message-1.0.0 (2022-02-11)
Initial Version