...
This Universal Extension provides the capability to send an AWS SQS message towards an existing queue.
Version Information
| Template Name | Extension Name | Extension Version |
|---|---|---|
| Amazon SQS Message | ue-aws-sqs-message | 1.1.1 |
Refer to Changelog for version history information.
Software Requirements
This integration requires a Universal Agent and a Python runtime to execute the Universal Task.
Software Requirements for Universal Template and Universal Task
- Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.
...
- Universal Controller Version 7.0.0.0 and later.
Key Features
This Universal Extension provides the following main features:
...
Network and Connectivity Requirements
Extension's Universal Agent host should be able to reach AWS SQS Queue Message REST endpoints. The AWS Credentials provided in the Amazon SQS Queue Message Universal Task, should have sufficient permissions on AWS to send messages.
Key Features
- Actions
- Send an AWS SQS message towards a
...
- Standard or a FIFO queue.
- Authentication
- AWS Credentials.
...
- IAM Role-Based Access Control (RBAC) strategy.
...
- Input/Output
- Capability to Print the JSON Response .
- Other
- Communication through Proxy with use of HTTP or HTTPS.
Import Universal Template
To use the Universal Template, you first must perform the following steps:.
This Universal Task requires the
...
Resolvable Credentials feature. Check that the
...
Resolvable Credentials Permitted system property has been set to true.
...
Download the provided ZIP file.
...
In the Universal Controller UI, select Administration >Configuration > Universal Templates to display the current list of Universal Templates.
...
Click Import Template.
To import the Universal Template into your Controller, follow the instructions here.
When the
...
files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list
...
.
Configure Universal Task
For the a new Universal Task type, create a new task, and enter the task-specific details that were created in the Universal Templaterequired input fields.
Input Fields
The input fields for this Universal Extension are described belowin the following table.
Field | Input Type | Default Value | Type | Description |
|---|---|---|---|---|
Action | Required | Send Message | Choice | The action performed upon the task execution. |
AWS Region |
Optional since version 1.1.0 | Optional | - | Text | Region for the Amazon Web Service. Find more information about the AWS Service endpoints and quotas here. |
Credentials
Required
-
Credentials
The AWS account credentials.
They are comprised of:
AWS access key ID
AWS secret access key
Queue Name
Required
-
Dynamic Choice
Dynamic fetched list of queue names. The user can select the required queue name from a drop-down list.
Queue Name Prefix
Optional
-
Text
A prefix to use for filtering the Queue Names list results. Only those queues whose name begins with the specified string are returned.
Queue Names are case-sensitive.
Role Based Access
Optional
False
Boolean
Flag to configure whether the type of authorization is based on IAM Role-Based Access Control(RBAC) strategy.
Find more information about the IAM RBAC authorization model here.
Message Body Source
Required
Raw
Choice
Type of message to be sent.
Available options are:
Raw
Script
Message Attributes
Optional
-
Script
The message attributes to send alongside the message body. Each message attribute consists of a Name, Type, and Value. It should be in JSON format and can have up to 10 attributes.
Example of Attribute Script with 2 Attributes:
{
"CustomerFirstname": {
"DataType": "String",
"StringValue": "John"
},
"CustomerLastname": {
"DataType": "String",
"StringValue": "Doe"
}
}
Message Body
Optional
-
Large Text
The message to send to queue.
A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed:
#x9, #xA, #xD, #x20 to #xD7FF, #xE000 to #xFFFD, #x10000 to #x10FFFF
Any characters not included in this list will be rejected.
For more information, see the W3C specification for characters.
Required when Message Body Source is configured for "Raw"
Message Body Script
Optional
-
When AWS Region is not populated as part of the task definition, during task execution the integration will look for AWS Region on the task execution environment. Refer to configuration options for more information. | ||||
AWS Credentials Optional since version 1.1.0 | Optional | - | Credentials | The Credentials definition should be as follows.
When AWS Credentials are not populated as part of the task definition, during task execution the integration will look for AWS Credentials on the task execution environment. Refer to configuration options for more information. |
Queue Name | Required | - | Dynamic Choice | Dynamic fetched list of queue names. The user can select the required queue name from a drop-down list. |
Queue Name Prefix | Optional | - | Text |
The Role Amazon Resource Name (ARN) to have access to the SQS queue.
Role ARN format: arn:aws:iam::<AWS Account ID>:instance-profile/<Role name>
Required when Role Based Access has been clicked.
A prefix to use for filtering the Queue Names list results. Only those queues whose name begins with the specified string are returned. Queue Names are case-sensitive. | ||||
Role Based Access | Optional | False | Boolean | Special type of authorization is provided by Role Assumption where the client sends his own credentials and the role he wants to assume from another user. If allowed, the client receives temporary credentials with limited time access to some resources. |
Role ARN | Optional | - | Text | Role Arn: Amazon Role, which is applied for the connection. Role ARN format: |
Message Body Source | Required | Raw | Choice | Type of message to be sent. Available options are:
|
Message Attributes | Optional | - | Script | The message attributes to send alongside the message body. Each message attribute consists of a Name, Type, and Value. It should be in JSON format and can have up to 10 attributes. Example of Attribute Script with 2 Attributes:
|
Message Body | Optional | - | Large Text | The message to send to queue. |
Required when Message Body
A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed:
Any characters not included in this list will be rejected. For more information, see the W3C specification for characters. Required when Message Body Source is configured for " |
Raw" |
Is Queue Fifo
Required
False
Boolean
Message Body Script | Optional | - | Script | The message to send to queue. The minimum message size is 1 byte (1 character). The maximum is 262,144 bytes (256 KB). Required when Message Body Source is configured for "Script". |
Is Queue Fifo | Required | False | Boolean | Flag to configure whether the message will be sent to a FIFO Queue. It should be enabled, if the message will be sent to a FIFO Queue. |
Delay Seconds | Optional | - | Integer | The length of time, in seconds, for which to delay a specific message. Messages with a positive value become available for processing after the delay period is finished. Valid values from 0 to 900. Used when Is Queue Fifo is disabled. |
Message Deduplication ID | Optional | - | Text | Specifies ID for deduplication of sent messages. If a Message Deduplication ID is not provided and the queue doesn't have Content Based Deduplication set on Queue, an error will be raised. Maximum Length is 128 characters. Used when Is Queue Fifo is enabled. |
Message Group ID | Optional | - | Text | Group ID that specifies that a message belongs to a specific message group. Maximum Length is 128 characters. Required when Is Queue Fifo is enabled. |
Use Proxy | Required | False | Boolean | Flag to configure whether Proxy should be used in communication. |
Proxy Type | Optional |
HTTP | Choice | Type of proxy connection to be used. Available options are |
the following. |
|
https
https with password
Visible only when Use Proxy |
= "True". | |||
Proxy | Optional | - | Text |
Comma separated list of Proxy servers. Valid formats are the following. |
|
checked. | ||||
Proxy CA Bundle File | Optional | - | Text | The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy. Used when Proxy Type is configured for " |
HTTPS" or " |
username
HTTPS With Credentials". | ||||
Proxy Credentials | Optional | - | Credentials | Credentials to be used for the proxy communication. |
The credential definition should be as follows.
Required when Proxy Type is configured for " |
HTTPS With Credentials". |
Field Descriptions for AWS SQS Universal Task - Actions
The AWS SQS Task provides three different Actions.
receive-message
list-queues
send-message
For each action, the specific fields are described and an example is provided.
list-queues - Action
The Action list the available SQS queues for the given AWS Account and Role ARN ( optional ).
...
Action
...
list-queues action
...
AWS_ACCESS_KEY_ID
...
AWS Account credentials Access Key
...
AWS_DEFAULT_REGION
...
AWS Account Region
...
AWS_SECRET_ACCESS_KEY
...
AWS Account credentials Secret Access Key
...
Useproxy ( default is NO )
...
[NO | YES]
If set to YES, the fields to set-up the proxy server connections are displayed:
Proxy Server IP or hostname
Proxy Server Port
Proxy Server Credentials (optional)
...
Queue Name Prefix
...
A string to use for filtering the list results. Only those queues whose name begins with the specified string are returned.
Queue URLs and names are case-sensitive.
e.g. Queue Name Prefix = orders , will list all Queues starting with the name orders
...
Loglevel ( default is INFO )
...
Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]
...
Role Based Access (STS)
...
[NO | YES]
If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
Role Arn: Amazon Role, which is applied for the connection
Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.
...
Endpoint URL
...
Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system
Examples for AWS SQS Universal Tasks - Action: list-queues
List-Queues - Action
Send-message - Action
This Action inserts a message into the given AWS SQS queue. The message Body and Attributes are configurable.
...
Action
...
send-message action
...
AWS_ACCESS_KEY_ID
...
AWS Account credentials Access Key
...
AWS_DEFAULT_REGION
...
AWS Account Region
...
AWS_SECRET_ACCESS_KEY
...
AWS Account credentials Secret Access Key
...
Useproxy ( default is NO )
...
[NO | YES]
If set to YES, the fields to set-up the proxy server connections are displayed:
Proxy Server IP or hostname
Proxy Server Port
Proxy Server Credentials (optional)
...
SQS Queue Name
...
Name of the SQS Message queue Queue names are case-sensitive.
...
Message Body
...
SQS Message Body
The message must contain the parameter Message Body. Parameter is of Type string.
Example: {"Category":"Books"}
...
Attribute Script
...
The Attribute Script is a script in json format, which is saved in the Controller script library.
Each message attribute consists of a Name, Type, and Value.
The message attributes can be provided using JSON format.
Example of Attribute Script with 2 Attributes:
{
"CustomerFirstname": {
"DataType": "String",
"StringValue": "Nils"
},
"CustomerLastname": {
"DataType": "String",
"StringValue": "Buer"
}
}Amazon SQS lets you include structured metadata (such as timestamps, geospatial data, signatures, and identifiers) with messages using message attributes. Each message can have up to 10 attributes. Message attributes are optional and separate from the message body (however, they are sent alongside it).
...
Delay Seconds
...
The length of time, in seconds, for which to delay a specific message. Valid values: 0 to 900. Maximum: 15 minutes. Messages with a positive DelaySeconds value become available for processing after the delay period is finished. If you don't specify a value, the default value for the queue applies.
...
Loglevel ( default is INFO )
...
Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]
...
Role Based Access (STS)
...
[NO | YES]
If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
Role Arn: Amazon Role, which is applied for the connection
Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.
...
Endpoint URL
...
Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system
Examples for AWS SQS Universal Tasks - Action: send-message
Send-message - Action
Attribute Script Example
Script Example: orders.json
...
Task Examples
Send Message to Standard Queue
Example of Universal Task for sending a message towards a standard queue with minimum parameters.
Send Message to FIFO Queue with all optional input arguments
Example of Universal Task for sending a message via script towards a FIFO queue, with all optional input argument.
Send Message with Environment Variables as Region
Example of Universal Task for sending a message to StandardQueue providing no AWS Credentials in task definition and providing AWS Region as Environment Variable, leaving the respective input fields empty. AWS Credentials are expected in this case to be configured on the task execution environment. Please refer to AWS Credentials input field for more information.
Task Output
Exit Codes
The exit codes for this Universal Extension are described below.
Exit Code | Status Classification Code | Status Classification Description | Status Description |
|---|---|---|---|
0 | SUCCESS | Successful Execution | SUCCESS: Successful Task execution |
1 | FAIL | Failed Execution | FAIL: < Error Description > |
3 | AUTHORIZATION_ERROR | Insufficient Permissions | AUTHORIZATION_ERROR: The authorization credentials provided for the request are invalid. |
10 | CONNECTION_ERROR | Bad connection data or connection timed out | CONNECTION_ERROR: < Error Description > |
20 | DATA_VALIDATION_ERROR | Bad input fields validation | DATA_VALIDATION_ERROR: Some of the input fields cannot be validated. See STDERR for more details. |
Extension Output
Upon Task's successful completion, the extension always produces an Extension Output.
This Universal Extension sends a message towards a Standard or FIFO queue. When the message is sent successfully, related information is published as Extension Output.
Attribute changed is populated as follows.
- true in case the message was sent successfully.
An example of the Extension Output for a successful triggering job is presented below.
| Attribute | Type | Description |
|---|---|---|
| MD5OfMessageBody | string | MD5 digest of the non-URL-encoded message body string. |
| MD5OfMessageAttributes | string | MD5 digest of the non-URL-encoded message attribute string. |
| MessageId | string | Attribute containing the MessageId of the message sent to the queue. |
| SequenceNumber | string | Non-consecutive number that Amazon SQS assigns to each message of FIFO queues. |
| ResponseMetadata | json | Response Metadata of the REST API. |
The following is an example of Extension Output for this Universal Extension.
| Code Block | ||||
|---|---|---|---|---|
| ||||
{ "exit_code": 0, "status_description": "SUCCESS: Task executed successfully.", "changed": true, "invocation": { "extension": "ue-aws-sqs-message", "version": "1.1.0" , "fields": { "action": "Send Message", "credentials_user": "AWSUSER123456789ABCDE", "credentials_password": "********", "region": "us-east-2", "queue_name": "StandardQueue", "queue_name_prefix": "", "role_based_access": true, "role_arn": "arn:aws:iam::123456789:role/arn_test", "message_body_source": "raw", "message_body": "This is a SQS Message", "message_body_script": null, " |
...
message_attributes": |
...
{ |
...
|
...
" |
...
CustomerFirstname": { |
...
|
...
" |
...
DataType": " |
...
String", |
...
" |
...
StringValue": |
...
"John" |
...
|
...
|
...
|
...
} |
...
, |
...
" |
...
CustomerLastname": { |
...
" |
...
DataType": " |
...
String", |
...
" |
...
StringValue": |
...
"Doe" |
...
|
...
|
...
} }, |
...
|
...
|
...
"is_queue_fifo": |
...
false, |
...
|
...
"message_deduplication_id": null, |
...
" |
...
message_group_id": |
...
null, " |
...
delay_seconds": |
...
0, " |
...
use_proxy": |
...
false, |
...
"proxy_type": null, |
...
|
...
" |
...
proxy_credentials_user": |
...
null, |
...
"proxy_credentials_password": |
...
null, |
...
"proxy": "", |
...
|
...
" |
...
proxy_ca_bundle_file": |
...
null |
...
|
...
} |
...
}, "result": |
...
Receive-message - Action
The Action polls in configurable interval the provided SQS queue. If a message is found a Task can be launched ( optional). Optionally it be configured after a message is received the task goes to success and does not continue to poll for new messages.
...
Action
...
receive-message action
...
AWS_ACCESS_KEY_ID
...
AWS Account credentials Access Key
...
AWS_DEFAULT_REGION
...
AWS Account Region
...
AWS_SECRET_ACCESS_KEY
...
AWS Account credentials Secret Access Key
...
Useproxy ( default is NO )
...
[NO | YES]
If set to YES, the fields to set-up the proxy server connections are displayed:
Proxy Server IP or hostname
Proxy Server Port
Proxy Server Credentials (optional)
...
Universal Controller URL
...
Universal Controller URL
URL has no backslash “/” at the end.
Example: http://192.168.88.10:8080/uc
...
Universal Controller Credentials
...
Universal Controller Credentials
The Credentials need to have “Web Service Access” Permissions
Attribute Names
...
[All|Policy|VisibilityTimeout|MaximumMessageSize|MessageRetentionPeriod|ApproximateNumberOfMessages|ApproximateNumberOfMessagesNotVisible|CreatedTimestamp|LastModifiedTimestamp|QueueArn|ApproximateNumberOfMessagesDelayed|DelaySeconds|ReceiveMessageWaitTimeSeconds|RedrivePolicy|FifoQueue|ContentBasedDeduplication|KmsMasterKeyId|KmsDataKeyReusePeriodSeconds]
For details on the attributes refer to: AWS-SQS-CLI-AttributeNames
...
MessageAttributeNames
...
[ALL | list of attributes in CSV format]
Message Attributes to be returned
‘ALL' returns all attributes of the message
Example: MessageAttributeNames = Author, Title will return only the attribute for Author and Title.
...
WaitTimeSeconds
...
The duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than WaitTimeSeconds . If no messages are available and the wait time expires, the call returns successfully with an empty list of messages.
...
Launch Task
...
Taskname of the Task to launch in Universal Controller.
The task will be always launched with two Task Variables:
{Body}: contains the message body in json format
{Attributes}: contains the message attributes in json format.
The Variable content can be used in further processing.
e.g. If a Linux task is launched by the AWS SQS Task than an echo {Attributes} command executed by the Linux task will print out all message attributes in json format to STDOUT.
...
Delete Messages from queue after reading
...
[ Yes | No ]
If “Yes” the received Messages is deleted from the queue after reading
...
Interval in seconds
...
Message Polling Interval in seconds
...
Run Mode
...
[Run Once | Run Forever]
Run once : if a message is found the task goes to success
Run Forever : each time a new message is found the configured task is launched
Example:
Run once can be used in Workflows
Run Forever can be used as Standalone Task, where each message triggers a task launch.
...
MaxNumberOfMessages
...
The maximum number of messages to return. Amazon SQS never returns more messages than this value (however, fewer messages might be returned). Valid values: 1 to 10. Default: 1.
...
Loglevel ( default is INFO )
...
Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]
...
Role Based Access (STS)
...
[NO | YES]
If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
Role Arn: Amazon Role, which is applied for the connection
Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.
...
Endpoint URL
...
Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system
Examples for AWS SQS Universal Tasks - Action: receive-message
Receive-message - Action
...
{
"MD5OfMessageBody": "c6d58fdf00486074745c40334ab44780",
"MD5OfMessageAttributes": null,
"MessageId": "f65bc887-0376-42c3-b354-85749c6d1449",
"SequenceNumber": null,
"ResponseMetadata": {
"RequestId": "3f579b52-6995-5c56-83dd-c14b19cbc99b",
"HTTPStatusCode": 200,
"HTTPHeaders": {
"x-amzn-requestid": "3f579b52-6995-5c56-83dd-c14b19cbc99b",
"date": "Fri, 26 Nov 2021 11:06:06 GMT",
"content-type": "text/xml",
"content-length": "378"
},
"RetryAttempts": 0
}
}
} |
The result section, as displayed above, includes information of the dispatched SQS message, providing information about the message and response metadata.
Universal Controller provides Output Functions that can be used to resolve the JSON Extension Output for further use (for example, by a sibling task).
STDOUT and STDERR
STDOUT and STDERR provide additional information to User. The populated content can be changed in future versions of this extension without notice. Backward compatibility is not guaranteed.
Document References
This document references the following documents:
| Document Link | Description |
|---|---|
| Universal Templates | User documentation for creating, working with and understanding Universal Templates and Integrations. |
| Universal Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. |
| Amazon SQS Message | User documentation for Amazon SQS. |
| IAM RBAC authorization model | User Documentation for Comparing ABAC to the traditional RBAC model. |
| Anchor | ||||
|---|---|---|---|---|
|
Changelog
ue-aws-sqs-message-1.1.1 (2023-08-09)
Fixes
- Fixed: Field 'Proxy Type' raised a data validation error on Controller 7.3.0.0 and later. (#33871)
ue-aws-sqs-message-1.1.0 (2022-06-30)
Enhancements
Added: Allow AWS Credentials and AWS Region as optional fields enabling their configuration on the task execution environment. (#28251)
ue-aws-sqs-message-1.0.0 (2022-02-11)
Initial Version