Amazon SQS: Create, Monitor, and Send Messages

This integration has been sunset

It is replaced by:
- Amazon SQS: Message: for the functionalities related to sending messages.
- Amazon SQS: Monitor: for the functionalities related to monitoring messages.

Disclaimer

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/

Introduction

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.

Overview

Using SQS, you can send, store, and receive messages between software components.

The Universal Task for SQS allows you to create, send and monitor AWS SQS messages and automatically trigger a Task in Universal Controller once a message has been received.

Software Requirements

Software Requirements Universal Agents and Controller

  • Universal Agent for Linux or Windows Version 6.9.0.0 or later are required

Software Requirements Universal Controller

  • Universal Controller 6.9.0.0. or later is required

  • A Universal Controller license key with support for SAP connector is required

Software Requirements for the Application to be Scheduled

In order to connect to the SAP System the SAP NetWeaver RFC SDK 7.50 libraries are required from SAP.

Those can be downloaded from the SAP Software Download: SAP NetWeaver RFC SDK 7.50

Universal Task for AWS SQS Key Features

Some details about the Universal Tasks for AWS SQS:

  • The Universal Tasks for SQS allows you to create, send and monitor for new AWS SQS messages.

  • The Universal Task for SQS can trigger a Task in Universal Controller upon each arrival of a new message

  • Credentials for AWS S3 are stored in an encrypted format in the database

  • IAM Role-Based Access Control (RBAC) is supported

  • Communication to AWS is done via the HTTPS protocol

  • A Proxy Server connection to AWS with basic authentication is supported

  • You can create and send a SQS message out of any modern application by calling the Universal Controller Remote Webservice API

  • The new SQS task can be integrated into any existing or new automation workflow like any other task.

  • The Universal template script is using the Python Boto3 Module. This allows to quickly introduce new AWS Service and to update the current SQS Task when new requirements occur. 

  • You can set different log-levels for the Universal task, providing you more information in case of issues

Import AWS SQS Virtual Machine Downloadable Universal Template

To use this downloadable Universal Template, you first must perform the following steps:

  1. This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true
  2. To import the Universal Template into your Controller, follow the instructions here.
  3. When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.

Configure AWS SQS Universal Tasks

For the new Universal Task type AWS SQS, create a new task and enter the task-specific Details that were created in the Universal Template.

Field Descriptions for AWS SQS Universal Task - Actions

The AWS SQS Task provides three different Actions.

  • receive-message

  • list-queues

  • send-message

For each action, the specific fields are described and an example is provided.

list-queues - Action

The Action list the available SQS queues for the given AWS Account and Role ARN ( optional ).

FieldDescription

Action

list-queues action

AWS_ACCESS_KEY_ID

AWS Account credentials Access Key

AWS_DEFAULT_REGION

AWS Account Region

AWS_SECRET_ACCESS_KEY

AWS Account credentials Secret Access Key

Useproxy ( default is NO )

[NO | YES]

If set to YES, the fields to set-up the proxy server connections are displayed:

  • Proxy Server IP or hostname

  • Proxy Server Port

  • Proxy Server Credentials (optional)

Queue Name Prefix

A string to use for filtering the list results. Only those queues whose name begins with the specified string are returned.

Queue URLs and names are case-sensitive.

e.g. Queue Name Prefix = orders , will list all Queues starting with the name orders

Loglevel ( default is INFO )

Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]

Role Based Access (STS)

[NO | YES]

If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:

  • Role Arn: Amazon Role, which is applied for the connection

Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly

STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.

Endpoint URL

Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system


Examples for AWS SQS Universal Tasks - Action: list-queues

List-Queues - Action

Send-message - Action

This Action inserts a message into the given AWS SQS queue. The message Body and Attributes are configurable.

FieldDescription

Action

send-message action

AWS_ACCESS_KEY_ID

AWS Account credentials Access Key

AWS_DEFAULT_REGION

AWS Account Region

AWS_SECRET_ACCESS_KEY

AWS Account credentials Secret Access Key

Useproxy ( default is NO )

[NO | YES]

If set to YES, the fields to set-up the proxy server connections are displayed:

  • Proxy Server IP or hostname

  • Proxy Server Port

  • Proxy Server Credentials (optional)

SQS Queue Name

Name of the SQS Message queue Queue names are case-sensitive.

Message Body

SQS Message Body

The message must contain the parameter Message Body. Parameter is of Type string.

Example: {"Category":"Books"}

Attribute Script

The Attribute Script is a script in json format, which is saved in the Controller script library.

Each message attribute consists of a NameType, and Value.

The message attributes can be provided using JSON format.

Example of Attribute Script with 2 Attributes:

{
	"CustomerFirstname": {
		"DataType": "String",
		"StringValue": "Nils"
	},
	"CustomerLastname": {
		"DataType": "String",
		"StringValue": "Buer"
	}
}

Amazon SQS lets you include structured metadata (such as timestamps, geospatial data, signatures, and identifiers) with messages using message attributes. Each message can have up to 10 attributes. Message attributes are optional and separate from the message body (however, they are sent alongside it).

Delay Seconds

The length of time, in seconds, for which to delay a specific message. Valid values: 0 to 900. Maximum: 15 minutes. Messages with a positive DelaySeconds value become available for processing after the delay period is finished. If you don't specify a value, the default value for the queue applies.

Loglevel ( default is INFO )

Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]

Role Based Access (STS)

[NO | YES]

If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:

  • Role Arn: Amazon Role, which is applied for the connection

Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly

STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.

Endpoint URL

Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system

Examples for AWS SQS Universal Tasks - Action: send-message

Send-message - Action


Attribute Script Example


Script Example: orders.json

{
    "CustomerFirstname": {
        "DataType": "String",
        "StringValue": "${CustomerFirstname}"
           },
    "CustomerLastname": {
        "DataType": "String",
        "StringValue": "${CustomerLastname}"
           },
    "City": {
        "DataType": "String",
        "StringValue": "${City}"
           },
    "Street": {
        "DataType": "String",
        "StringValue": "${Street}"
           },
    "Zipcode": {
        "DataType": "String",
        "StringValue": "${Zipcode}"
           },
    "Title": {
        "DataType": "String",
        "StringValue": "${Title}"
           },
    "Author": {
        "DataType": "String",
        "StringValue": "${Author}"
           },
    "ISBN": {
        "DataType": "String",
        "StringValue": "${ISBN}"
           },
    "ID": {
        "DataType": "String",
        "StringValue": "${ID}"
           }
}

Receive-message - Action

The Action polls in configurable interval the provided SQS queue. If a message is found a Task can be launched ( optional). Optionally it be configured after a message is received the task goes to success and does not continue to poll for new messages.

FieldDescription

Action

receive-message action

AWS_ACCESS_KEY_ID

AWS Account credentials Access Key

AWS_DEFAULT_REGION

AWS Account Region

AWS_SECRET_ACCESS_KEY

AWS Account credentials Secret Access Key

Useproxy ( default is NO )

[NO | YES]

If set to YES, the fields to set-up the proxy server connections are displayed:

  • Proxy Server IP or hostname

  • Proxy Server Port

  • Proxy Server Credentials (optional)

Universal Controller URL

Universal Controller URL

URL has no backslash “/” at the end.

Example: http://192.168.88.10:8080/uc

Universal Controller Credentials

Universal Controller Credentials

The Credentials need to have “Web Service Access” Permissions

Attribute Names


[All|Policy|VisibilityTimeout|MaximumMessageSize|MessageRetentionPeriod|ApproximateNumberOfMessages|ApproximateNumberOfMessagesNotVisible|CreatedTimestamp|LastModifiedTimestamp|QueueArn|ApproximateNumberOfMessagesDelayed|DelaySeconds|ReceiveMessageWaitTimeSeconds|RedrivePolicy|FifoQueue|ContentBasedDeduplication|KmsMasterKeyId|KmsDataKeyReusePeriodSeconds]

For details on the attributes refer to: AWS-SQS-CLI-AttributeNames

MessageAttributeNames

[ALL | list of attributes in CSV format]

Message Attributes to be returned

‘ALL' returns all attributes of the message

Example: MessageAttributeNames = Author, Title will return only the attribute for Author and Title.

WaitTimeSeconds

The duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than WaitTimeSeconds . If no messages are available and the wait time expires, the call returns successfully with an empty list of messages.

Launch Task

Taskname of the Task to launch in Universal Controller.

The task will be always launched with two Task Variables:

  • {Body}: contains the message body in json format

  • {Attributes}: contains the message attributes in json format.

The Variable content can be used in further processing.

e.g. If a Linux task is launched by the AWS SQS Task than an echo {Attributes} command executed by the Linux task will print out all message attributes in json format to STDOUT.

Delete Messages from queue after reading

[ Yes | No ]

If “Yes” the received Messages is deleted from the queue after reading

Interval in seconds

Message Polling Interval in seconds

Run Mode

[Run Once | Run Forever]

  • Run once : if a message is found the task goes to success

  • Run Forever : each time a new message is found the configured task is launched

Example:

  • Run once can be used in Workflows

  • Run Forever can be used as Standalone Task, where each message triggers a task launch.

MaxNumberOfMessages 

The maximum number of messages to return. Amazon SQS never returns more messages than this value (however, fewer messages might be returned). Valid values: 1 to 10. Default: 1.

Loglevel ( default is INFO )

Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]

Role Based Access (STS)

[NO | YES]

If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:

  • Role Arn: Amazon Role, which is applied for the connection

Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly

STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.

Endpoint URL

Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system

Examples for AWS SQS Universal Tasks - Action: receive-message

Receive-message - Action