Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Disclaimer

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/

Overview

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.

This Universal Extension provides the capability to send an AWS SQS message towards an existing queue.

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Software Requirements for Universal Template and Universal Task

  • Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.

Software Requirements Universal Agent

  • Both Windows and Linux agents are supported:

    • Universal Agent for Windows x64 Version 7.1.0.0 and later.

    • Universal Agent for Linux Version 7.1.0.0 and later.

Software Requirements Universal Controller

  • Universal Controller Version 7.0.0.0 and later.

Key Features

This Universal Extension provides the following main features:

  • Support to send an AWS SQS message towards a standard or a FIFO queue.

  • Support for authorization via IAM Role-Based Access Control (RBAC) strategy.

  • Support for Proxy communication via HTTP/HTTPS protocol.

Import Universal Template

To use the Universal Template, you first must perform the following steps:

  1. This Universal Task requires the /wiki/spaces/UC71x/pages/5178443 feature. Check that the/wiki/spaces/UC71x/pages/5177877 system property has been set to true.

  2. Download the provided ZIP file.

  3. In the Universal Controller UI, select Administration >Configuration > Universal Templates to display the current list of Universal Templates.

  4. Click Import Template.

  5. Select the template ZIP file and Import.

When the template has been imported successfully, the Universal Template will appear on the list, Refresh your Navigation Tree to see these tasks in the Automation Center Menu.

Configure Universal Task

For the new Universal Task type, create a new task, and enter the task-specific details that were created in the Universal Template.

Input Fields

The input fields for this Universal Extension are described below.

Field

Input Type

Default Value

Type

Description

Action

Required

Send Message

Choice

The action performed upon the task execution.

Region

Required

-

Text

Region for the Amazon Web Service.

Find more information about the AWS Service endpoints and quotas here.

Credentials

Required

-

Credentials

The AWS account credentials.

They are comprised of:

  • AWS access key ID

  • AWS secret access key

Queue Name

Required

-

Dynamic Choice

Dynamic fetched list of queue names.

The user can select the required queue name from a drop-down list.

Queue Name Prefix

Optional

-

Text

A prefix to use for filtering the Queue Names list results.

Only those queues whose name begins with the specified string are returned.

Queue Names are case-sensitive.

Role Based Access

Optional

False

Boolean

Flag to configure whether the type of authorization is based on IAM Role-Based Access Control(RBAC) strategy.

Find more information about the IAM RBAC authorization model here.

Role ARN

Optional

-

Text

The Role Amazon Resource Name (ARN) to have access to the SQS queue.

Role ARN format: arn:aws:iam::<AWS Account ID>:instance-profile/<Role name>

Required when Role Based Access has been clicked.

Message Body Source

Required

Raw

Choice

Type of message to be sent.

Available options are:

  • Raw

  • Script

Message Attributes

Optional

-

Script

The message attributes to send alongside the message body. Each message attribute consists of a Name, Type, and Value. It should be in JSON format and can have up to 10 attributes.

Example of Attribute Script with 2 Attributes:

{
 "CustomerFirstname": {
  "DataType": "String",
  "StringValue": "John"
 },
 "CustomerLastname": {
  "DataType": "String",
  "StringValue": "Doe"
 }
}

Message Body

Optional

-

Large Text

The message to send to queue.

A message can include only XML, JSON, and unformatted text.

The following Unicode characters are allowed:

#x9, #xA, #xD, #x20 to #xD7FF, #xE000 to #xFFFD, #x10000 to #x10FFFF

Any characters not included in this list will be rejected.

For more information, see the W3C specification for characters.

Required when Message Body Source is configured for "Raw"

Message Body Script

Optional

-

Script

The message to send to queue. The minimum message size is 1 byte (1 character).

The maximum is 262,144 bytes (256 KB).

Required when Message Body Source is configured for "Script".

Is Queue Fifo

Required

False

Boolean

Flag to configure whether the message will be sent to a FIFO Queue.

It should be enabled, if the message will be sent to a FIFO Queue.

Delay Seconds

Optional

-

Integer

The length of time, in seconds, for which to delay a specific message. Messages with a positive value become available for processing after the delay period is finished.

Valid values from 0 to 900.

Used when Is Queue Fifo is disabled.

Message Deduplication ID

Optional

-

Text

Specifies ID for deduplication of sent messages. If a Message Deduplication ID is not provided and the queue doesn't have Content Based Deduplication set on Queue, an error will be raised.

Maximum Length is 128 characters.

Used when Is Queue Fifo is enabled.

Message Group ID

Optional

-

Text

Group ID that specifies that a message belongs to a specific message group.

Maximum Length is 128 characters.

Required when Is Queue Fifo is enabled.

Use Proxy

Required

False

Boolean

Flag to configure whether Proxy should be used in communication.

Proxy Type

Optional

http

Choice

Type of proxy connection to be used.

Available options are:

  • http

  • https

  • https with password

Required when Use Proxy is enabled.

Proxy

Optional

-

Text

Proxy server and port. Valid format: http://proxyip:port or https://proxyip:port.

Required when Use Proxy is enabled.

Proxy CA Bundle File

Optional

-

Text

The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy.

Used when Proxy Type is configured for "https" or "https with password".

Proxy Credentials

Optional

-

Credentials

Credentials to be used for the proxy communication.

They are comprised of:

  • username

  • password

Required when Proxy Type is configured for "https with password".

Task Examples

Send Message to Standard Queue

Example of Universal Task for sending a message towards a standard queue:

Send Message to FIFO Queue

Example of Universal Task for sending a message via script towards a FIFO queue:

Send Message with Role Based Access Enabled

Example of Universal Task for sending a message with Role Based Access Enabled.

http Proxy Type Task Configuration

Example of Universal Task for sending a message through http proxy connection.







Field Descriptions for AWS SQS Universal Task - Actions

The AWS SQS Task provides three different Actions.

  • receive-message

  • list-queues

  • send-message

For each action, the specific fields are described and an example is provided.

list-queues - Action

The Action list the available SQS queues for the given AWS Account and Role ARN ( optional ).

FieldDescription

Action

list-queues action

AWS_ACCESS_KEY_ID

AWS Account credentials Access Key

AWS_DEFAULT_REGION

AWS Account Region

AWS_SECRET_ACCESS_KEY

AWS Account credentials Secret Access Key

Useproxy ( default is NO )

[NO | YES]

If set to YES, the fields to set-up the proxy server connections are displayed:

  • Proxy Server IP or hostname

  • Proxy Server Port

  • Proxy Server Credentials (optional)

Queue Name Prefix

A string to use for filtering the list results. Only those queues whose name begins with the specified string are returned.

Queue URLs and names are case-sensitive.

e.g. Queue Name Prefix = orders , will list all Queues starting with the name orders

Loglevel ( default is INFO )

Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]

Role Based Access (STS)

[NO | YES]

If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:

  • Role Arn: Amazon Role, which is applied for the connection

Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly

STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.

Endpoint URL

Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system


Examples for AWS SQS Universal Tasks - Action: list-queues

List-Queues - Action

Send-message - Action

This Action inserts a message into the given AWS SQS queue. The message Body and Attributes are configurable.

FieldDescription

Action

send-message action

AWS_ACCESS_KEY_ID

AWS Account credentials Access Key

AWS_DEFAULT_REGION

AWS Account Region

AWS_SECRET_ACCESS_KEY

AWS Account credentials Secret Access Key

Useproxy ( default is NO )

[NO | YES]

If set to YES, the fields to set-up the proxy server connections are displayed:

  • Proxy Server IP or hostname

  • Proxy Server Port

  • Proxy Server Credentials (optional)

SQS Queue Name

Name of the SQS Message queue Queue names are case-sensitive.

Message Body

SQS Message Body

The message must contain the parameter Message Body. Parameter is of Type string.

Example: {"Category":"Books"}

Attribute Script

The Attribute Script is a script in json format, which is saved in the Controller script library.

Each message attribute consists of a NameType, and Value.

The message attributes can be provided using JSON format.

Example of Attribute Script with 2 Attributes:

{
	"CustomerFirstname": {
		"DataType": "String",
		"StringValue": "Nils"
	},
	"CustomerLastname": {
		"DataType": "String",
		"StringValue": "Buer"
	}
}

Amazon SQS lets you include structured metadata (such as timestamps, geospatial data, signatures, and identifiers) with messages using message attributes. Each message can have up to 10 attributes. Message attributes are optional and separate from the message body (however, they are sent alongside it).

Delay Seconds

The length of time, in seconds, for which to delay a specific message. Valid values: 0 to 900. Maximum: 15 minutes. Messages with a positive DelaySeconds value become available for processing after the delay period is finished. If you don't specify a value, the default value for the queue applies.

Loglevel ( default is INFO )

Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]

Role Based Access (STS)

[NO | YES]

If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:

  • Role Arn: Amazon Role, which is applied for the connection

Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly

STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.

Endpoint URL

Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system

Examples for AWS SQS Universal Tasks - Action: send-message

Send-message - Action


Attribute Script Example


Script Example: orders.json

{
    "CustomerFirstname": {
        "DataType": "String",
        "StringValue": "${CustomerFirstname}"
           },
    "CustomerLastname": {
        "DataType": "String",
        "StringValue": "${CustomerLastname}"
           },
    "City": {
        "DataType": "String",
        "StringValue": "${City}"
           },
    "Street": {
        "DataType": "String",
        "StringValue": "${Street}"
           },
    "Zipcode": {
        "DataType": "String",
        "StringValue": "${Zipcode}"
           },
    "Title": {
        "DataType": "String",
        "StringValue": "${Title}"
           },
    "Author": {
        "DataType": "String",
        "StringValue": "${Author}"
           },
    "ISBN": {
        "DataType": "String",
        "StringValue": "${ISBN}"
           },
    "ID": {
        "DataType": "String",
        "StringValue": "${ID}"
           }
}

Receive-message - Action

The Action polls in configurable interval the provided SQS queue. If a message is found a Task can be launched ( optional). Optionally it be configured after a message is received the task goes to success and does not continue to poll for new messages.

FieldDescription

Action

receive-message action

AWS_ACCESS_KEY_ID

AWS Account credentials Access Key

AWS_DEFAULT_REGION

AWS Account Region

AWS_SECRET_ACCESS_KEY

AWS Account credentials Secret Access Key

Useproxy ( default is NO )

[NO | YES]

If set to YES, the fields to set-up the proxy server connections are displayed:

  • Proxy Server IP or hostname

  • Proxy Server Port

  • Proxy Server Credentials (optional)

Universal Controller URL

Universal Controller URL

URL has no backslash “/” at the end.

Example: http://192.168.88.10:8080/uc

Universal Controller Credentials

Universal Controller Credentials

The Credentials need to have “Web Service Access” Permissions

Attribute Names


[All|Policy|VisibilityTimeout|MaximumMessageSize|MessageRetentionPeriod|ApproximateNumberOfMessages|ApproximateNumberOfMessagesNotVisible|CreatedTimestamp|LastModifiedTimestamp|QueueArn|ApproximateNumberOfMessagesDelayed|DelaySeconds|ReceiveMessageWaitTimeSeconds|RedrivePolicy|FifoQueue|ContentBasedDeduplication|KmsMasterKeyId|KmsDataKeyReusePeriodSeconds]

For details on the attributes refer to: AWS-SQS-CLI-AttributeNames

MessageAttributeNames

[ALL | list of attributes in CSV format]

Message Attributes to be returned

‘ALL' returns all attributes of the message

Example: MessageAttributeNames = Author, Title will return only the attribute for Author and Title.

WaitTimeSeconds

The duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than WaitTimeSeconds . If no messages are available and the wait time expires, the call returns successfully with an empty list of messages.

Launch Task

Taskname of the Task to launch in Universal Controller.

The task will be always launched with two Task Variables:

  • {Body}: contains the message body in json format

  • {Attributes}: contains the message attributes in json format.

The Variable content can be used in further processing.

e.g. If a Linux task is launched by the AWS SQS Task than an echo {Attributes} command executed by the Linux task will print out all message attributes in json format to STDOUT.

Delete Messages from queue after reading

[ Yes | No ]

If “Yes” the received Messages is deleted from the queue after reading

Interval in seconds

Message Polling Interval in seconds

Run Mode

[Run Once | Run Forever]

  • Run once : if a message is found the task goes to success

  • Run Forever : each time a new message is found the configured task is launched

Example:

  • Run once can be used in Workflows

  • Run Forever can be used as Standalone Task, where each message triggers a task launch.

MaxNumberOfMessages 

The maximum number of messages to return. Amazon SQS never returns more messages than this value (however, fewer messages might be returned). Valid values: 1 to 10. Default: 1.

Loglevel ( default is INFO )

Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL]

Role Based Access (STS)

[NO | YES]

If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:

  • Role Arn: Amazon Role, which is applied for the connection

Example RoleArn: arn:aws:iam::119322085622:role/SB-SQS-ReadOnly

STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources like SQS Queues.

Endpoint URL

Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system

Examples for AWS SQS Universal Tasks - Action: receive-message

Receive-message - Action





  • No labels