Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
This Universal Extension provides the capability to monitor AWS SQS messages from an existing queue and run job(s) and/Universal Task or workflows accordingly.

Version Information

Template NameExtension NameExtension Version
Amazon SQS Monitorue-aws-sqs-monitor1.1.1

Refer to Changelog for version history information.

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Software Requirements for for Universal Template and  and Universal Task

Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.

Software Requirements Universal Agent

...

This Universal Task requires that Universal Controller property /wiki/spaces/UC71x/pages/5177877 Web Service Default Response Content (uc.web_service.response.content.default) is set to JSON.

Network and Connectivity Requirements

Extension's Universal Agent host should be able to establish connection with:

  • AWS SQS Queue REST endpoints (The AWS Credentials provided in the Amazon SQS Queue Monitor Universal Task, should have sufficient permissions on AWS to get messages from the specified queue).
  • The Universal Controller host URL (The Universal Controller Credentials provided in permissions to launch a Universal Controller Task).

Key Features

This Universal Extension provides the following main features:

...

  • Action
    • Monitor AWS SQS messages from a standard or a FIFO queue.

...

    • Launch a task in Universal Controller with variables holding the id, body, attributes, message attributes and receipt handle for each fetched message.
  • Authentication
    • AWS Credentials.

...

    • IAM Role-Based Access Control (RBAC) strategy.

...

  • Other
    • Communication through Proxy with use of HTTP or HTTPS.

Import Universal Template

To use the Universal Template, you first must perform the following steps:.

  1. This Universal Task requires

...

  1. the Resolvable Credentials feature. Check that the

...

  1. Resolvable Credentials Permitted system property has been set to true.

...

Download the provided ZIP file.

...

  1. To import the Universal Template into your Controller

...

Click Import Template.

...

  1. , follow the instructions here.

  2. When the

...

  1. files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list

...

  1. .

Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information refer to Integration Modifications.

Configure Universal Task

For the a new Universal Task type, create a new task, and enter the task-specific details that were created in the Universal Templaterequired input fields.

Input Fields

The input fields for this Universal Extension are described in the following table.

Field

Input type

Default value

Type

Description

Action

Required

Send Message

Monitor Messages

Choice

The action performed upon the task execution.

Region

RequiredRequired

Optional since version 1.1.0

Optional

-

Text

Region for the Amazon Web Service.

Find more information about the AWS Service endpoints and quotas here.

When AWS Region is not populated as part of the task definition, during task execution the integration will look for AWS Region on the task execution environment. Refer to configuration options for more information.

AWS Credentials



Optional since version 1.1.0

Optional

-

Credentials

The

AWS account credentials.

They are comprised of:

  • AWS access key ID

  • AWS secret access key

    Credentials definition should be as follows.

    • AWS Access Key ID as "Runtime User".
    • AWS Secret Access Key as "Runtime Password".

    When AWS Credentials are not populated as part of the task definition, during task execution the integration will look for AWS Credentials on the task execution environment. Refer to configuration options for more information.

    Queue Name

    Required

    -

    Dynamic Choice

    Dynamic fetched list of queue names.

    The user can select the required queue name from a drop-down list.

    Queue Name Prefix

    Optional

    -

    Text

    A prefix to use for filtering the Queue Names list results.

    Only those queues whose name begins with the specified string are returned.

    Queue Names are case-sensitive.

    Role Based Access

    Optional

    False

    Boolean

    Flag to configure whether the type of authorization is based on IAM Role-Based Access Control(RBAC) strategy.

    Find more information about the IAM RBAC authorization model here.

    Role ARN

    Optional

    -

    Text

    The Role Amazon Resource Name (ARN) to have access to the SQS queue.

    Role ARN format: arn:aws:iam::<AWS Account ID>:instance-profile/<Role name>

    Required when Role Based Access has been clicked.

    Use Proxy

    Required

    Optional

    False

    Boolean

    Flag to configure whether Proxy should be used in communication.

    Proxy Type

    Optional

    http

    HTTP

    Choice

    Type of proxy connection to be used.

    Available options are:

    http
    • HTTP

  • https

  • https with password
    • HTTPS

    • HTTPS With Credentials

    Required when Use Proxy is enabled.

    Proxy

    Optional

    -

    Text

    Proxy server and port. Valid format: http://proxyip:port or https://proxyip:port.

    Required when Use Proxy is enabled.

    Proxy CA Bundle File

    Optional

    -

    Text

    The path to a custom certificate bundle to use when establishing SSL/TLS connections with proxy.

    Used when Proxy Type is configured for "

    https

    HTTPS" or "

    https with password

    HTTPS With Credentials".

    Proxy Credentials

    Optional

    -

    Credentials

    Credentials to be used for the proxy communication

    .

    They are comprised of:

    • username

    • password

    Required when Proxy Type is configured for "https with password

    . The credential definition should be as follows.

    • Proxy Username as "Runtime User".
    • Proxy Password as "Runtime Password".

    Required when "Proxy Type" is configured for "HTTPS" or "HTTPS With Credentials".

    Attribute Names

    Optional

    All

    Choice

    A list of attributes to be returned along with each message.

    Available options are:

    • All - Returns all values

    • Approximate First Receive Timestamp – Returns the time the message was first received from the queue (epoch time in milliseconds).

    • Approximate Receive Count – Returns the number of times a message has been received across all queues but not deleted.

    • AWS Trace Header – Returns the X-Ray trace header string.

    • SenderId:

      • For an IAM user, returns the IAM user ID.

      • For an IAM role, returns the IAM role ID.

    • Sent Timestamp – Returns the time the message was sent to the queue (epoch time in milliseconds).

    • Message Deduplication Id – Returns the value provided by the producer of the message for FIFO queue.

    • Message Group Id – Returns the value provided by the producer of the message for FIFO queue.

    • Sequence Number – Returns the value provided by Amazon SQS.

    Message Attribute Names

    Optional

    -

    Text

    Attribute names of the message to be returned from the request, separated by a comma.

    For example: FirstName,LastName,Title.

    ALL returns all attributes of the message.

    Retrieve all message attributes starting with a prefix, using for example bar.*.

    Naming rules and restrictions:

    • Name can contain alphanumeric characters and the underscore (_ ), hyphen (- ), and period (. ).

    • Name is case-sensitive and must be unique among all attribute names for the message.

    • Name must not start with AWS-reserved prefixes such as AWS. or Amazon. (or any casing variants).

    • Name must not start or end with a period (. ), and it should not have periods in succession (.. ).

    • Name can be up to 256 characters long.

    Maximum Number of Messages

    Required

    10

    Integer

    The maximum number of messages to return within each request.

    Valid values from 1 to 10.

    Visibility Timeout (seconds)

    Required

    1

    Integer

    The duration that the received messages are hidden from subsequent retrieve requests.

    Valid values range from 1 to 43200.

    Wait Time (seconds)

    Required

    1

    Integer

    The duration for which the call waits for Maximum Number of Messages to arrive in the queue before returning.

    If the requested number of messages is available, the messages are returned sooner than Wait Time (seconds).

    Polling Interval (seconds)

    Required

    60

    Integer

    The amount of time the task will wait before pooling for a new message batch when no messages are found during the last cycle.

    Valid values from 0.

    Delete After Processing

    Required

    True

    Boolean

    Flag to delete the received message after processing.

    If an error occurs during the launching task process, the received message will not be deleted regardless of the Delete after processing flag.

    Launch Task Name

    Required

    -

    Text

    The name of the Task to be launched in the Universal Controller.

    The task will always be launched with the Task variables:

    • messageId - Returns the MessageId.

    • messageBody - Returns the Body of message.

    • attributes - Returns the list of specified attributes of the message in JSON format.

    • messageAttributes - Returns the list of specified message attributes in JSON format.

    • receiptHandle - Returns the received ReceiptHandle value of message.

    Universal Controller URL

    Required

    -

    Text

    The Universal Controller URL to launch the Task

    after

    when receiving a message.

    For example

    Valid URL format is:

    http(s)://

    192.168.88.10:8080

    controller_ip:controller_port/uc.

    Universal Controller Credentials

    Required

    -

    Credentials

    The credentials of Universal Controller URL to be used for launching the task

    The Credentials must have “Web Service Access” Permissions.

    Task Examples

    Monitor Messages from Queue

    Example of Universal Task for monitoring messages from an existing queue.

    Image RemovedImage Added

    Monitor Messages with Role Based Access Enabled

    Example of Universal Task for monitoring messages with Role Based Access Enabled.

    Image Removed

    HTTP Proxy Type Task Configuration

    Example of Universal Task for monitoring messages through HTTP proxy connection.

    Image Removed

    ...

     The credentials in this case are optional and will be stored as environment variables on the execution environment.

    Image Added

    HTTPS With Credentials Proxy Type Task Configuration

    Example of Universal Task for monitoring messages through HTTPS With Credentials proxy connection:

    Image Removed

    HTTPS with password Proxy Type Task Configuration

    Example of Universal Task for monitoring messages through HTTPS with password proxy connection.

    Image Removed

    Image Added

    Task Output

    Output Only Fields

    ...

    The output fields for this Universal Extension are described in the following table.

    Field

    Type

    Preserved on re-run

    Description

    Last Monitor Datetime

    Text

    False

    Timestamp of the last time the messages were pooled from the queue.

    Last Monitor Message Count

    Integer

    False

    Number of fetched messages since the start of the monitoring (task instance).

    Launched Task IDs

    Text

    False

    sysID (32 char) of the last launched task as returned by UAC REST API.

    Read Message Count

    Integer

    False

    Number of fetched messages since the start of the monitoring (task instance).

    Launched Task Count

    Integer

    False

    Number of launched tasks instances since the start of the monitoring (task instance).

    Exit Codes

    The exit codes for this Universal Extension are described in the following table.

    Exit Code

    Status Classification Code

    Status Classification Description

    Status Description

    0

    SUCCESS

    Successful Execution

    SUCCESS: Successful Task execution

    1

    FAIL

    Failed Execution

    FAIL: < Error Description >

    3

    AUTHORIZATION_ERROR

    Insufficient Permissions

    AUTHORIZATION_ERROR: The authorization credentials provided for the request are invalid.

    10

    CONNECTION_ERROR

    Bad connection data or connection timed out

    CONNECTION_ERROR: < Error Description >

    20

    DATA_VALIDATION_ERROR

    Bad input fields validation

    DATA_VALIDATION_ERROR: Some of the input fields cannot be validated.

    See STDERR for more details.

    STDOUT and STDERR

    STDOUT and STDERR provide additional information to User. The populated content can be changed in future versions of this extension without notice. Backward compatibility is not guaranteed.

    Anchor
    im
    im

    Integration Modifications

    Modifications applied by users or customers, before or after import, might affect the supportability of this integration. The following modifications are discouraged to retain the support level as applied for this integration.

    • Python code modifications should not be done.
    • Template Modifications
      • General Section
        • "Name", "Extension", "Variable Prefix", "Icon" should not be changed.
      • Universal Template Details Section
        • "Template Type", "Agent Type", "Send Extension Variables", "Always Cancel on Force Finish" should not be changed.
      • Result Processing Defaults Section
        • Success and Failure Exit codes should not be changed.
        • Success and Failure Output processing should not be changed.
      • Fields Restriction Section
        The setup of the template does not impose any restrictions, However with respect to "Exit Code Processing Fields" section.
        1. Success/Failure exit codes need to be respected.
        2. In principle, as STDERR and STDOUT outputs can change in follow-up releases of this integration, they should not be considered as a reliable source for determining success or failure of a task.

    Users and customers are encouraged to report defects, or feature requests at Stonebranch Support Desk.

    Document References

    This document references the following documents:

    Name

    Location
    Document LinkDescription
    Universal Templates
    https://docs.stonebranch.com/confluence/display/UC71x/
    Universal
    +
    TemplatesUser documentation for creating, working with and understanding Universal Templates
    in the Universal Controller user interface.

    Universal Tasks

    https://docs.stonebranch.com/confluence/display/UC71x/Universal+
    and Integrations.
    Universal TasksUser documentation for creating Universal Tasks in the Universal Controller user interface.
    Amazon SQS MessageUser documentation for Amazon SQS
    quotas

    https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-quotas.html

    User documentation for Amazon SQS quotas.

    Amazon Simple Queue Service endpoints and quotas

    https://docs.aws.amazon.com/general/latest/gr/sqs-service.html

    User Guide for Amazon Simple Queue Service endpoints and quotas.

    IAM RBAC authorization model

    https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html#introduction_attribute-based-access-control_compare-rbac

    User Documentation for Comparing ABAC to the traditional RBAC model.
    .
    IAM RBAC authorization modelUser Documentation for Comparing ABAC to the traditional RBAC model.

    Changelog

    ue-aws-sqs-monitor-1.1.1 (2023-07-28)

    Fixes

    • Fixed: Field 'Proxy Type' raised a data validation error on Controller 7.3.0.0 and later. (#33720)
    • Fixed: Error handling when launching a UAC task in case of a successfully monitored message. (#33922)

    ue-aws-sqs-monitor-1.1.0 (2022-09-15)

    Enhancements

    • Added: Allow AWS Credentials and AWS Region as optional fields enabling their configuration on the task execution environment. (#28271)
    • Added: Allow more data in the STDERR (Printing the name of the launched task, message id and task instance sys_id ). (#30006)
    • Added: Task status goes to fail if the proxy can not be reached. (#30066)
    • Added: Task status goes to fail if UAC conectivity error. (#30049)
    • Added: Task status goes to fail if lauch task does not exist. (#30006)

    ue-aws-sqs-monitor-1.0.0 (2022-02-03)

    Initial Version