Universal Data Mover Gateway 1.5.x Maintenance

Universal Data Mover Gateway 1.5.x Maintenance

Introduction

Stonebranch provides product maintenance in the form of updated product installation packages. This document lists the product installation package maintenance history for Universal Data Mover Gateway 1.5.x.

For Universal Data Mover Gateway 1.5.x, applying maintenance refers to the increase from a currently installed and supported version of the Universal Data Mover Gateway to a later 1.5.x version of the Universal Data Mover Gateway (for example, increase Universal Data Mover Gateway 1.0.0.0 to Universal Data Mover Gateway 1.5.0.0).

Product Packaging

A package maintenance level is incremented when the package changes or the package installation changes.

Stonebranch changes product version, release, or modification identifiers at its discretion when it deems an appropriate number of enhancements or capabilities have been introduced to warrant the change.

Versioning

Package version numbers are comprised of four numeric identifiers: version, release, modification level, and maintenance level.

For example, for Universal Data Mover Gateway 1.5.0.0:

  • 1 = Version 1

  • 5 = Release 5

  • 0 = Modification Level 0

  • 0 = Maintenance Level 0

Packaging Methods

The Universal Data Mover Gateway 1.5.x packages are provided in formats appropriate for the target platforms.

See the Installation Guide for information on installing, upgrading, and applying maintenance to the Universal Data Mover Gateway.

Package Maintenance Levels

This section identifies the changes included in Universal Data Mover Gateway 1.5.x.

This table identifies the maintenance level of every component in each package.

Release

Release date

UDMG Server

UDMG Agent Proxy

UDMG Authentication Proxy

UDMG Admin UI

UDMG Web Transfer Client

Release

Release date

UDMG Server

UDMG Agent Proxy

UDMG Authentication Proxy

UDMG Admin UI

UDMG Web Transfer Client

1.5.0.3

May 1, 2024

1.5.0.3 (d58b4b40)

1.5.0.0

1.5.0.3 (5cbd9a2)

1.5.0.3 ()be5f1cd

1.5.0.0

1.5.0.2

February 1, 2024

1.5.0.2 (b5f51e5e)

1.5.0.0

1.5.0.0

1.5.0.1 (97c1967)

1.5.0.0

1.5.0.1

December 21, 2023

1.5.0.1 (5a9b7a3f)

1.5.0.0

1.5.0.0

1.5.0.1 (97c1967)

1.5.0.0

1.5.0.0

November 20, 2023

1.5.0.0 (1292ee83)

1.5.0.0

1.5.0.0

1.5.0.0 (98b18e3)

1.5.0.0

UDMG 1.5.0.3 - May 1, 2023

Change ID

Component

Description

Change ID

Component

Description

#36607

UDMG Server

Include the retry configuration parameters from 1.5.0.2 in the Linux DEB/RPM packages

#36608

UDMG Server

Fix the error message when database connection max retries is reached.

#36609

UDMG Server

Fix the log message levels for the database connection attempts:

WARNING: Transaction error, attempt: %v of %v" ERROR: Transaction error - MaxRetries reach %v of %v

#36456

UDMG Authentication Proxy

Improve the validation of login provider JSON files during startup, invalid configuration is reported and the processs stops.

The error was previously reported as "[PANIC RECOVER] runtime error" during login attempts and not easily catchable as a configuration error.

Example of startup error:

level=error TS=2024-04-30T11:28:48.202909409Z Configuration=Auth Service=udmg_openldaps Error="ldap: read ca file: open ca-cert.pem: no such file or directory"
level=error TS=2024-04-30T11:28:48.202945585Z Error="Please validate you configuration file"

#36603

UDMG Admin UI

Fix the inability to show the dashboard after successful LDAP authentication. 

#36415

UDMG Authentication Proxy

Document the various LDAP/S configuration parameters.

UDMG 1.5.0.2 - February 1, 2023

Change ID

Component

Description

Change ID

Component

Description

#35690

UDMG Server

Fix handling of database errors and deadlocks.

New configuration parameters, in the database section:

[database];Maximumnumberoftransactionsretries,thedefaultis3.;MaxRetries=3;Delayinmillisecondsbetweenretries,thedefaultis100.;MaxRetriesWait=100

In case of a database error, the insert is attempted again until MaxRetries and with a waiting time of MaxRetriesWait between each attempts.

UDMG 1.5.0.1 - December 21, 2023

Change ID

Component

Description

Change ID

Component

Description

#35246

UDMG Server

HTTP client transfer is set to ERROR when the HTTP response status code indicates a failure.

For HTTP code 401, the error code is TeBadAuthentication.
For HTTP code 403, the error code is TeForbidden.
For HTTP code 404, the error code is TeFileNotFound.
For HTTP code between 100 and 199, the error code is TeUnknown.
For HTTP code between 300 and 399, the error code is TeUnknown.
For HTTP code between 400 and 499, the error code is TeConnection.
For HTTP code between 500 and 599, the error code is TeConnection.
The HTTP code and description is added to the error message, the response payload is kept in the temporary file to allow troubleshooting.

#35211

UDMG Admin UI, 
UDMG Server

Fix the inability to resume the TOTP user registration when it was not completed the first time. On further attempts to login while the TOTP registration on the user device was not confirmed (step 2), the presented QR code was not correct and prevented the user to define the TOTP account on the authenticator app. 

#35316

UDMG Admin UI

The TOTP user registration can be reset at any time for a user, even when the initial registration is not completed. It allows to force the generation of a new secret key (and QR code)

#35253

UDMG Server

Fix the resolution of the local path for Web Transfer Client transfers when the local-auth server root directory is a relative path. The local path on udmg-server transfer request was not correct and caused an error at the end of the transfer.

#35280

UDMG Server

Fix the startup of enabled local servers after udmg-server instance restart. In some case a local server could not be started with a "bind: address already in use" error message.

#35189

UDMG Server

Fix the listing of local servers in the status report. It will appear on the active instance after a "start" command instead of appearing in the instance that processed the creation request.

#36259

UDMG Server

Fix the inability to change the allowed commands (file delete and rename) for a Web Transfer instance with a simple restart of the local-auth server. The change was only effective after the whole udmg-server restart.

#35164

NGINX

Linux installation packages: change default port for UDMG Authentication Proxy from 5000 to 5775. This is to avoid conflict with other software as this port is very commonly use. 

UDMG 1.5.0.0 - November 20, 2023

Change ID

Component

Description

Change ID

Component

Description

B-17739,
#31690,
#34875

UDMG Admin UI, 
UDMG Server,
UDMG Authentication Proxy, 
UDMG Client

Standard Login Method with Authenticator App (TOTP) 2FA.

UDMG user can be configured with the "Standard / Authenticator App (TOTP)" (local-otp) login method.


When enabled, the login requires a TOTP code from an authenticator mobile application.

The registration with a generated QR code or secret is performed on first login or after the OTP registration has been reset.

The OTP registration can be reset from the CLI or from the UDMG Admin UI, this requires the 'user write' permissions.

 

$ udmg-client user update --login-type=local-otp bob The user bob was successfully updated. $ udmg-client user get bob ● User bob User groups: Generic Email: Login Type: local-otp Permissions: ├─Transfers: --- ├─Servers: --- ├─Partners: --- ├─Rules: --- ├─Users: r-- ├─Administration: --- └─Pgp: --- $ mft user reset-otp bob The user bob was successfully reset.

#34498

UDMG Server

High Availability: Active/Passive instances with dynamic handover

Support for cluster awareness where multiple udmg-server instances are coordinating to have 1 active instance and the remaining ones remain in passive state. Continuous monitoring is performed to ensure that a passive instance will takeover in case of failure or shutdown of the active instance.

The MFT servers are owned by the active instance, which also manage the transfer requests in client mode.

See High Availability Behavior and Processing

#34810

UDMG Server

Web services for the multi instance monitoring and load balancing.

  • Existing /api/sb_healthcheck is enhanced with additional information: node details and node status. 
    Requires authentication but no specific permissions.
    Example of output for a passive instance:

    {

        "status": "operational",

        "nodeId": "gateway_1:8080-mft-gw-0",

        "nodeHostname": "gateway_1",

        "nodeIPAddress": "172.99.0.101",

        "nodePort": "8080",

        "nodeStatus": "PASSIVE",

        "nodeUptime": "15h6m4.810854748s",

        "nodeLastUpdate": "2023-11-14T08:07:54.848374Z",

        "nodeLastActiveDate": "2023-11-13T16:57:57.026091Z"

    }

  • New /ping endpoint returns the plain text status: 'ACTIVE', 'PASSIVE', or 'OFFLINE' and also the 403 (Forbidden) HTTP code when the node is not active. This API is provided without authentication for load balancers.

 

  • New /api/sb_mgmt_nodes endpoint returns the list of the instances with their details and status.
    Requires the 'administration read' permission.
    Example of output for 2 instances:

    { "nodes": [ { "nodeId": "gateway:8080-mft-gw-0", "nodeHostname": "gateway", "nodeIPAddress": "172.99.0.100", "nodePort": "8080", "nodeStatus": "ACTIVE", "nodeUptime": "16m15.413255244s", "nodeLastUpdate": "2023-11-09T15:24:20.562225Z", "nodeLastActiveDate": "2023-11-09T15:08:40.105002Z" }, { "nodeId": "gateway_1:8080-mft-gw-0", "nodeHostname": "gateway_1", "nodeIPAddress": "172.99.0.101", "nodePort": "8080", "nodeStatus": "PASSIVE", "nodeUptime": "16m0.631810433s", "nodeLastUpdate": "2023-11-09T15:24:35.289412Z", "nodeLastActiveDate": "2023-11-09T14:40:28.491424Z" } ] }
    See REST API connections

#34873

UDMG Admin UI

UI: Management of user sessions

The open sessions for the users of the UDMG Admin UI can be displayed and terminated from the user menu.
Requires the 'administration write' permissions.

#34872

UDMG Server

REST/CLI: Management of user sessions

REST API: new /api/sb_session endpoint with GET and DELETE 

GET /api/sb_session list all the user sessions requires Admin permission { "sessions": [ { "sessionId": 116, "userId": 8, "username": "oli", "ipAddr": "185.70.76.140", "completed": true, "creationDate": "2023-11-09T12:38:07.718294Z", "expirationDate": "2023-11-09T13:38:07.718291Z", "lastUsedDate": "2023-11-09T12:41:20.852212Z" }, { "sessionId": 117, "userId": 1, "username": "admin", "ipAddr": "185.70.76.140", "completed": true, "creationDate": "2023-11-09T12:38:17.470503Z", "expirationDate": "2023-11-09T13:38:17.470492Z", "lastUsedDate": "2023-11-09T12:41:26.508968Z" } ] } DELETE /api/sb_session/{id} delete a user session, forcing its expiration requires Admin permission

CLI: new 'session list' and 'session revoke' commands

udmg-client [CONNECTION-OPTIONS] session <list | revoke> Available commands: list List current sessions revoke Revoke user session session udmg-client session list Sessions: ● Session 196 User: admin IP Address: 79.129.6.7 Completed: true Creation Date: 2023-11-13T09:05:20.780075Z Expiration Date: 2023-11-13T10:05:20.780072Z Last Used Date: 2023-11-13T09:58:34.707537Z ● Session 204 User: admin IP Address: 172.99.0.7 Completed: true Creation Date: 2023-11-13T09:51:04.091549Z Expiration Date: 2023-11-13T10:51:04.091546Z Last Used Date: 2023-11-13T09:51:04.091551Z udmg-client session revoke 204 The session 204 was successfully revoked.

#34820

NGINX, 
UDMG Admin UI

NGINX configuration update so that the client IP address (X-Real-IP, X-Forwarded-For) is passed for UDMG authentication proxy, thus the address can be kept with the user session details. See Installing NGINX Server.

#34890

UDMG Admin UI

UI: Management of the 'Administration' permission for users

#34112

UDMG Client

CLI: Management of the 'administration' user permission. 

  • Administration permissions are displayed on 'user list' or 'user get' commands

  • read/write/delete can be assigned with the 'A' tag for administration permission on 'user add' and 'user update' commands

  • 'superuser' shortcut is provided to create or convert a user to a superuser with all possible permissions, including the administration permissions

    udmg-client user update bob -r'superuser' 

#34834

Linux services

Improved usability of the Linux services:

  • Service description is prefixed with "Stonebranch" for parity with Universal Agent ubroker service.

  • Syslog messages are labeled with the service name (udmg-server, udmg-auth-proxy, ...) instead of 'sh'.

  • Syslog messages are sent to 'local0' utility for easier filtering.

#34247

UDMG Server

Allow absolute paths in SFTP client requests. If the remote filename stub in the transfer request starts with a '/' it is considered an absolute path and will be used without prefixing with the transfer rule remote directory path. 

For example, the client transfer udmg-client transfer add -f localFile.txt -o /remotepath/remoteFile.txt -p SftpPartner -l remoteAccount -w send -r Rule1S will now send /remotepath/remoteFile.txt as the path in the SFTP request whereas before, the path would have been {rule remote_directory}/other/remoteFile.txt.

#33063

UDMG Admin UI

Environment customization with the service settings on the UDMG Authentication Proxy configuration.

  • "udmg.system_identifier": name of the system or the environment.

  • "udmg.banner.background_color": color of the banner background, as HTML color name ("Brown"), RGB code ("rgb(165,42,42)"), or hexadecimal code ("#A52A2A").

  • "udmg.banner.logo": company logo, optional picture to display next to the system identifier.

Example of a production environment with a brown banner and a development environment with an orange banner:

#33062

UDMG Authentication Proxy

Custom properties can be associated to a udmg service for use by the UDMG Admin UI.
Refer to the installation guide for the configuration file syntax and the [settings] section.

#31488

UDMG Server

Business Service as Tag

Improved the usability of Business Service : they can be used for tagging objects and not only to enforce security permissions. A Business Service without any User Group can be attached to an object as a way labeling it.

Permissions over the new business service are not required anymore when assigning an object to a business service thus allowing them to be used as tags (i.e. to group resources). The permissions required to assign an object to a business service are either: having the corresponding global permission (object write at user level) or having the corresponding business service permission (object write at user group level) in at least one of the object's business services.

REST API changes:

  • Added new endpoints to add business server members:

PUT /api/sb_businessservices/{name}/local_agents/{object_name}
PUT /api/sb_businessservices/{name}/remote_agents/{object_name}
PUT /api/sb_businessservices/{name}/rules/{object_name}/{direction:send|receive}
PUT /api/sb_businessservices/{name}/sb_pgp/{object_name}

  • Added new endpoints to delete business server members:

DELETE /api/sb_businessservices/{name}/local_agents/{object_name}
DELETE /api/sb_businessservices/{name}/remote_agents/{object_name}
DELETE /api/sb_businessservices/{name}/rules/{object_name}/{direction:send|receive}
DELETE /api/sb_businessservices/{name}/sb_pgp/{object_name}

  • Added new endpoints to add user group members:

PUT /api/sb_usergroups/{name}/users/{user}

  • Added new endpoints to delete user group members:

DELETE /api/sb_usergroups/{name}/users/{user}

  • Added new endpoints to list business server members:

GET /api/sb_businessservices/{name}/servers
GET /api/sb_businessservices/{name}/partners
GET /api/sb_businessservices/{name}/rules
GET /api/sb_businessservices/{name}/sb_pgp

#34725

UDMG Server

Easier upgrade procedure, the migrate command now uses the last version by default.

$ /opt/udmg/bin/udmg-server migrate -l -c /opt/udmg/etc/udmg-server/server.ini | tail -1 1.5.0 $ /opt/udmg/bin/udmg-server migrate -c /opt/udmg/etc/udmg-server/server.ini -v -v -v 2023/10/30 10:57:24 [INFO ] Migration: Starting upgrade migration... ... 2023/10/30 10:57:25 [INFO ] Migration: Applying migration 'Bump database version to 1.5.0'

#33071

UDMG Server

Disable Rule Task type

Ability to globally disable each rule task type.

New udmg-server configuration parameters, in the [tasks] section, for each task type allow to disable the related task processing during any file transfer. For the following task types: COPY, COPYRENAME, MOVE, MOVERENAME, DELETE, RENAME, EXEC, EXECMOVE, EXECOUTPUT, TRANSFER, CHECKREGEX, PUBLISHEVENT, ICAP. 

B-17654,
#34136

UDMG Server, UDMG Client

REST and CLI support for Personal Access Token.

Note that a token owner is always allowed to manage its own tokens.

New Web Services:

  • /api/sb_users/{user}/tokens