Universal Data Mover Gateway 2.0.x Maintenance

Introduction

Stonebranch provides product maintenance in the form of updated product installation packages. This document lists the product installation package maintenance history for Universal Data Mover Gateway 2.0.x.

Product Packaging

A package maintenance level is incremented when the package changes or the package installation changes.

Stonebranch changes product version, release, or modification identifiers at its discretion when it deems an appropriate number of enhancements or capabilities have been introduced to warrant the change.

Versioning

Package version numbers are comprised of four numeric identifiers: version, release, modification level, and maintenance level.

For example, for Universal Data Mover Gateway 2.0.0.0:

  • 2 = Version 2

  • 0 = Release 0

  • 0 = Modification Level 0

  • 0 = Maintenance Level 0

Packaging Methods

The Universal Data Mover Gateway 2.0.x packages are provided in formats appropriate for the target platforms.

See Installation Guide for information on installing, upgrading, and applying maintenance to the Universal Data Mover Gateway.

Package Maintenance Levels

This section identifies the changes included in the Universal Data Mover Gateway 2.0.x.

This table identifies the maintenance level of every component in each package.

Release

Release Date

UDMG Server

UDMG Agent ProxyUDMG Authentication ProxyUDMG Admin UIUDMG Web Transfer Client
2.0.0.0June 7,  20242.0.0.02.0.0.02.0.0.02.0.0.02.0.0.0

UDMG 2.0.0.0 - June 7, 2024

Change IDComponentDescription
#35208UDMG Server

Adds a parameter to disable the implicit assignment of Transfer Rules. Before this change, a Transfer Rule was allowed for all servers, partners, and accounts right after creation. It was only restricted after an explicit assignment (whitelisting) to at least one server, partner, or account. The prior functionality created confusion and allowed unauthorized access to files and folders for third-party accounts during the time between the rule creation and its explicit assignment. The change allows the functionality to be disabled with the new UDMG Server configuration parameter ExplicitRuleAssignment. When set to true, a Transfer Rule is only effective when it is explicitly linked with the intended target (local server, remote partner, remote or local account).

The default value remains false but is likely to change with future releases.

 Configuration Section: lick here to expand...
[rule]
; Disables global rules, requiring rules to be explicitly allowed to be used.
; ExplicitRuleAssignment = false
#36187, #35684UDMG Server/Client

Provides a new option to select the login name for the SFTP connection attempts during the CLI SSH keyscan (show or save). The change can prevent the remote server from blocking connections from an unknown user.

 keyscan usage and example: click here to expand...
$ udmg-client partner keyscan partner show -h
Usage:
  udmg-client [GLOBAL-OPTIONS] partner keyscan [partner] show [show-OPTIONS]

[keyscan command arguments]
  partner:                                                                                                                                                               The partner's name

[show command options]
      -t, --type=[ssh-rsa|ssh-dss|ecdsa-sha2-nistp256|sk-ecdsa-sha2-nistp256@openssh.com|ecdsa-sha2-nistp384|ecdsa-sha2-nistp521|ssh-ed25519|sk-ssh-ed25519@openssh.com] Specifies the type of the key to fetch from the host (default: all). Can be repeated multiple times to filter multiple statuses.
      -l, --login=                                                                                                                                                       Specifies the account for the connection attempts.

$ udmg-client partner keyscan my_remote_sftp show -t ssh-rsa -l ec2-user
ven. mars 15 14:12:45 UTC 2024
● Partner remote_sftp SSH public keys:
    Public key 0 :
        Type:       ssh-rsa
        Public key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCY1rAsf...9rmQ6ve5WD+iK5fNvsfT+SLINHEQRJN8=
D-12533UDMG Agent ClientFixes a memory leak where the Client was opening unnecessary connections towards the UDMG Server. The connections are opened only when needed after a request from the UDMG Agent Server for a session forwarding and properly closed at the end. 
B-19917UDMG Authentication ProxyImproves security between UDMG Admin UI and UDMG Authentication Proxy with the use of session ID instead of credentials after the initial login (secured the JWT token).
B-19918,
B-19919,
B-19920,
B-19921
UDMG Admin UI, NGINX

Improves security with a default NGINX configuration including the recommended security HTTP headers, hiding the server version, and disabling weak TLS ciphers.

The following changes are now the default for the manual installation procedure and the installation with Linux packages (RPM/DEB). For upgrades, the parameter must be manually reviewed and added to the NGINX configuration file.

  • HTTP response headers: Content-Security-Policy, X-Content-Type-Options, Referrer-Policy, Strict-Transport-Security, Permissions-Policy, X-Permitted-Cross-Domain-Policies, X-Frame-Options. X-XSS-Protection
  • Minimum TLS version 1.2
  • Recommended cipher list from https://ssl-config.mozilla.org
  • Hide NGINX version
#32516UDMG Server/Client

Adds the display and management of the Audit permission for UDMG users. The permission grants the ability to list and view the audit records. The shortcode for the Audit permission on the CLI is 'a'.

 udmg-client example: click here to expand...
$ udmg-client user update wsadm -r "a=rwd" 
The user wsadm was successfully updated.

$ udmg-client user get wsadm
● User: wsadm
    User groups: Generic
    Email:       
    Login Type:  local
    Permissions:
    ├─Transfers:      r--
    ├─Servers:        r--
    ├─Partners:       r--
    ├─Rules:          r--
    ├─Users:          r--
    ├─Administration: r--
    ├─Pgp:            r--
    └─Audit:          rwd
#35978UDMG Server

Shows a summary of the UDMG Server disk usage on the api/status endpoint.

New "Storage" service:

  • "state" is "Running" or "Error" when used percentage is more than 90%.
  •  "reason" is "<used storage, in human size> (<ratio used/total in percent)" and only reports the filesystem for GatewayHome path.
 API examples: click here to expand...
GET /api/status
{
    "Admin": {
        "state": "Running",
        "reason": "Release 2.0.0" 
    },
    "Storage": {
        "state": "Running",
        "reason": "213.58 GB (16%)" 
    },
    
GET /api/status
{
    "Admin": {
        "state": "Running",
        "reason": "Release 2.0.0" 
    },
    "Storage": {
        "state": "Error",
        "reason": "25.0 GB (96%)" 
    },  

#36459UDMG Server

Displays the udmg-server instance node Id on the api/status endpoint, under the Node service.

For instance: gateway_1:8080-mft-gw-0

 API examples: click here to expand...
GET /api/status
{
    "Admin": {
        "state": "Running",
        "reason": "Release 2.0.0.0"
    },
    "Controller": {
        "state": "Running",
        "reason": "ACTIVE"
    },
    "Database": {
        "state": "Running",
        "reason": ""
    },
    "License": {
        "state": "Running",
        "reason": ""
    },
    "Node": {
        "state": "Running",
        "reason": "gateway_1:8080-mft-gw-0"
    },
    "Storage": {
        "state": "Running",
        "reason": "5.8 GB (37%)"
    },
    "local_auth_test_server": {
        "state": "Running",
        "reason": ""
    },
    "test_ftp": {
        "state": "Offline",
        "reason": ""
    }
}
#34778UDMG ServerExport/import: supports the user email field.
#35132UDMG Server

Migrate command: improves log message; the list command shows the current and the target versions with the [DATABASE] and [DEFAULT] tags.

 CLI examples: click here to expand...
$ /opt/udmg/bin/udmg-server migrate -c /opt/udmg/etc/udmg-server/server.ini --list
1.5.0 [DATABASE]
2.0.0 [DEFAULT]
#34502UDMG Client

Adds option and environment variable UDMG_COLOR to disable the color output.

 CLI syntax: click here to expand...
Output Options:
      --color=[always|auto|never] Control color output (default: auto) [$UDMG_COLOR]
#35917UDMG Server/Client

Adds new timeout options:

  • New udmg-server configuration parameter:

 Configuration Section: click here to expand...
[database]
; Threshold before warning for long-running queries, the default is 10 seconds
WarningTimeout=10s
  • This will only report a warning in the udmg-server log.
 Log example: click here to expand...
[WARNING ] Database: transaction is taking an unusually long time, printing stack for debugging purposes:


  • New udmg-client connection option:
 CLI syntax: click here to expand...
-t, --timeout= Client Connection Timeout (default: 30) [$UDMG_CLIENT_TIMEOUT]


  • This will report an error on CLI after the timeout.
 CLI error message: click here to expand...
an error occurred while sending the HTTP request: ...: context deadline exceeded
#34948UDMG Admin UI

Adds "Cluster Nodes" service to the UDMG Admin UI navigation pane with the list of the UDMG instances. 

B-18766UDMG Admin UI

Adds the local server "Status" column to the Local Servers list. The change makes it easier to view server status without having to open the UDMG Server status pop-up for quick monitoring.

#34488UDMG Admin UI, 
UDMG Server

FTP Server: adds support for the REMOVE and RENAME commands.

Each command can be enabled or disabled individually for each local FTP/FTPS/FTPES server.

#35133UDMG ServerFixes the panic error when the application is terminated during the startup sequence, while the node status is being determined.
#32532UDMG ServerFixes the insertion of the audit record for the transfer retry command. 
#33370UDMG ServerAdds additional language to error message for invalid JSON syntax in API payload. 
#35148UDMG Server

Adds additional language to error messages for the duplicate assignment of an object to a Business Service. 

the <object appellation> '<object name>' (ID '<object id>') is already is a member of the '<target name>' <target appellation>

#35244UDMG Server

Adds additional language to error messages for unknown Business Services:

 CLI error message: click here to expand...
$ udmg-client sb_biz member no_bs add partner atmoz_partner
business service 'no_bs' not found
#34817UDMG Server

Fixes the error message when setting an object name or login value that is too long. It was reported as a database error and now displays as: 

name cannot contain more than 100 characters

#34412UDMG ServerFixes the validation for host-based authentication on the SFTP server configuration. The local and remote account names must be the same.
#34567UDMG ServerFixes a panic error when restarting a local PeSIT server.
#36137UDMG ServerFixes a panic error during transfer post-task when the serviceName parameter of an ICAP task does not match the service name on the ICAP server.
#34965UDMG Server

Fixes the missing banner in the FTP* server welcome message (220 response).

 CLI example: click here to expand...
$ curl -v ftp://localhost:4100
*   Trying 127.0.0.1:4100...
* Connected to localhost (127.0.0.1) port 4100 (#0)
< 220 UDMG-FTP-2.0.0.0
#35147UDMG ServerFixes the possibility of creating a user group with an assigned Business Service. The Generic Business Service is enforced as the default value.
#35641UDMG Server

Adds the missing option for the udmg-server import command to select and process only the users from an export file.

 CLI syntax: click here to expand...
$ /opt/udmg/bin/udmg-server export -h
Usage:
  udmg-server [OPTIONS] export [export-OPTIONS]

Help Options:
  -h, --help                                                                                       Show this help message

[export command options]
      -c, --config=                                                                                The configuration file to use
      -f, --file=                                                                                  The destination file. If none is given, the content of the export will be written to the standard output
      -t, --target=[rules|servers|partners|users|business-services|user-groups|local-accounts|all] Limit the export to a subset of data. Can be repeated to export multiple subsets. (default: all)
      -v, --verbose                                                                                Show verbose debug information. Can be repeated to increase verbosity

$ /opt/udmg/bin/udmg-server import -h
Usage:
  udmg-server [OPTIONS] import [import-OPTIONS]

Help Options:
  -h, --help                                                                        Show this help message

[import command options]
      -c, --config=                                                                 The configuration file to use.
      -s, --source=                                                                 The data file to import. If none is given, the content will be read from the standard output.
      -t, --target=[rules|servers|partners|users|business-services|user-groups|all] Limit the import to a subset of data. Can be repeated to import multiple subsets. (default: all)
      -d, --dry-run                                                                 Do not make any changes, but simulate the import of the file.
      -v, --verbose                                                                 Show verbose debug information. Can be repeated to increase verbosity.
      -r, --reset-before-import                                                     Empty the database tables before importing the elements from the file. Cannot be used without the -s option.
          --force-reset-before-import                                               Empty the database tables before importing the elements from the file without confirmation prompt.
#35639UDMG Server

Fixes the missing revoked field from the get certificate details API.

 API example: click here to expand...
GET /api/sb_servers/:local_server_name/certificates/:cert_name
{
    "id": 30,
    "name": "TestKey",
    "revoked": "2024-01-25T12:55:25.898261Z",
    "privateKey": "-----BEGIN PRIVATE KEY-----\nMIIJQgIBADA...----END PRIVATE KEY-----" 
}
#35608UDMG Server

Fixes the confusion between sb_pgp and pgp as object types in the udmg-client commands. The sb_pbp is used for all actions involving the PGP records.

 CLI syntax: click here to expand...
Usage:
  udmg-client [GLOBAL-OPTIONS] sb_biz member [business-service] add [object-type] [object-name] [direction]

Connection Options:
  -a, --address=                  The address of the UDMG Server [$UDMG_SERVER_ADDRESS]
  -i, --insecure=                 Skip certificate verification [$UDMG_SERVER_INSECURE]
  -l, --legacy=                   Use legacy API version [$UDMG_SERVER_LEGACY]
  -t, --timeout=                  Client Connection Timeout (default: 30) [$UDMG_CLIENT_TIMEOUT]

Output Options:
      --color=[always|auto|never] Control color output (default: auto) [$UDMG_COLOR]

Help Options:
  -h, --help                      Show this help message

[member command arguments]
  business-service:               The business service's name

[add command arguments]
  object-type:                    The object's type. [server|partner|rule|sb_pgp]
  object-name:                    The object's name
  direction:                      The rule's direction (required only for rule objects)
#35165UDMG Server

Fixes to prevent the update of a local server protocol type.

 CLI error message: click here to expand...
$ udmg-client server update my_ftp_server -p sftp 
the protocol cannot be updated after creation
#35875,
#34210
AllUpgrade of the Go compiler version from 1.18 to 1.21. Switch to the Microsoft build of the Go toolset for better support of FIPS 140-2. 
#35952UDMG Agent ProxyLinux installation packages configure the client API default port to 2280 instead of 8080. This was already the recommended value for the manual installation procedure.
#35232UDMG Authentication ProxyUpdates documentation to reflect the change of the default port from 5000 to 5775.
#35538UDMG Server

Linux package names now have 4 digits for the version number.  

For example, udmg-admin-ui-2.0.0.0.build.99.x86_64.rpm

#36052UDMG ServerFixes the ownership of the log folder after the Linux package installation. The ownership of the log folder is now set to udmg:adm.
#35963UDMG ServerFixes the error message when a local server log file cannot be opened.
#34828UDMG ServerFixes the error message to include the temporary path when the temp file for an inbound file transfer cannot be opened.
#35786UDMG ServerFixes the occurrence of error messages in the UDMG Server log regarding the "superfluous call" from the REST API handler.
#34904UDMG ServerFixes the duplicate messages for the received list commands on the SFTP local server logs.
#34955UDMG Server

Adds a new error message during the SFTP partner keyscan command when the remote host is not reachable.

CommandError(CeInternal): failed to dial host: dial tcp <ip:port>: connect: connection refused
#35264UDMG ServerFixes the internal database error that was raised when attempting to delete a user with an open session. The user session is dropped, and the user is disconnected after the timeout.
#35273UDMG Server

Fixes the error message for deletion of a user group with members.

 CLI error message: click here to expand...
$ udmg-client_b sb_usr delete  my_user_group
unexpected error: user group cannot be deleted because of objects associated to it
#35376UDMG Server

Fixes the name of the fields in the personal access token API payload. The name of the fields is now in camel case.

 API example: click here to expand...
GET /api/sb_users/:user/tokens
{
  "user_tokens": [
    {
      "id": 5,
      "name": "token_02",
      "creationDate": "2024-01-10T08:59:21.899845Z",
      "expirationDate": "2024-02-01T08:04:05Z"
    }
  ]
}
#34832UDMG ServerRemoves the deprecated root, inDir, outDir, workDir fields from REST API schemas and CLI command arguments.
#35490UDMG Authentication ProxyFixes the invalid API sample response field in the Swagger UI.
#35958NGINXFixes the 'unknown log format main' after Linux Debian installation. The 'main' log format is disabled on the NGINX package from the Debian/Ubuntu repository unlike the official NGINX repository or the RHEL/Rocky/CENTOS RPM packages. The log format is reverted to the default ('combined') and is left to the administrator to enable a more detailed log format.
#35276UDMG Client

Adds a colon in the list output to separate the object type and the object name.

 CLI example: click here to expand...
$ udmg-client server list | grep Server
Servers:
● Server: "ftp_test_server" [Enabled]
● Server: "ftpes_test_server" [Enabled]
#35301UDMG Server

Export/import: prevents the reset of the optional fields.

  • Introduces the 'optional' field concept to import/export. The fields are included in the backup file only if they have a non-zero value during exports. The fields keep their pre-existing value if not present in the source file during imports.
  • Servers

    • Updates the RootDir, ReceiveDir, SendDir and TmpReceiveDir fields to optional.
  • Rules

    • Updates the LocalDir, RemoteDir and TmpLocalRcvDir fields to optional.
  • Users

    • Updates the Email, Login, TOTPSecretKey and TOTPRegistered fields to optional.
  • Business Services

    • Removes the ID field from the backup file.
    • Removes the 'omitempty' tag from the Name field.
    • Updates the Description field to optional.
  • User Groups

    • Removes the ID field from the backup file.
    • Removes the 'omitempty' tag from the Name and BusinessServiceID fields.
    • Updates the Description field to optional.
#35247UDMG Server

Ends the support for releases before 1.2.1.1 (0.7.1-sb.3). Migration from older releases is not supported anymore.

 CLI example: click here to expand...
$ /opt/udmg/bin/udmg-server migrate -c /opt/udmg/etc/udmg-server/server.ini --list
0.7.1-sb.3
1.3.0
1.3.1
1.3.2
1.4.0
1.4.1
1.5.0
2.0.0 [DATABASE, DEFAULT]
#36449UDMG ServerFixes the initialization failure when using a non-default database schema for MS SQL Server.
failed to init the database: command failed: mssql: The object 'dbo.local_agents' does not exist or is invalid for this operation.
#35752UDMG Server

Improves the "internal database error" message with the addition of the actual message from the database.

 Log example: click here to expand...
[ERROR   ] Admin: Unexpected error: internal database error: Error 1054 (42S22): Unknown column 'local_agent_id' in 'field list'

#36609UDMG ServerFixes the log level for database errors that are retried. A warning message appears instead of an error message.
#35359UDMG Admin UIFixes the listing of user sessions, the 100 most recent sessions are now displayed, instead of the 20 oldest. 
#36602 UDMG Server

Set the default network interface for UDMG Server to 0.0.0.0 instead of localhost.

 Configuration Section: click here to expand...
[admin]
; The address used by the admin interface.
Host = 0.0.0.0
#40910, #33566UDMG Admin UIFixes the autocompletion in username and password field by the browser on the new User and new Shared Account screens.
#36208UDMG ServerFixes an error condition with the FTP server: when receiving a file with a subdirectory path, it was not created. The server was only creating the necessary folders up to the rule local path.
#36701UDMG ServerFixes the import of UDMG users from the command line. When loading a user without specifying the usergroup membership, it is created and useable (connect, get, update, delete) but not listed (udmg-client user list). The default usergroup is now assigned for the user in this case.
#36839UDMG Server

Fixes the list of authorized transfer rules in the account endpoints 

Changes on UDMG Server API:

The available virtual paths for a given account are a combination of:

1) the rules that are authorized for this account,
2) the rules that are authorized for the server or partner,
3) the global rules, that are not assigned to any account, server, or partner (unless explicit assignment is enforced).

The list of server or partner rules was incorrect.

GET /api/sb_local_accounts
GET /api/sb_remote_accounts

Changes on UDMG Admin UI:

The rules at server/partner and account level are marked as private in the account Rules tab. 

#36816UDMG Web Transfer Client

Fixes the list of folders on the dashboard, the virtual paths for rules that were assigned at the local-auth server level were not displayed.

The available folders are selected by :

1) the rules that are authorized for this account,
2) the rules that are authorized for the server,
3) the global rules, that are not assigned to any account, server, or partner (unless explicit assignment is enforced).

#36799UDMG Server

Improves the logging of API calls to the UDMG Server.

TrackID information was only shown when the rate limiter was enabled, it is now always displayed along with the remote IP address and the user agent. 

 Log example: click here to expand...

Previous format:

2024/05/14 09:09:27 [INFO ] Admin: Request GET: /api/transfers
2024/05/14 09:09:27 [INFO ] Admin: TrackID: 50bf24fa-0221-4344-bf00-4d0e6b0f04ac - Received GET on /api/transfers?sort=start-&limit=1000&offset=0&start=2024-05-13T09%3A09%3A27.277Z

New format:

2024/05/23 18:42:26 [INFO ] Admin: ID='Direct Request' Method='GET' URL='/api/status' RemoteAddr='192.168.10.72' UserAgent='curl/8.7.1'
#35194UDMG Authentication Proxy

Improves the logging of HTTP requests on the UDMG Authentication Proxy with the addition of the remote IP and the user agent.

 Click here to expand...
level=error TS=2024-06-05T15:21:06.424084373Z Service=HTTP Request="Method='POST' URL='/auth/local' RemoteAddr='185.70.76.140' UserAgent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36'" Error="code=401, message=authentication error"
#36862UDMG ServerAdds a 1-minute timeout to publish event task (PUBLISHEVENT).
#35530UDMG Admin UI

Fixes the inability to see and apply business service filters for the Local/Shared Account list. 

and the Remote Accounts list:

#33109UDMG Agent ClientAdds OpenAPI Swagger /swagger/index.html endpoint on the UDMG Agent Client.
#33106UDMG Agent Client

Adds the option to connect with HTTPS to the UDMG Server API.

New secure option in the client config file, set to true to select https, false by default.

 Configuration Section: click here to expand...
[gateway]
# UDMG Server Hostname or IP, and port
hostname = "localhost" 
port = "18080" 
# UDMG Server Username/Password
username = "admin" 
password = "admin_password" 
secure = true

#40782UDMG Server

Shows the UDMG Server version with HEAD /api/status as a hint for integrators to easily know the API version with a low-impact request.

 API example: click here to expand...
curl -L -I "http://user:password@udmg.example.com:8080/api/status" 
HTTP/1.1 200 OK
server: UDMG/2.0.0
date: Wed, 29 May 2024 10:25:50 GMT
#36085UDMG Web Transfer ClientFixes the mkdir error when the server root directory has more than 1 level and was not created beforehand. The module is now able to create nested local directories.
#33993UDMG Agent Proxy Client

Renames the configuration section for the target UDMG Server from [gateway] to [server]