Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Panel

Table of Contents

Universal Data Mover Gateway release 2.0.0.0 - June 7, 2024, contains the following high-level features. For a complete list of all the included features and fixes, please refer to Universal Data Mover Gateway 2.0.x Maintenance.

File Transfer Enhancements

Backlog

Title

Description

B-18770

Ad-hoc File Transfers

This enhancement allows the transfer of files between users using a quick share link for a given file, which is protected with a password, expiration date, and number of downloads. The links and associated files can be shared with any type of user, including external users as long as the user is provided the password. Each file for which an ad-hoc file transfer link is generated is given a unique ID/token used in the associated URL.

All ad-hoc file transfer links are generated through the Web Transfer Client and managed through the Web Transfer Client and UDMG Admin UI. The user must have a local auth account and a local server in UDMG must be created with the local auth protocol attached. 

Note: Due to a system limitation with the UDMG Web Transfer Client, the authentication server for the UDMG Web Transfer Client must have the same path value set for both the Local and Temp directoriesThis constraint will be removed in future releases.

For the creation of ad-hoc file transfer links, the following fields are required:

  • Selected File / File Path
  • Expiration Date (number of days or unlimited)
  • Max Downloads (number of downloads or unlimited)
  • Password (optional)

For all generated ad-hoc file transfer links, the following actions are available:

  • Extend: Updates the expiration date. The expiration is removed by clearing the date value, this is then shown as unlimited. 
  • Revoke: Disables the live link and makes it unusable. It can be later reactivated with the Extend action. 
  • Copy Link to Clipboard: Copies the URL.

See Tutorial - Creating an Ad-hoc File Transfer Link for detailed instructions on how to create, edit, and manage ad-hoc file transfer links.

Updates to the local-auth server configuration:

  • New flag to enable ad-hoc sharing feature.
  • New parameter for the public-facing hostname and port. This is required when the feature is on as it is a part of the generated link URL.


Web Transfer Client

UDMG Admin UI

From UDMG Admin UI, ad-hoc file transfer information is visible in the Transfers Activity Dashboard and Ad-Hoc Activity service via the navigation pane. 

Transfers Dashboard 

Ad-Hoc Activity 

New endpoints added on the UDMG Server API: 

Note that the creation of the share link is only possible from the UDMG Web Transfer Client.

The following endpoints allow the management of the existing links.

  • GET /api/sb_adhoc_shares : list the ad-hoc share links. The Administration Read permission (PermAdminRead) is required.
  • GET /api/sb_adhoc_shares/:token retrieve specific link details. The Administration Read permission (PermAdminRead) is required.
  • PUT /api/sb_adhoc_shares/:token/extend: updates the expiration date of the link or removes the expiration date, setting it as unlimited. The Administration Write permission (PermAdminWrite) is required. If the link was disabled after a revoke action (REVOKED status), or expiration (EXPIRED status), it is reactivated (ACTIVE status), If the link was disabled after reaching the maximum number of downloads (DONE status), it cannot be reactivated.
  • PUT /api/sb_adhoc_shares/:token/revoke (PermAdminWrite): deactivates the link. The expiration date and maximum number of downloads are not affected and the link can be reactivated with the extend command. The Administration Write permission (PermAdminWrite) is required.
  • DELETE /api/sb_adhoc_shares/:token : deletes the link entry from the system. It cannot be restored. The Administration Write permission (PermAdminWrite) is required.

Updates to the udmg-client CLI: 


Expand


Code Block
languagetext
Usage:
  udmg-client [GLOBAL-OPTIONS] adhoc_share <command>

Help Options:
  -h, --help                      Show this help message

Available commands:
  delete  Delete an adhoc share
  extend  Extend the expiration time of an adhoc share
  get     Retrieve an adhoc share's information
  list    List the known adhoc shares
  revoke  Revoke the usage of an adhoc share


 

B-18765

Delete file after download from SFTP (MOVE Command)

This enhancement allows the deletion of a file after it is downloaded from SFTP using the MOVE command. The enhancement only applies to send or receive files. The change removes the need for additional configuration and monitoring in UAC and UDMG.

Before this change, the local file could only be deleted in the UDMG Server with the DELETE post-task. The remote file could not be deleted in UDMG and UAC was required for the transfers where remote delete was required. 

  • New transfer info flag udmg_xfer_move, can have values true and false (bool). If set to true, the customer will delete the file being sent during the removepipeline step.
    • udmg_xfer_move flag is checked for accepted values when the transfer is created.
    • udmg_xfer_move flag is checked for protocol implementation when the pipeline is created. This is only supported for SFTP.
  • New pipeline step: remove, between data and post tasks.
  • Local file is removed after transfer for send rule (PUT).
  • Remote file is removed after transfer for receive rule (GET).

Examples:

Expand


Code Block
udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1"  -w receive -i udmg_xfer_move:true

udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1S" -w send -i udmg_xfer_move:true



Implemented Error Messages:

  • TeInternal: move not supported for this protocol
  • TeInternal: failed to remove local file: %s
  • TeDataTransfer: Failed to remove remote SFTP file: %s
#35208UDMG Server

Adds a parameter to disable the implicit assignment of Transfer Rules. Before this change, a Transfer Rule was allowed for all servers, partners, and accounts right after creation. It was only restricted after an explicit assignment (whitelisting) to at least one server, partner, or account. The prior functionality created confusion for UDMG users and allowed unauthorized access to files and folders for third-party accounts during the time between the rule creation and its explicit assignment. The change allows the functionality to be disabled with the new UDMG Server configuration parameter ExplicitRuleAssignment. When set to true, a Transfer Rule is only effective when it is explicitly linked with the intended target (local server, remote partner, remote or local account).

The default value remains false but is likely to change with future releases.

Expand


Code Block
[rule]
; Disables global rules, requiring rules to be explicitly allowed to be used.
; ExplicitRuleAssignment = false



...

Backlog

Title

Description

B-19917Updates to Secure JWT Token

This enhancement improves the security between UDMG Admin UI and UDMG Authentication Proxy with the use of session ID instead of user credentials after the initial login (secured the JWT token). Before this change, the user credentials were stored in the JWT payload, which exposed it to potential unauthorized access.  

The password was removed from the JWT Token and now the header 'X-Session-Id' must be included in each request.

B-19918,
B-19919,
B-19920,
B-19921



Security Hardening


Improves security with a default NGINX configuration including the recommended security HTTP headers, hiding the server version, and disabling weak TLS ciphers. The following changes are now the default for the manual installation procedure and the installation with Linux packages (RPM/DEB). For upgrades, the parameter must be manually reviewed and added to the NGINX configuration file.

Note: When installing UDMG 2.0 a new configuration file is generated with "new" added at end.

  • B-19918: Hides NGINX Version
    • Changes improve defenses against specific security vulnerabilities by removing the NGINX server banner information (technical and detailed web server version information in HTTP response header) from displaying on every server response.
  • B-19919: Improves Clickjacking Prevention
    • To improve malicious Clickjacking attacks, the following enhancements were implemented:

      • Prevents the browser from loading the page in frame using the X-Frame-Options or Content Security Policy (frame-ancestors) HTTP headers.

      • Prevents session cookies from being included when the page is loaded in a frame using the SameSite cookie attribute.

      • Implements JavaScript code in the page to attempt to prevent it from being loaded in a frame (known as a "frame-buster").

  • B-19920: Security Headers
    • The following HTTP response headers were added to provide an additional layer of security to Universal Data Mover Gateway.
      • HTTP response headers: Content-Security-Policy, X-Content-Type-Options, Referrer-Policy, Strict-Transport-Security, Permissions-Policy, X-Permitted-Cross-Domain-Policies, X-Frame-Options. X-XSS-Protection
Expand


Code Block
# DEPRECATED Security Headers
    add_header X-XSS-Protection "0";
    add_header X-Frame-Options "SAMEORIGIN";
    # Security Headers
    add_header Content-Security-Policy "frame-ancestors 'self'";
    add_header X-Content-Type-Options nosniff;
    add_header Referrer-Policy "strict-origin";
    add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains; preload';
    add_header Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()";
    add_header X-Permitted-Cross-Domain-Policies none;


 

  • B-19921: Disables Weak TLS Ciphers
    • Changes improve defenses against LUCKY13, a security vulnerability that specifically targets certain implementations of the TLS (Transport Layer Security) and DTLS (Datagram Transport Layer Security) protocols when they use the CBC mode with block ciphers for encryption, by disabling the CBC mode cipher encryption and enabling CTR or GCM cipher mode encryption. 
    • Recommended cipher list from https://ssl-config.mozilla.org
    • Minimum TLS version 1.2

...

Backlog

Title

Description


User Interface Updates 

This enhancement improves the usability of the UDMG Admin UI, allowing for easier access to information, customizations, searching/filtering, and new features. The changes align with the look and feel of Universal Automation Center.

Key Changes Include: 

Landing Page / Homepage

  1. Updated icons across the UDMG Admin UI navigation pane and User Task Bar.
  2. UDMG Admin UI navigation pane and pages are expandable and collapsible.
  3. Rearranged and added new UDMG Admin UI navigation pane services.
  4. Console details are accessible via the bell icon on the User Task Bar. The details are displayed in a pop-up box at the bottom of the page. The pop-up box must be closed out before returning to the page. 

Transfers Activity

  1. Transfer details are viewable after a file transfer record is selected.
  2. Filters in the Transfer Activity Dashboard are not pre-selected.

License

  1. Added a new service in the UDMG Admin UI navigation pane for applying and managing the UDMG license. See B-18755 for additional details. 

Ad-hoc Activity

  1. Added a new service in the UDMG Admin UI navigation pane to view the list of ad-hoc file transfer links generated and associated details

Local Servers

  1. Updated Server Details tabs:
    1. Consolidated the Server and Configuration tabs into one Server tab.
    2. Reordered tabs to align with the Universal Automation Controller order.
  2. Added UDMG Server "Status" column in the local server list. See B-18766 for additional details. 
  3. Removed open text search and added filtering functionality at the column level.

Shared Accounts 

  1. Updated service name from Local Accounts to Shared Accounts in UDMG Admin UI navigation pane. See B-18759 for additional details. 
  2. Updated the Accounts tab:
    1. Added the ability to add any created shared account to the server. See B-18759 for additional details. 
    2. Added the ability to disable a shared account. The account can be disabled without requiring to delete it or unlink it from Local Servers.

Rules

  1. Updated Rules Details tabs:
    1. Consolidated Rules, Pre-Tasks, Post-Tasks, Error Tasks, and Configuration tabs into one Rule tab.
    2. Added real-time error checking to editor text boxes.

Cluster Nodes

  1. Added a new service to view the list of UDMG Server instances.
B-18766Added Server Status and Information in Details

This enhancement allows the user to quickly view the server status, including server state and information in the Local Servers list for all local servers. The "Status" column was added to the list of local server details.  Before this change, the local server status details were only displayed in the “UDMG Server Status” popup.

The color-coded bars show the internal service status:

  • Green: Service is operational and is running.
  • Red: Service is failing and reported an error.
  • Grey: Service is offline; the service was not started because it was disabled, it was stopped, and it has not yet been started because the UDMG Server is not in an ACTIVE state.
  • Light Green: Service is starting up.
  • Black: Service is shutting down.

B-19796

SSH Key Parsing Tool

This enhancement provides a tool to parse an SSH public key. The UDMG Server only accepts the OpenSSH format for public keys, so customers who use the PKCS8 and DER format were not previously able to convert keys to OpenSSH using standard tools (openssl and ssh-keygen). 

The udmg-sshkey tool enables the conversion from another format to the OpenSSH format.

The supported input formats are:

  • OpenSSH authorized key line
  • PEM encoded formats (PKCS8, PKCS1)
  • SSH wire format (Binary ASN1 DER)

The output of the tool shows the key details and the OpenSSH format that is suitable for use during the setup of SFTP servers on UDMG:

  • Key format
  • Key algorithm
  • Key size (only for RSA)
  • SHA256 fingerprint
  • MD5 fingerprint
  • Conversion to authorized key line format

Command line usage:

Expand


Code Block
udmg-sshkey -h
Usage:
  udmg-sshkey [OPTIONS] <parse | version>
Help Options:
  -h, --help  Show this help message
Available commands:
  parse    Parse an SSH Public key file
  version  Print version and exit
   
udmg-sshkey parse -h
Usage:
  udmg-sshkey [OPTIONS] parse [parse-OPTIONS]
Help Options:
  -h, --help      Show this help message
[parse command options]
      -f, --file= The public key file, accepted formats are PKCS8 (x509), OpenSSH (authorized keys), and SSH wire.



See Utilities Reference Guide: udmg-sshkey for additional details.

...

Backlog

Title

Description

B-18759

Allowing the Sharing of Accounts Between Servers

This enhancement allows the sharing of accounts between servers. It avoids duplicate configurations for FTP and SFTP local servers by allowing accounts to be shared between local servers.

Before this change, a local account was only defined for a given local server and not allowed to have the same account for multiple protocols. For instance, to allow a partner to transfer files over SFTP and FTP, it was required to have both an SFTP and FTP local server, each with their own local account. Each account was then maintained independently, which created additional overhead in configuration and maintenance (password or key updates).

The Local Account service located in the UDMG Admin UI navigation pane is replaced by Shared Accounts service. 

Image Modified

Updated Shared Account process:

  1. Click on Shared Accounts from UDMG Admin UI navigation pane.
  2. Create a Shared Account by populating the required details and following the current process (tabs are the same).  
    1. Note: A local server does not have to be created yet.
    2. Note: If a local server has been created, then click on the Local Servers tab. 
      1. Click the Pencil (edit) icon. 
      2. Add and/or remove the shared accounts via the pop-up box.

  1. Click on Local Servers from UDMG Admin UI navigation pane.
  2. Create a local server by populating the required details and following the current process OR select a Local Server record. 
  3. Click the Accounts tab.
  4. Click the Pencil (edit) icon.
  5. Add and/or remove the shared accounts via the pop-up box. 
    1. Note: All created shared accounts are available to be add to the local server.

New permission for the management of Shared Accounts:

sharedAccounts on User Permissions

and user group Permissions

New endpoints added on the UDMG Server API: 

  • GET /api/sb_local_accounts
  • POST /api/sb_local_accounts
  • GET /api/sb_local_accounts/:account
  • DELETE /api/sb_local_accounts/:account
  • PUT /api/sb_local_accounts/:account
  • PATCH /api/sb_local_accounts/:account
  • PUT /api/sb_local_accounts/:account/enable
  • PUT /api/sb_local_accounts/:account/disable
  • PUT /api/sb_local_accounts/:account/allow/:server
  • PUT /api/sb_local_accounts/:account/disallow/:server
  • GET /api/sb_local_accounts/:account/certificates
  • PUT /api/sb_local_accounts/:account/certificates/:cert/revoke
  • PUT /api/sb_local_accounts/:account/certificates/:cert/revoke
  • PUT /api/sb_local_accounts/:account/certificates/:cert/authorize
  • PUT /api/sb_local_accounts/account/authorize/:rule/receive
  • PUT /api/sb_local_accounts/account/authorize/:rule/send
  • PUT /api/sb_local_accounts/account/revoke/:rule/receive
  • PUT /api/sb_local_accounts/account/revoke/:rule/send

Updates to the udmg-client CLI: 

Both the new and old command lines can be used to add a new shared account. 

New CLI commands:

Expand


Code Block
udmg-client account share add -l user -p pass -n user



Current CLI commands: 

Expand


Code Block
udmg-client account local $SERVER add -l user -p password




Note

If the old command line is used, then the command line automatically creates a shared account and assigns it to the local server.



The permission code for udmg-client is 'L'

Expand


Code Block
languagetext
$ udmg-client_20 user get bob
● User: bob
    User groups: Generic
    Email:
    Login Type:  local-otp
    Permissions:
    ├─Transfers:       ---
    ├─Servers:         ---
    ├─Partners:        ---
    ├─Rules:           ---
    ├─Users:           rwd
    ├─Administration:  ---
    ├─Pgp:             ---
    ├─Audit:           ---
    └─Shared Accounts: ---

$ udmg-client_20 user update bob -r 'Lrwd'
The user bob was successfully updated.

$ udmg-client_20 user get bob
● User: bob
    User groups: Generic
    Email:
    Login Type:  local
    Permissions:
    ├─Transfers:       ---
    ├─Servers:         ---
    ├─Partners:        ---
    ├─Rules:           ---
    ├─Users:           rwd
    ├─Administration:  ---
    ├─Pgp:             ---
    ├─Audit:           ---
    └─Shared Accounts: rwd




Updated CLI Commands:

Expand


Code Block
Usage:
  udmg-client [GLOBAL-OPTIONS] account share <command>

Help Options:
  -h, --help                      Show this help message

Available commands:
  add        Add a new shared account
  allow      Allow a shared account to use a Local Agent
  authorize  Authorize a shared account to use a rule
  cert       Manage a share account's certificates
  delete     Delete a shared account
  disable    Disable shared account
  disallow   Disallow a shared  account's permission to use a local Agent
  enable     Enable shared account
  get        Retrieve a shared account's information
  list       List shared accounts
  revoke     Revoke a shared account permission to use a rule
  update     Update a shared account


See Tutorial - Creating and Associating a Shared Account for additional details. 

B-18755

License Control

This change incorporates license management within Universal Data Mover Gateway. All environments are required to have an attached license key, including non-production environments.  The Universal Data Mover Gateway is licensed for a set time period, number of UDMG Server instances, and number of transfer executions per month and environment. The counted file transfer executions only include the number of transfer instances completed as DONE over one month. 

To receive and apply your license keys, please follow the below steps and reference UDMG Licensing


Step 1Contact your Stonebranch representative or Customer Support to receive the license keys for the intended operating system and environment.
Step 2Navigate to the UDMG Admin UI.

Step 3

Click the License service in the UDMG Admin UI navigation pane. The License service allows admin users to enter and update license keys and view license details:

  • License Status: "Unlicensed" when no license has been entered, "Licensed" otherwise.
  • License Customer: name of the customer.
  • Environment: environment name (e.g., test, dev, non-prod, prod).
  • Expires On: the date when the license expires (at 00:00:00).
  • Nodes: the number of allowed cluster nodes, or unlimited.
  • Transfers: the number of allowed transfers per month, or unlimited.

Step 4

Input the license key and press the Update License button.

Step 5

After a few minutes, the UDMG Server status switches to ACTIVE mode (see node status).


B-18762

Using the Same Virtual Folder Name for Multiple User Accounts

This enhancement allows users to use the same virtual folder name for multiple user accounts. Multiple user accounts can now use the same local server and the same vpath, but each account will point to a separate ("private") local directory.

Note: The local directory must have read/write permission for the UDMG Server operating system user.


For the protocols: SFTP, FTP/E/S, local-auth

Placeholders are allowed in the transfer rule paths (local directory and temp directory):
#REQUESTERHOST# : local account name, for SFTP, local-auth, and FTP protocols.
#REQUESTEDHOST# : local server name, only for SFTP and local-auth.

They are expanded at runtime (when the transfer is starting or when an FTP/SFTP command is executed).

For instance, /home/Users/Input/#REQUESTERHOST# is expanded to differently for each account:

  • /home/Users/Input/sftp_user1 for account "sftp_user1"   
  • /home/Users/Input/sftp_user2 for account "sftp_user2".


Example:

Expand


Code Block
udmg-client rule add -n Rule10   -d receive -p /data-host        --local-dir '/data/#REQUESTEDHOST#'



#33083Harmonization of module usage

The UDMG Authentication Proxy, UDMG Agent Proxy Server, UDMG Agent Proxy Client, and the UDMG Web Transfer Client are updated with the addition of a mandatory command line option to start the modules, and the specification of the configuration file on the command line (it remains possible to use an environment variable for the same)

  • start the module: start [-f config_file]
  • check configuration file syntax: test [-f config_file]
  • show program version: version
#32518Harmonization of configuration file format

The UDMG Authentication Proxy supports TOML and YAML file formats for the configuration of the SSO service providers.

This is in addition to JSON format that is already supported. The main configuration file remains in TOML format.

Examples of SSO Google provider in the different formats:

Expand

JSON

{
    "issuer": "https://accounts.google.com",
    "clientID": "xxxxxxx-6le4dqr2cabn80bc9bel444q0cirnkom.apps.googleusercontent.com", "clientSecret": "GOCSPX-lhtwS4mtJ3ryKtDw9_xxxxxxx", "redirectURI": "https://mft-udmg-rhel-qa.stonebranch.org/service/auth/sso/google/callback" } 

TOML

issuer = "https://accounts.google.com" 
clientID = "xxxxxxx-6le4dqr2cabn80bc9bel444q0cirnkom.apps.googleusercontent.com" clientSecret = "GOCSPX-lhtwS4mtJ3ryKtDw9_xxxxxxx" redirectURI = "https://mft-udmg-rhel-qa.stonebranch.org/service/auth/sso/google/callback" 

YAML

issuer: https://accounts.google.com
clientID: xxxxxxx-6le4dqr2cabn80bc9bel444q0cirnkom.apps.googleusercontent.com clientSecret: GOCSPX-lhtwS4mtJ3ryKtDw9_xxxxxxx redirectURI: https://mft-udmg-rhel-qa.stonebranch.org/service/auth/sso/google/callback