Panel | |
---|---|
|
Universal Data Mover Gateway release 2.0.0.0 - May 31, 2024, contains the following high-level features. For a complete list of all the included features and fixes, please refer to Universal Data Mover Gateway 2.0.x Maintenance.
File Transfer Enhancements
...
Backlog
...
Title
...
Description
...
B-18770
...
Ad-hoc File Transfers
This enhancement allows the transfer of files between users using a quick share link for a given file, which is protected with a password, expiration date, and restriction on number of downloads.
uses the API from local auth protocol in the web transfer client – is this too internal;
to create the link/access in the Web Transfer Client to create is through authentication through local auth; any user (udmg user/external) can access the link if they have the password
need to have local server in udgm created with the local auth protocol – not new
*ldap authentication will be incorporated in future release.
pull in screenshot of page + fields +steps to create –
navg pane to ad-hoc activity - seeing all links created from the web transfer client; nothing can be created/edited
from wb transfer client ---
- New Fields:
- filename, for info
- size, for info
- the owner of a link (the local/shared account that created the link).
This is needed:
- for link maintenance, it will allow to show the created links to the owner and for admin to know who created the link...
- to inform the recipient - creation date (could be default automatic orm created_date column)
- list of recipients, possibly free text values that could be emails. we can start by supporting only 1 recipient
New end point
- Added new endpoint:
POST /link
. The endpoint is used to create a file link for a given remote path. The remote path will be computed to a local path during the link creation, and that local path will be stored along the link. Expiration date and file password are optional.
Body:
Code Block |
---|
{
"username":"user",
"password":"userpassword",
"remote_path":"path/to/filename.txt",
"expiration_date":"2030-01-01T00:00:00Z",
"file_password":"secret"
}
Response:
{
"file_link":"343bb79e-a476-459e-890f-32d34134612a"
} |
- Added new endpoint:
GET /link/{token}?password=secret
. The endpoint is used to download a given file link.- Status codes:
- 404: if the file link is not found
- 400: if the file link is expired
- 403: if the password is not correct
- 404: if the file does not exist in the server
- 500: if any internal error happens
- Status codes:
...
B-18765
...
Delete file after download from SFTP (MOVE Command)
This enhancement allows the deletion of a file after it is downloaded from SFTP using the MOVE command. only applies to send or receive files
Before this change, the local file could only be deleted in the UDMG Server with the DELETE post-task and a remote file could not be deleted in Universal Data Mover Gateway. For remote files, a customer had to use UAC for the transfers where remote delete is required. The prior process created additional customer overhead for configuration and monitoring (UAC + UDMG).
- New transfer info flag
udmg_xfer_move
, can have values true and false (bool). If set to true, the customer will delete the file being sent during theremove
pipeline step.udmg_xfer_move
flag is checked for accepted values when the transfer is created.udmg_xfer_move
flag is checked for protocol implementation when the pipeline is created. This is only supported for SFTP.
- New pipeline step:
remove
, betweendata
andpost tasks.
- Local file is removed after transfer for send rule (PUT).
- Remote file is removed after transfer for receive rule (GET).
Examples:
Code Block |
---|
udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1" -w receive -i udmg_xfer_move:true
udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1S" -w send -i udmg_xfer_move:true
|
Implemented Error Messages:
- TeInternal: move not supported for this protocol
- TeInternal: failed to remove local file: %s
- TeDataTransfer: Failed to remove remote SFTP file: %s
Security
...
Backlog
...
Title
...
Description
...
This enhancement improves the security between UDMG Admin UI and UDMG Authentication Proxy with the use of session ID instead of user credentials after the initial login (secured the JWT token). Before this change, the user credentials were stored in the JWT payload, which exposed it to potential unauthorized access.
The password was removed from the JWT Token and now the header 'X-Session-Id' must be included in each request.
B-19918,
B-19919,
B-19920,
B-19921
Security Hardening
Improves security with a default NGINX configuration including the recommended security HTTP headers, hiding the server version, and disabling weak TLS ciphers.
no impact; when installing the new 2.0 version a new configuration file is generated with"new" added at end file; no action/fyi
- B-19918: Hides NGINX Version
- Changes improve defenses against specific security vulnerabilities by removing the NGINX server banner information (technical and detailed web server version information in HTTP response header) from displaying on every server response.
- B-19919: Improves Clickjacking Prevention
To improve malicious Clickjacking attacks, the following enhancements were implemented:
Prevents the browser from loading the page in frame using the X-Frame-Options or Content Security Policy (frame-ancestors) HTTP headers.
Prevents session cookies from being included when the page is loaded in a frame using the SameSite cookie attribute.
Implements JavaScript code in the page to attempt to prevent it being loaded in a frame (known as a "frame-buster").
- B-19920: Security Headers
- The following HTTP response headers were added to provide an additional layer of security to Universal Data Mover Gateway.
- HTTP response headers: Content-Security-Policy, X-Content-Type-Options, Referrer-Policy, Strict-Transport-Security, Permissions-Policy, X-Permitted-Cross-Domain-Policies, X-Frame-Options. X-XSS-Protection
Resolution:
# DEPRECATED Security Headers add_header X-XSS-Protection "0"; add_header X-Frame-Options "SAMEORIGIN"; # Security Headers add_header Content-Security-Policy "frame-ancestors 'self'"; add_header X-Content-Type-Options nosniff; add_header Referrer-Policy "strict-origin"; add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains; preload'; add_header Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()"; add_header X-Permitted-Cross-Domain-Policies none;
- The following HTTP response headers were added to provide an additional layer of security to Universal Data Mover Gateway.
- B-19921: Disables Weak TLS Ciphers
- Changes improve defenses against LUCKY13, a security vulnerability that specifically targets certain implementations of the TLS (Transport Layer Security) and DTLS (Datagram Transport Layer Security) protocols when they use the CBC mode with block ciphers for encryption, by disabling the CBC mode cipher encryption and enabling CTR or GCM cipher mode encryption.
- Recommended cipher list from https://ssl-config.mozilla.org
- Minimum TLS version 1.2
User Experience
...
Backlog
...
Title
...
Description
...
Need to specify what pages/web
This enhancement improves usability of the UI, allowing for improved viewing of information, customizations, searching/filtering, and new features. Aligns look and feel and features to UAC.
DATA Dump per Demo – some might be not be things we want to document/highlight
Universal Data Mover Gateway Admin UI Changes
Landing Page
- Updated icon for dark/light mode
- Pages are resizable using XX
- Left Navigation Pane (idk what we call this throughout) is expandable and collapsible
- Rearranged Left Navigation Pane tabs and removed groupings
Left Sidebar Menu - Transfers tab
- Server details are only loaded in right panel after the selection of a server
- Filters in XX are not preselected
- Console information shown as pop-up message - requires pop-up to be closed out before continuing
- Consolidated Server and Configuration tabs into Server tab
- All fields are combined into one tab - improves server creation, saving, and error messaging process
- Reordered XXX tabs to align with UAC order
Left Sidebar Menu - Rules tab
- Consolidation of Rules, Pre-Tasks, Post-Tasks, Error Tasks, XXX Configuration tabs into XX tab
- Updated editor text functionality - real time error checking
Left Sidebar Menu - Shared Accounts tab
- Updated tab name from Local Servers to Shared Accounts (see B-18759 for detailed functionality and UI changes)
- Removed open text search
- Added filtering functionality for each column in xxxxx
- Added new server status column with XXX icon - ; The following color indicates:
- Red: server error
- Green: server healthy
- Gray: server offline
Left Sidebar Menu - Licensing
- Added new tab for adding licensing information (See B-18755 for detailed functionality and UI changes)
Left Sidebar Menu - Cluster Nodes
- Added new tab for viewing the list of UDMG Server instances
- Displays UDMG Server status and detailed information?
left nav pane - ad hoc file transfer
...
This enhancement allows the user to quickly view the server status, including server state and information in the Details tab for all local servers. Before this change, the local server status details were only displayed in the “UDMG Server Status” popup.
status column within the local servers from nav pane is new; prior you clicked the bottom thing and onlyhad pop up
...
B-19796
...
SSH Key Parsing Tool
...
This enhancement provides a tool to parse a SSH public key. The UDMG Server only accepts the OpenSSH format for public keys, so customers who use the PKCS8 and DER format were not previously able to convert keys to OpenSSH using standard tools (openssl and ssh-keygen). The tool parses a SSH public key and shows the following details:
Format
Algorithm
Key Size (for RSA)
SHA256 fingerprint
MD5 fingerprint
Conversion to authorized key line format
Supported formats are:
Authorized key (OpenSSH)
PEM encoded formats (PKCS8, PKCS1)
SSH wire format (Binary ASN1 DER)
See Utilities Reference Guide: udmg-sshkey
Architecture
...
Backlog
...
Title
...
Description
...
B-18759
...
Allowing the Sharing of Accounts Between Servers
This enhancement allows the sharing of accounts between servers. It avoids the need for duplicate configuration for FTP and SFTP local servers by having accounts that are shared between local server. (another option would be to allow a server to support multiple protocols)-????
Prior to this change, a local account was only defined for a given local server and it was not possible to have the same account for multiple protocols. For instance, to allow a partner to transfer files over SFTP and FTP, it was required to have both an SFTP and FTP server, each with its own account. Each account was then maintained independently which created additional overhead in configuration and maintenance (password or key updates).
The implementation uses the current Local Account table and a New table for handling authorization.
Be able to list share accounts from server endpoint.
- command line
- don't break integration with accounts
- add new shared account
- New: udmg-client account share add -l user -p pass -n user
- Old: udmg-client account local $SERVER add -l user -p password
- if use this command line then it automatically creates share account and assigns to the local server
- for new command line for shared account ; very similar to old one
...
UI Changes - Left Navigation Pane - Shared Accounts tab
- Updated tab name from Local Servers to Shared Accounts
Prior to this change, in order to create a local account, the user had to create a local server on the Local Server tab first and then create a local account within the XX tab. The user could then edit the local account from the Local Account tab. **A local account could not be created from the Local Account tab directly.
The enhancement changes the local account creation process. The user must create the local account from the Shared Accounts tab and does not require the local server to be created yet.
- Click Shared Accounts tab
- Create local account – need steps
- Add BS etc
- Click Local Servers tab
- Add local account to server? can add any of the shared accounts that have been created
add info about disable
...
B-18755
...
License Control
This change incorporates license management within Universal Data Mover Gateway. All instances of Universal Data Mover Gateway are now required to have a license key attached to the installation. Universal Data Mover Gateway is licensed by the number of executions per month*; specifically, the number of file transfer instances that completed as DONE over a period of one month.
is there any distinction in the transfer types? like all count or only incoming vs outgoing?
To receive and apply your license keys, please follow the below steps and reference UDMG Licensing.
...
Step 3
...
Click the License option in the sidebar menu. The License option in the sidebar menu identifies license information for:
- License Status
- License Customer
- Environment
- Expiration Date
- Cluster Nodes
- Monthly Transfers
...
Step 4
...
Input the license key and press the Update License button.
...
The display is refreshed with the License details and after a few minutes, the UDMG Server status switches to ACTIVE mode (see node status).
Do we have a picture to add here?
...
B-18762
...
Using Same Virtual Folder Name for Multiple User Accounts
This enhancement allows a customer to use the same virtual folder name for multiple user accounts. Multiple accounts can now use the same local server and the same vpath, but each account will point to a separate ("private") local directory.
make sure the local directory has the privilege to post read/write by the udmg user
...
Placeholders are allowed in the transfer rule paths (local directory and temp directory):
#REQUESTERHOST# : local account name, for SFTP, local-auth and FTP protocols.
#REQUESTEDHOST# : local server name, only for SFTP and local-auth.
They are expanded at runtime (when transfer is starting or when a FTP/SFTP command is executed).
For instance: /home/Users/Input/#REQUESTERHOST# is expanded
to /home/Users/Input/sftp_user1 for account "sftp_user1"
and /home/Users/Input/sftp_user2 for account "sftp_user2".
example
...
Panel | |
---|---|
|
Universal Data Mover Gateway release 2.0.0.0 - June 7, 2024, contains the following high-level features. For a complete list of all the included features and fixes, please refer to Universal Data Mover Gateway 2.0.x Maintenance.
File Transfer Enhancements
Backlog | Title | Description | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
B-18770 | Ad-hoc File Transfers | This enhancement allows the transfer of files between users using a quick share link for a given file, which is protected with a password, expiration date, and number of downloads. The links and associated files can be shared with any type of user, including external users as long as the user is provided the password. Each file for which an ad-hoc file transfer link is generated is given a unique ID/token used in the associated URL. All ad-hoc file transfer links are generated through the Web Transfer Client and managed through the Web Transfer Client and UDMG Admin UI. The user must have a local auth account and a local server in UDMG must be created with the local auth protocol attached.
For the creation of ad-hoc file transfer links, the following fields are required:
For all generated ad-hoc file transfer links, the following actions are available:
See Tutorial - Creating an Ad-hoc File Transfer Linkfor detailed instructions on how to create, edit, and manage ad-hoc file transfer links. Updates to the local-auth server configuration:
Web Transfer Client
UDMG Admin UI From UDMG Admin UI, ad-hoc file transfer information is visible in the Transfers Activity Dashboard and Ad-Hoc Activity service via the navigation pane. Transfers Dashboard Ad-Hoc Activity New endpoints added on the UDMG Server API: Note that the creation of the share link is only possible from the UDMG Web Transfer Client. The following endpoints allow the management of the existing links.
Updates to the udmg-client CLI:
| |||||||||||
B-18765 | Delete file after download from SFTP (MOVE Command) | This enhancement allows the deletion of a file after it is downloaded from SFTP using the MOVE command. The enhancement only applies to send or receive files. The change removes the need for additional configuration and monitoring in UAC and UDMG. Before this change, the local file could only be deleted in the UDMG Server with the DELETE post-task. The remote file could not be deleted in UDMG and UAC was required for the transfers where remote delete was required.
Examples:
Implemented Error Messages:
| |||||||||||
#35208 | UDMG Server | Adds a parameter to disable the implicit assignment of Transfer Rules. Before this change, a Transfer Rule was allowed for all servers, partners, and accounts right after creation. It was only restricted after an explicit assignment (whitelisting) to at least one server, partner, or account. The prior functionality created confusion for UDMG users and allowed unauthorized access to files and folders for third-party accounts during the time between the rule creation and its explicit assignment. The change allows the functionality to be disabled with the new UDMG Server configuration parameter The default value remains false but is likely to change with future releases.
|
Security
Backlog | Title | Description | ||||
---|---|---|---|---|---|---|
B-19917 | Updates to Secure JWT Token | This enhancement improves the security between UDMG Admin UI and UDMG Authentication Proxy with the use of session ID instead of user credentials after the initial login (secured the JWT token). Before this change, the user credentials were stored in the JWT payload, which exposed it to potential unauthorized access. The password was removed from the JWT Token and now the header 'X-Session-Id' must be included in each request. | ||||
B-19918, | Security Hardening | Improves security with a default NGINX configuration including the recommended security HTTP headers, hiding the server version, and disabling weak TLS ciphers. The following changes are now the default for the manual installation procedure and the installation with Linux packages (RPM/DEB). For upgrades, the parameter must be manually reviewed and added to the NGINX configuration file. Note: When installing UDMG 2.0 a new configuration file is generated with "new" added at end.
|
User Experience
Backlog | Title | Description | ||||
---|---|---|---|---|---|---|
User Interface Updates | This enhancement improves the usability of the UDMG Admin UI, allowing for easier access to information, customizations, searching/filtering, and new features. The changes align with the look and feel of Universal Automation Center. Key Changes Include: Landing Page / Homepage
Transfers Activity
License
Ad-hoc Activity
Local Servers
Shared Accounts
Rules
Cluster Nodes
| |||||
B-18766 | Added Server Status and Information in Details | This enhancement allows the user to quickly view the server status, including server state and information in the Local Servers list for all local servers. The "Status" column was added to the list of local server details. Before this change, the local server status details were only displayed in the “UDMG Server Status” popup. The color-coded bars show the internal service status:
| ||||
B-19796 | SSH Key Parsing Tool | This enhancement provides a tool to parse an SSH public key. The UDMG Server only accepts the OpenSSH format for public keys, so customers who use the PKCS8 and DER format were not previously able to convert keys to OpenSSH using standard tools (openssl and ssh-keygen). The The supported input formats are:
The output of the tool shows the key details and the OpenSSH format that is suitable for use during the setup of SFTP servers on UDMG:
Command line usage:
See Utilities Reference Guide: udmg-sshkey for additional details. |
Architecture
Backlog | Title | Description | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
B-18759 | Allowing the Sharing of Accounts Between Servers | This enhancement allows the sharing of accounts between servers. It avoids duplicate configurations for FTP and SFTP local servers by allowing accounts to be shared between local servers. Before this change, a local account was only defined for a given local server and not allowed to have the same account for multiple protocols. For instance, to allow a partner to transfer files over SFTP and FTP, it was required to have both an SFTP and FTP local server, each with their own local account. Each account was then maintained independently, which created additional overhead in configuration and maintenance (password or key updates). The Local Account service located in the UDMG Admin UI navigation pane is replaced by Shared Accounts service. Updated Shared Account process:
New permission for the management of Shared Accounts: sharedAccounts on User Permissions and user group Permissions New endpoints added on the UDMG Server API:
Updates to the udmg-client CLI: Both the new and old command lines can be used to add a new shared account. New CLI commands:
Current CLI commands:
The permission code for udmg-client is 'L'
Updated CLI Commands:
See Tutorial - Creating and Associating a Shared Account for additional details. | |||||||||||||||||||||
B-18755 | License Control | This change incorporates license management within Universal Data Mover Gateway. All environments are required to have an attached license key, including non-production environments. The Universal Data Mover Gateway is licensed for a set time period, number of UDMG Server instances, and number of transfer executions per month and environment. The counted file transfer executions only include the number of transfer instances completed as DONE over one month. To receive and apply your license keys, please follow the below steps and reference UDMG Licensing.
| |||||||||||||||||||||
B-18762 | Using the Same Virtual Folder Name for Multiple User Accounts | This enhancement allows users to use the same virtual folder name for multiple user accounts. Multiple user accounts can now use the same local server and the same vpath, but each account will point to a separate ("private") local directory. Note: The local directory must have read/write permission for the UDMG Server operating system user.
Placeholders are allowed in the transfer rule paths (local directory and temp directory): They are expanded at runtime (when the transfer is starting or when an FTP/SFTP command is executed). For instance, /home/Users/Input/#REQUESTERHOST# is expanded to differently for each account:
Example:
| |||||||||||||||||||||
#33083 | Harmonization of module usage | The UDMG Authentication Proxy, UDMG Agent Proxy Server, UDMG Agent Proxy Client, and the UDMG Web Transfer Client are updated with the addition of a mandatory command line option to start the modules, and the specification of the configuration file on the command line (it remains possible to use an environment variable for the same)
| |||||||||||||||||||||
#32518 | Harmonization of configuration file format | The UDMG Authentication Proxy supports TOML and YAML file formats for the configuration of the SSO service providers. This is in addition to JSON format that is already supported. The main configuration file remains in TOML format. Examples of SSO Google provider in the different formats:
|