Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Panel

Table of Contents

Universal Data Mover Gateway release 2.0.0.0 - May 31, 2024, contains the following high-level features. For a complete list of all the included features and fixes, please refer to Universal Data Mover Gateway 2.0.x Maintenance.

File Transfer Enhancements

Backlog

Title

Description

B-18770

Ad-hoc File Transfers via Email

This enhancement allows the transfer of files between users using a quick share link for a given file, which is protected with a password, expiration date, and restriction on number of download. 

based on authentication using LDAP (meaning no configuration is needed when 2 TME members want to share large files).


From the Web Transfer Client

New end point

  • Added new endpoint: POST /link . The endpoint is used to create a file link for a given remote path. The remote path will be computed to a local path during the link creation, and that local path will be stored along the link. Expiration date and file password are optional.
    Body:
Code Block
{
    "username":"foo",
    "password":"bar",
    "remote_path":"r1/coco.txt",
    "expiration_date":"2030-01-01T00:00:00Z",
    "file_password":"secret" 
}

Response:

{
    "file_link":"343bb79e-a476-459e-890f-32d34134612a" 
}

  • Added new endpoint: GET /link/{token}?password=secret . The endpoint is used to download a given file link. The endpoint uses the http.ServeContent function that handles If-Match, If-Unmodified-Since, If-None-Match, If-Modified-Since and If-Range requests, as well as the Content-Type header.
    Status codes:
    • 404: if the file link is not found.
    • 400: if the file link is expired.
    • 403: if the password is not correct.
    • 404: if the file does not exist in the server.
    • 500: if any internal error happens.
  • New Fields:
    • filename, for info
    • size, for info
    • the owner of a link (the local/shared account that created the link).
      This is needed:
      - for link maintenance, it will allow to show the created links to the owner and for admin to know who created the link...
      - to inform the recipient
    • creation date (could be default automatic orm created_date column)
    • list of recipients, possibly free text values that could be emails. we can start by supporting only 1 recipient

B-18765

Delete file after download from SFTP (Move Command)

This enhancement allows the deletion of a file after download from SFTP using the MOVE command. 

Prior to this change, the local file could only be deleted in UDMG Server? with the DELETE post-task and a remote file could not be deleted in Universal Data Mover Gateway. For remote files, a customer had to use UAC for the transfers where remote delete is required. The prior process created additional customer overhead for configuration and monitoring (UAC + UDMG).

  • New transfer info flagudmg_xfer_move, can have values true and false (bool). If set to true, the customer will delete the file being sent during theremovepipeline step.
    • udmg_xfer_moveflag is checked for accepted values when the transfer is created.
    • udmg_xfer_moveflag is checked for protocol implementation when the pipeline is created. This is only supported for SFTP.
  • New pipeline step: remove, between dataandpost tasks.
  • Local file is removed after transfer for send rule (PUT).
  • Remote file is removed after transfer for receive rule (GET).

Examples:

Code Block
udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1"  -w receive -i udmg_xfer_move:true

udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1S" -w send -i udmg_xfer_move:true


Implemented Error Messages:

  • TeInternal: move not supported for this protocol
  • TeInternal: failed to remove local file: %s
  • TeDataTransfer: Failed to remove remote SFTP file: %s

...

Backlog

Title

Description


User Interface Updates 

Need to specify what pages/web

This enhancement improves usability of the UI, allowing for improved viewing of information, customizations, searching/filtering, and new features. Aligns look and feel and features to UAC.


DATA Dump per Demo – some might be not be things we want to document/highlight

Universal Data Mover Gateway Admin UI Changes

Landing Page

  1. Updated icon for dark/light mode
  2. Pages are resizable using XX
  3. Left Navigation Pane (idk what we call this throughout) is expandable and collapsible 
  4. Rearranged Left Navigation Pane tabs and removed groupings

Left Sidebar Menu - Transfers tab

  1. Server details are only loaded in right panel after the selection of a server
  2. Filters in XX are not preselected
  3. Console information shown as pop-up message - requires pop-up to be closed out before continuing
  4. Consolidated Server and Configuration tabs into Server tab
    1. All fields are combined into one tab - improves server creation, saving, and error messaging process
  5. Reordered XXX tabs to align with UAC order

Left Sidebar Menu - Rules tab

  1. Consolidation of Rules, Pre-Tasks, Post-Tasks, Error Tasks, XXX Configuration tabs into XX tab
    1. Updated editor text functionality - real time error checking

Left Sidebar Menu - Shared Accounts tab

  1. Updated tab name from Local Servers to Shared Accounts (see B-18759 for detailed functionality and UI changes)
  2. Removed open text search 
  3. Added filtering functionality for each column in xxxxx
  4. Added new server status column with XXX icon - ; The following color indicates:
    1. Red: server error
    2. Green: server healthy
    3. Gray: server offline

Left Sidebar Menu - Licensing

  1. Added new tab for adding licensing information (See B-18755 for detailed functionality and UI changes)

Left Sidebar Menu - Cluster Nodes

  1. Added new tab for viewing the list of UDMG Server instances
  2. Displays UDMG Server status and detailed information?


B-18766Added Server State and Information in Details

This enhancement allows the user to quickly view the server status, including server state and information in the Details tab for all local servers. Prior to this change, the local server status details were only displayed in the “UDMG SERVERS STATUS” GUIServer Status” popup.

B-19796

SSH Key Parsing Tool

This enhancement provides a tool to parse a SSH public key. The UDMG Server only accepts the OpenSSH format for public keys, so customers who use the PKCS8 and DER format were not previously able to convert keys to OpenSSH using standard tools (openssl and ssh-keygen). The tool parses a SSH public key and shows the following details:

  • Format 

  • Algorithm

  • Key Size (for RSA)

  • SHA256 fingerprint

  • MD5 fingerprint

  • Conversion to authorized key line format

Supported formats are:

  • Authorized key (OpenSSH)

  • PEM encoded formats (PKCS8, PKCS1)

  • SSH wire format (Binary ASN1 DER)

See Utilities Reference Guide: udmg-sshkey

Architecture

Backlog

Title

Description

B-18759

Allowing Sharing of Accounts Between Servers

This enhancement allows the sharing of accounts between servers. It avoids the need for duplicate configuration for FTP and SFTP local servers by having accounts that are shared between local server. (another option would be to allow a server to support multiple protocols)-????

Prior to this change, a local account was only defined for a given local server and it was not possible to have the same account for multiple protocols. For instance, to allow a partner to transfer files over SFTP and FTP, it was required to have both an SFTP and FTP server, each with its own account. Each account was then maintained independently which created additional overhead in configuration and maintenance (password or key updates).

The implementation uses the current Local Account table and a New table for handling authorization.


UI Changes - Left Navigation Pane - Shared Accounts tab

  1. Updated tab name from Local Servers to Shared Accounts

Prior to this change, in order to create a local account, the user had to create a local server on the Local Server tab first and then create a local account within the XX tab. The user could then edit the local account from the Local Account tab. **A local account could not be created from the Local Account tab directly. 

The enhancement changes the local account creation process. The user must create the local account from the Shared Accounts tab and does not require the local server to be created yet.

  1. Click Shared Accounts tab
  2. Create local account – need steps
  3. Add BS etc
  4. Click Local Servers tab
  5. Add local account to server? or create new local server and add newly created account?





Be able to list share accounts from server endpoint.

Need to remove the column local_accounts.local_agent_id

local and shared account are functionally the same.

the local command is kept for regression:
- if an account is created with the 'local' command or API, then the local server parameter is mandatory and the association is made in the link table.

- if an account is created with the 'share' command or API, then the account is created without any link and cannot be used until there is at least one explicit association to a local server.

- if an account is linked with more than 1 server then it cannot be deleted from the 'local' command. Return an error mentioning it is a shared account.

  • Deprecated LocalAccount -> Local Agent Id
    • Replace by sb_local_account_auth fields (If the server is removed will cascade)
  • Accounts will be managed as a separated account and then Assign into the server via the pivot table ? ( Same way for the rules)
  • API Changes * POST /api/sb_servers/<server-name>/account
    • Link the account with the server
    • Body { "login": <user-name> }
    • DELETE /api/sb_servers/<server-name>/account
      • Remove Link
  • New API Endpoint
    • POST /api/sb_account * Same fields
    • List Account GET /api/sb_account
    • Get Account GET /api/sb_account/<name>
    • Update Account PUT /api/sb_account/<name>
    • Delete Account DELETE /api/sb_account/<name>
  • API Endpoints

    Code Block
    ### List Accounts
    GET http://admin:admin_password@localhost:8080/api/sb_local_accounts
    
    ### Create Account
    POST http://admin:admin_password@localhost:8080/api/sb_local_accounts
    Content-Type: application/json
    
    {
      "login": "test",
      "password": "123" 
    }
    
    ### Get Account
    GET http://admin:admin_password@localhost:8080/api/sb_local_accounts/test
    
    ### Disable Account
    PUT http://admin:admin_password@localhost:8080/api/sb_local_accounts/user/disable
    
    ### Enable Account
    PUT http://admin:admin_password@localhost:8080/api/sb_local_accounts/test/enable
    
    ### Update Account
    PATCH  http://admin:admin_password@localhost:8080/api/sb_local_accounts/test
    Content-Type: application/json
    
    {
      "password": "123456" 
    }
    
    ### Replace Account
    PUT http://admin:admin_password@localhost:8080/api/sb_local_accounts/test
    Content-Type: application/json
    
    {
      "login":"user",
      "password": "123456" 
    }
    
    ### Authorize Account
    PUT  http://admin:admin_password@localhost:8080/api/sb_local_accounts/user/allow/sftp-local-account
    
    ### Revoke Account
    PUT http://admin:admin_password@localhost:8080/api/sb_local_accounts/user/disallow/sftp-local-account
    
    ### Delete Account
    DELETE  http://admin:admin_password@localhost:8080/api/sb_local_accounts/user
    
    



B-18755

License Control

This change incorporates license management within Universal Data Mover Gateway. All instances of Universal Data Mover Gateway are now required to have a license key attached to the installation. Universal Data Mover Gateway is licensed by the number of executions per month*; specifically, the number of file transfer instances that completed as DONE over a period of one month. 

is there any distinction in the transfer types? like all count or only incoming vs outgoing? 

To receive and apply your license keys, please follow the below steps and reference UDMG Licensing


Step 1Contact your Stonebranch representative or Customer Support to receive the license keys for the intended operating system and environment.
Step 2Navigate to the UDMG Admin UI.

Step 3

Click the License option in the sidebar menu. The License option in the sidebar menu identifies license information for:

  • License Status
  • License Customer
  • Environment
  • Expiration Date
  • Cluster Nodes
  • Monthly Transfers

Step 4

Input the license key and press the Update License button.

Step 5

The display is refreshed with the License details and after a few minutes, the UDMG Server status switches to ACTIVE mode (see node status).


Do we have a picture to add here?


*should we add a note about how some customers are licensed differently? or have unlimited?; thinking of the ones licensed per partners but theres no system mechanism to track (which we dont want to highlight i assume)

what happens when they hit the # of transfers? do we warn them ahead of time? where do they see this information? what if they need more, what is the process?  if not, what is the process --even if we dont stop them we should say something of PM / CS tracks usage and communicates xxxx. 







B-18762

Use Same Virtual Folder Name for Multiple User Accounts

This enhancement allows a customer to use the same virtual folder name for multiple user accounts. Multiple local accounts can now use the same local server and the same vpath, but each account will point to a separate ("private") local directory.


Prior to this change, a customer was unable to use the same virtual folder name for multiple user accounts. The below example was not valid:

Users send their files to a folder called “Input” and receive their files from a folder called “Output”. These virtual folders link to real folders which have individual names and can only be access by the appropriate SFTP account. Some folders are accessible to more than one account, in particular if there are users placing files into Stonebranch on behalf of a company.


For the protocols: SFTP, FTP/E/S, local-auth

Placeholders are allowed in the transfer rule paths (local directory and temp directory):
#REQUESTERHOST# : local account name, for SFTP, local-auth and FTP protocols.
#REQUESTEDHOST# : local server name, only for SFTP and local-auth.

They are expanded at runtime (when transfer is starting or when a FTP/SFTP command is executed).
For instance: /home/Users/Input/#REQUESTERHOST# is expanded
to /home/Users/Input/sftp_user1 for account "sftp_user1"
and /home/Users/Input/sftp_user2 for account "sftp_user2".

...