Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


...


Panel

Table of Contents

Universal Data Mover Gateway release 2.0.0.0 - May 31, 2024, contains the following high-level features. For a complete list of all the included features and fixes, please refer to Universal Data Mover Gateway 2.0.x Maintenance.

File Transfer Enhancements

Backlog

Title

Description

B-18770

Ad-hoc File Transfers

This enhancement allows the transfer of files between users using a quick share link for a given file, which is protected with a password, expiration date, and number of downloads. The links and associated files can be shared with any type of user, including external users as long as the user is provided the password. Each file for which an ad-hoc file transfer link is generated is given a unique ID/token that is used in the associated URL.All used in the associated URL.

All ad-hoc file transfer links are generated and managed through the Web Transfer Client. The user must have local auth and admin??(read/write?). LDAP authentication will be incorporated in future releases.  A local server in UDMG must be created with the local auth protocol attached.

For new ad-hoc file transfer links, the following fields are generated and managed through the Web Transfer Client. The user must have local auth and admin??(read/write?). LDAP authentication will be incorporated in future releases.  A local server in UDMG must be created with the local auth protocol attached.

Best Practices: ?? share the link and password via different methods; store in different locations?

Web Transfer Client

pull in screenshot of page + fields +steps to create –

  • Selected File / File Path
  • Expiration Date
  • Max Downloads
  • File Size

For all generated ad-hoc file transfer links, the following actions are available.

  • Edit: Update/extend the Expiration date, Max Downloads attributes
  • Revoke:  ad-hoc file transfer link 
  • Cancel: 

    required:

    • Selected File / File Path
    • Expiration Date
    • Max Downloads

    For all generated ad-hoc file transfer links, the following actions are available:

    • Edit: Update/extend the Expiration date
    • Revoke: Disable the live link and make it unusable 
    • Copy Link to Clipboard: 

    Web Transfer Client

    1. Open the Web Transfer Client.
    2. Select the Environment drop-down and select Adhoc Activity.

    Image Added

    3. View the list of current ad-hoc file transfers from the landing page. Image Added4. To create a new ad-hoc file transfer link – I have  no option to do this

    5. To edit an ad-hoc file transfer link:

    1. Select the ad-hoc file transfer record checkbox.
    2. Use the Task bar icons to Extend, Revoke, and Copy Link to Clipboard.
      1. Image Added
      2. Note: If the Extend Ad-hoc Transfer field is left empty the link will not have an expiration date
      3. Note: A revoked ad-hoc file transfer link cannot be restored. A new link must be generated for the associated file and parameters. 


    UDMG Admin UI

    From UDMG Admin UI navigation pane, select Ad-Hoc Activity to view the list of ad-hoc file transfer links generated and associated details. The ad-hoc file transfer links cannot be edited via the UDMG Admin UI. The Ad-Hoc Activity Details include:

    • File Name
    • Status: Open, Expired, Revoked
    • Comment
    • Downloads
    • Max Downloads
    • File Size
    • Path
    • Urls


    New endpoints added: 

    • New endpoint:POST /link. The endpoint creates a file link for a given remote path. The remote path will be computed to a local path during the link creation, and that local path will be stored along the link. The expiration date and file password are optional.
      Body:
    Code Block
    {
        "username":"user",
        "password":"userpassword",
        "remote_path":"path/to/filename.txt",
        "expiration_date":"2030-01-01T00:00:00Z",
        "file_password":"secret" 
    }
    
    Response:
    
    {
        "file_link":"343bb79e-a476-459e-890f-32d34134612a" 
    }

    • New endpoint:GET /link/{token}?password=secret. The endpoint is used to download a given file link. 
      • Status codes:
        • 404: if the file link is not found
        • 400: if the file link is expired
        • 403: if the password is not correct
        • 404: if the file does not exist in the server
        • 500: if any internal error happens

    B-18765

    Delete file after download from SFTP (MOVE Command)

    This enhancement allows the deletion of a file after it is downloaded from SFTP using the MOVE command. The enhancement only applies to send or receive files. The change removes the need for additional configuration and monitoring in UAC and UDMG.

    Before this change, the local file could only be deleted in the UDMG Server with the DELETE post-task. The remote file could not be deleted in UDMG and UAC was required for the transfers where remote delete was required. 

    • New transfer info flagudmg_xfer_move, can have values true and false (bool). If set to true, the customer will delete the file being sent during the removepipeline step.
      • udmg_xfer_moveflag is checked for accepted values when the transfer is created.
      • udmg_xfer_moveflag is checked for protocol implementation when the pipeline is created. This is only supported for SFTP.
    • New pipeline step: remove, between dataandpost tasks.
    • Local file is removed after transfer for send rule (PUT).
    • Remote file is removed after transfer for receive rule (GET).

    Examples:

    Code Block
    udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1"  -w receive -i udmg_xfer_move:true
    
    udmg-client transfer add -f "README.md" -p "TestSftpPartner" -l "user" -r "Rule1S" -w send -i udmg_xfer_move:true
    
    


    Implemented Error Messages:

    • TeInternal: move not supported for this protocol
    • TeInternal: failed to remove local file: %s
    • TeDataTransfer: Failed to remove remote SFTP file: %s

    Security

    Backlog

    Title

    Description

    User Interface Updates 

    This enhancement improves the usability of the UDMG Admin UI and Web Transfer Client, allowing for easier access to information, customizations, searching/filtering, and new features. The changes align closer to UAC.

    Key Changes Include: HAVENT FINISHED UPDATES for this one

    Landing Page / Homepage

    1. Updated icons across the ADMIN UI navigation pane and User Task Bar
    2. Pages and sections are resizable
    3. ADMIN UI navigation pane is expandable and collapsible 
    4. Rearranged and added new ADMIN UI navigation pane services

    Transfers 

    1. Server details are only loaded in right panel after the selection of a server
    2. Filters in XX are not preselected
    3. Console information shown as pop-up message - requires pop-up to be closed out before continuing
    4. Consolidated Server and Configuration tabs into Server tab
      1. All fields are combined into one tab - improves server creation, saving, and error messaging process
    5. Reordered XXX tabs to align with UAC order

    License

    1. Added new tab for adding licensing information (See B-18755 for detailed functionality and UI changes)

    Ad-hoc Activity

    asdfasdf

    Shared Accounts 

    1. Updated tab name from Local Servers to Shared Accounts (see B-18759 for detailed functionality and UI changes)
    2. Removed open text search 
    3. Added filtering functionality for each column in xxxxx
    4. Added new server status column with XXX icon - ; The following color indicates:
      1. Red: server error
      2. Green: server healthy
      3. Gray: server offline

    Rules

    Consolidation of

    Backlog

    Title

    Description

    B-19917Updates to Secure JWT Token

    This enhancement improves the security between UDMG Admin UI and UDMG Authentication Proxy with the use of session ID instead of user credentials after the initial login (secured the JWT token). Before this change, the user credentials were stored in the JWT payload, which exposed it to potential unauthorized access.  

    The password was removed from the JWT Token and now the header 'X-Session-Id' must be included in each request.

    B-19918,
    B-19919,
    B-19920,
    B-19921



    Security Hardening


    Improves security with a default NGINX configuration including the recommended security HTTP headers, hiding the server version, and disabling weak TLS ciphers.

    Note: When installing UDMG 2.0 a new configuration file is generated with "new" added at end.

    • B-19918: Hides NGINX Version
      • Changes improve defenses against specific security vulnerabilities by removing the NGINX server banner information (technical and detailed web server version information in HTTP response header) from displaying on every server response.
    • B-19919: Improves Clickjacking Prevention
      • To improve malicious Clickjacking attacks, the following enhancements were implemented:

        • Prevents the browser from loading the page in frame using the X-Frame-Options or Content Security Policy (frame-ancestors) HTTP headers.

        • Prevents session cookies from being included when the page is loaded in a frame using the SameSite cookie attribute.

        • Implements JavaScript code in the page to attempt to prevent it from being loaded in a frame (known as a "frame-buster").

    • B-19920: Security Headers
      • The following HTTP response headers were added to provide an additional layer of security to Universal Data Mover Gateway.
        • HTTP response headers: Content-Security-Policy, X-Content-Type-Options, Referrer-Policy, Strict-Transport-Security, Permissions-Policy, X-Permitted-Cross-Domain-Policies, X-Frame-Options. X-XSS-Protection
    Code Block
    # DEPRECATED Security Headers
        add_header X-XSS-Protection "0";
        add_header X-Frame-Options "SAMEORIGIN";
        # Security Headers
        add_header Content-Security-Policy "frame-ancestors 'self'";
        add_header X-Content-Type-Options nosniff;
        add_header Referrer-Policy "strict-origin";
        add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains; preload';
        add_header Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()";
        add_header X-Permitted-Cross-Domain-Policies none;
    
    
     
    • B-19921: Disables Weak TLS Ciphers
      • Changes improve defenses against LUCKY13, a security vulnerability that specifically targets certain implementations of the TLS (Transport Layer Security) and DTLS (Datagram Transport Layer Security) protocols when they use the CBC mode with block ciphers for encryption, by disabling the CBC mode cipher encryption and enabling CTR or GCM cipher mode encryption. 
      • Recommended cipher list from https://ssl-config.mozilla.org
      • Minimum TLS version 1.2

    User Experience

      • they use the CBC mode with block ciphers for encryption, by disabling the CBC mode cipher encryption and enabling CTR or GCM cipher mode encryption. 
      • Recommended cipher list from https://ssl-config.mozilla.org
      • Minimum TLS version 1.2

    User Experience

    Backlog

    Title

    Description


    User Interface Updates 

    This enhancement improves the usability of the UDMG Admin UI, allowing for easier access to information, customizations, searching/filtering, and new features. The changes align with the look and feel of Universal Automation Center.

    Key Changes Include: 

    Landing Page / Homepage

    1. Updated icons across the UDMG ADMIN UI navigation pane and User Task Bar.
    2. UDMG ADMIN UI navigation pane and pages are expandable and collapsible.
    3. Rearranged and added new UDMG ADMIN UI navigation pane services.
    4. Console details are accessible via the bell icon on the User Task Bar. The details are displayed in a pop-up box at the bottom of the page. The pop-up box must be closed out before returning to the page. 

    Transfers 

    1. Transfer details are viewable after a file transfer record is selected.
    2. Filters in the Transfer Activity Dashboard are not pre-selected.

    License

    1. Added a new service in the UDMG Admin UI navigation pane for applying and managing the UDMG license. See B-18755 for additional details. 

    Ad-hoc Activity

    1. Added a new service in the UDMG Admin UI navigation pane to view the list of ad-hoc file transfer links generated and associated details

    Local Servers

    1. Updated Server Details tabs:
      1. Consolidated the Server and Configuration tabs into one Server tab.
      2. Reordered tabs to align with the Universal Automation Controller order.
    2. Added UDMG Server "Status" column to the local server list. See B-18766 for additional details. 
    3. Removed open text search and added filtering functionality at the column level.

    Shared Accounts 

    1. Updated service name from Local Accounts to Shared Accounts in UDMG Admin UI navigation pane. See B-18759 for additional details. 
    2. Updated the Accounts tab:
      1. Added the ability to add any created shared account to the server. See B-18759 for additional details. 
      2. Added the ability to disable a shared account. — is this also new for servers?

    Rules

    1. Updated Rules Details tabs:
      1. Consolidated Rules, Pre-Tasks, Post-Tasks, Error Tasks,
      XXX
      1. and Configuration tabs into
      XX tab
      1. Updated editor text functionality - real one Rule tab.
      2. Added real-time error checking to editor text boxes.

    Cluster Nodes

    1. Added new tab for viewing a new service to view the list of UDMG Server instancesDisplays UDMG Server status and detailed information?.
    B-18766Added Server Status and Information in Details

    This enhancement allows the user to quickly view the server status, including server state and information in the Local Servers list for all local servers. The "Status" column was added to the list of local server details.  Before this change, the local server status details were only displayed in the “UDMG Server Status” popup.

    Image RemovedImage Added

    The color-coded bars show the internal service status:

    • Green: Service is operational, and all services are Running or Offline (either disabled or stopped by a user).
    • Orange: Service is degraded and at least one (1) service is in the Error state.
    • Red: Service is failing and one (1) of the main services (Admin, Database, Controller) is not running.
    • Grey: Service is not reachable and UDMG Admin UI cannot refresh the status.

    **is service right here or server?


    B-19796

    SSH Key Parsing Tool

    This enhancement provides a tool to parse an SSH public key. The UDMG Server only accepts the OpenSSH format for public keys, so customers who use the PKCS8 and DER format were not previously able to convert keys to OpenSSH using standard tools (openssl and ssh-keygen). 

    The udmg-sshkey tool enables the conversion from another format to the OpenSSH format.

    The supported input formats are:

    • OpenSSH authorized key line
    • PEM encoded formats (PKCS8, PKCS1)
    • SSH wire format (Binary ASN1 DER)

    The output of the tool shows the key details and the OpenSSH format that is suitable for use during the setup of SFTP servers on UDMG:

    • Key format
    • Key algorithm
    • Key size (only for RSA)
    • SHA256 fingerprint
    • MD5 fingerprint
    • Conversion to authorized key line format

    Command line usage:

    Code Block
    dmg-sshkey -h
    Usage:
      udmg-sshkey [OPTIONS] <parse | version>
    Help Options:
      -h, --help  Show this help message
    Available commands:
      parse    Parse an SSH Public key file
      version  Print version and exit
       
    udmg-sshkey parse -h
    Usage:
      udmg-sshkey [OPTIONS] parse [parse-OPTIONS]
    Help Options:
      -h, --help      Show this help message
    [parse command options]
          -f, --file= The public key file, accepted formats are PKCS8 (x509), OpenSSH (authorized keys), and SSH wire.
    
    


    See Utilities Reference Guide: udmg-sshkey for additional details.

    ...

    Backlog

    Title

    Description

    B-18759

    Allowing the Sharing of Accounts Between Servers

    This enhancement allows the sharing of accounts between servers. It avoids the need for duplicate configuration configurations for FTP and SFTP local servers by having allowing accounts that are shared between local server. (another option would be to allow a server to support multiple protocols)-????Prior to to be shared between local servers.

    Before this change, a local account was only defined for a given local server and it was not possible allowed to have the same account for multiple protocols. For instance, to allow a partner to transfer files over SFTP and FTP, it was required to have both an SFTP and FTP local server, each with its their own local account. Each account was then maintained independently, which created additional overhead in configuration and maintenance (password or key updates).

    The implementation uses the current Local Account table and a New table for handling authorization.

    Be able to list share accounts from server endpoint.

    • don't break integration with accounts

    New command line:

    Code Block
    udmg-client account share add -l user -p pass -n user
    
    Old command line: 
    Code Block
    udmg-client account local $SERVER add -l user -p password
    
    Note: if the old command line is used, then the command line automatically creates a shared account and assigns it to the local server.

    Both the new and old command line can be used to add a new shared account. 

    • for new command line for shared account ; very similar to old one
    Usage:
    udmg-client [GLOBAL-OPTIONS] account share <command>
    Code Block
    Usage:
    udmg-client [GLOBAL-OPTIONS] account share <command>
    
    Connection Options:
    -a, --address= The address of the UDMG Server [$UDMG_SERVER_ADDRESS]
    -i, --insecure= Skip certificate verification [$UDMG_SERVER_INSECURE]
    -l, --legacy= Use legacy API version [$UDMG_SERVER_LEGACY]
    -t, --timeout= Client Connection Timeout (default: 30) [$UDMG_CLIENT_TIMEOUT]
    
    Output Options:
    --color=[always|auto|never] Control color output (default: auto) [$UDMG_COLOR]
    
    Help Options:
    -h, --help Show this help message
    
    Available commands:
    add Add a new shared account
    allow Allow a shared account to use a Local Agent
    authorize Authorize a shared account to use a rule
    cert Manage a share account's certificates
    delete Delete a shared account
    disable Disable shared account
    disallow Disallow a shared account's permission to use a local Agent
    enable Enable shared account
    get Retrieve a shared account's information
    list List shared accounts
    revoke Revoke a shared account permission to use a rule
    update Update a shared account
    
    UI Changes - Left Navigation Pane - Shared Accounts tab
    1. Updated tab name from Local Servers to Shared Accounts

    Prior to this change, in order to create a local account, the user had to create a local server on the Local Server tab first and then create a local account within the XX tab. The user could then edit the local account from the Local Account tab. **A local account could not be created from the Local Account tab directly. 

    The enhancement changes the local account creation process. The user must create the local account from the Shared Accounts tab and does not require the local server to be created yet.

    1. Click Shared Accounts tab
    2. Create local account – need steps
    3. Add BS etc
    4. Click Local Servers tab
    5. Add local account to server? can add any of the shared accounts that have been created

    add info about disable

    overhead in configuration and maintenance (password or key updates).

    The Local Account service located in the UDMG Admin UI navigation pane is replaced by Shared Accounts service. 

    Image Added

    Updated Shared Account process:

    1. Click on Shared Accounts from UDMG Admin UI navigation pane.
    2. Create a Shared Account by populating the required details and following the current process (tabs are the same).  
      1. Note: A local server does not have to be created yet.
    3. Click on Local Servers from UDMG Admin UI navigation pane.
    4. Create a local server by populating the required details and following the current process OR select a Local Server record. 
    5. Click the Accounts tab.
    6. Click the Pencil (edit) icon.
    7. Add and/or remove the shared accounts via the pop-up box. 
      1. Note: All created shared accounts are available to be add to the local server.

    Image Added


    Be able to list share accounts from server endpoint. What else do we need to share about it? The change uses the current Local Account table and a New table for authorization.


    Both the new and old command lines can be used to add a new shared account. 

    New CLI commands:

    Code Block
    udmg-client account share add -l user -p pass -n user
    

    Current CLI commands: 


    Code Block
    udmg-client account local $SERVER add -l user -p password
    
    Note: If the old command line is used, then the command line automatically creates a shared account and assigns it to the local server.


    Updated CLI Commands:

    Code Block
    Usage:
    udmg-client [GLOBAL-OPTIONS] account share <command>
    
    Connection Options:
    -a, --address= The address of the UDMG Server [$UDMG_SERVER_ADDRESS]
    -i, --insecure= Skip certificate verification [$UDMG_SERVER_INSECURE]
    -l, --legacy= Use legacy API version [$UDMG_SERVER_LEGACY]
    -t, --timeout= Client Connection Timeout (default: 30) [$UDMG_CLIENT_TIMEOUT]
    
    Output Options:
    --color=[always|auto|never] Control color output (default: auto) [$UDMG_COLOR]
    
    Help Options:
    -h, --help Show this help message
    
    Available commands:
    add Add a new shared account
    allow Allow a shared account to use a Local Agent
    authorize Authorize a shared account to use a rule
    cert Manage a share account's certificates
    delete Delete a shared account
    disable Disable shared account
    disallow Disallow a shared account's permission to use a local Agent
    enable Enable shared account
    get Retrieve a shared account's information
    list List shared accounts
    revoke Revoke a shared account permission to use a rule
    update Update a shared account
    


    B-18755

    License Control

    This change incorporates license management within Universal Data Mover Gateway. All instances of Universal Data Mover Gateway are now environments are required to have a an attached license key attached to the installation, including non-production environments The Universal Data Mover Gateway Gateway is licensed by the for a set time period, number of UDMG Server instances, and number of transfer executions per month *; specifically, and environment. The counted file transfer executions only include the number of file transfer instances that completed as DONE over a period of one month. is there any distinction in the transfer types? like all count or only incoming vs outgoing? 

    To receive and apply your license keys, please follow the below steps and reference UDMG Licensing


    Step 1Contact your Stonebranch representative or Customer Support to receive the license keys for the intended operating system and environment.
    Step 2Navigate to the UDMG Admin UI.

    Step 3

    Click the License option service in the sidebar menuUDMG Admin UI navigation pane. The License option in the sidebar menu identifies license information for:

    • License Status
    • License Customer
    • Environment
    • Expiration Date
    • Cluster Nodes
    • Monthly Transfers

    Image Removedservice allows admin users to enter and update license keys and view license details:

    • License Status: "Unlicensed" when no license has been entered, "Licensed" otherwise.
    • License Customer: name of customer
    • Environment: environment name (e.g., test, dev, non-prod, prod)
    • Expires On: the date when the license expires (at 00:00:00)
    • Nodes: the number of allowed cluster nodes, or unlimited
    • Transfers: the number of allowed transfers per month, or unlimited

    Step 4

    Input the license key and press the Update License button.

    Image RemovedImage Added

    Image Added

    Step 5

    The display is refreshed with the License details and after After a few minutes, the UDMG Server status switches to ACTIVE mode (see see node status).

    Do we have a picture to add here?Image Added


    B-18762

    Using the Same Virtual Folder Name for Multiple User Accounts

    This enhancement allows

    a customer

    users to use the same virtual folder name for multiple user accounts. Multiple user accounts can now use the same local server and the same vpath, but each account will point to a separate ("private") local directory.

    make sure the

    Note: The local directory

    has the privilege

    must have permission to post read/write by the

    udmg

    UDMG user.


    For the protocols: SFTP, FTP/E/S, local-auth

    Placeholders are allowed in the transfer rule paths (local directory and temp directory):
    #REQUESTERHOST# : local account name, for SFTP, local-auth and FTP protocols.
    #REQUESTEDHOST# : local server name, only for SFTP and local-auth.

    They are expanded at runtime (when transfer is starting or when a FTP/SFTP command is executed).

    For instance: /home/Users/Input/#REQUESTERHOST# is expanded to

    to

    /home/Users/Input/sftp_user1 for account "sftp_user1"    AND

    and

    /home/Users/Input/sftp_user2 for account "sftp_user2".

    example


    Example:

    Code Block
    udmg-client rule add -n Rule10   -d receive -p /data-host        --local-dir '/data/#REQUESTEDHOST#'