Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Panel

Table of Contents

...

Upgrading Universal Data Mover Gateway

Upgrading UDMG refers to the increase of a currently installed pre-2.0.x Version, Release, or Modification level of UDMG (1.5.x, 1.4.x, 1.3.x, 1.2.x) to UDMG 2.0.x.

Warning

As a precautionary measure, it is highly recommended that you back up the UDMG database prior to upgrading.

Upgrading from a pre-2.0 UDMG release

If you are upgrading an installation of UDMG from any release before 2.0.0.0, the following changes must be reviewed carefully.

Note
titleLicense Key

Starting with UDMG version 2.0, a license key must be provided to enable file transfers.

Contact your Stonebranch representative or Customer Support to receive the license key before upgrading to version 2.0 or later.

...

Note
titleStart parameters

The start parameters for several services have changed.


Please review the start scripts, especially for manual installation or if the Systemd service files have been edited. 
For a standard upgrade with the provided Linux packages, the service configuration is updated automatically 

The following modules now require a 'start' command in server mode. 

udmg-auth-proxy start -f configuration_file

udmg-agent-client start -f configuration_file

udmg-agent-server start -f configuration_file

Without it, they will only display the command line usage information.

In addition, a 'test' command allows the syntax of the configuration file to be verified without starting the server.

New configuration parameter with UDMG 2.0.0.0

Note
titleConfiguration Parameters

UDMG Server

  • Adds new database timeout option
Code Block
languagetext
[database]
; Threshold before warning for long-running queries, the default is 10 seconds
WarningTimeout=10s
  • Adds a parameter to disable the implicit assignment of Transfer Rules. Before this change, a Transfer Rule was allowed for all servers, partners, and accounts right after creation. It was only restricted after an explicit assignment (whitelisting) to at least one server, partner, or account. The prior functionality created confusion and allowed unauthorized access to files and folders for third-party accounts during the time between the rule creation and its explicit assignment. The change allows the functionality to be disabled with the new UDMG Server configuration parameter ExplicitRuleAssignment. When set to true, a Transfer Rule is only effective when it is explicitly linked with the intended target (local server, remote partner, remote or local account).


Code Block
languagetext
[rule]
; Disables global rules, requiring rules to be explicitly allowed to be used.
; ExplicitRuleAssignment = false


UDMG Admin UI

  • The recommended security settings for NGINX have been modified, with the inclusion of the following header directives. Please review the sample configuration on Installing NGINX Server page.
Code Block
languagetext
    # Server Banner
    server_tokens off;
 
    # DEPRECATED Security Headers
    add_header X-XSS-Protection "0";
    add_header X-Frame-Options "SAMEORIGIN";
 
    # Security Headers
    add_header Content-Security-Policy "frame-ancestors 'self'";
    add_header X-Content-Type-Options nosniff;
    add_header Referrer-Policy "strict-origin";
    add_header Strict-Transport-Security 'max-age=31536000; includeSubDomains; preload';
    add_header Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()";
    add_header X-Permitted-Cross-Domain-Policies none;
 
    location /service/ {
      proxy_pass         http://udmg_auth_proxy/;
      proxy_set_header   X-Real-IP       $remote_addr;
      proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    }


Agent Proxy Client

  • The section for the configuration of the target UDMG Server is renamed from [gateway] to [server].
  • New secure option to support HTTPS for the connection to the UDMG Server API 
Code Block
languagetext
[server]
# UDMG Server Hostname or IP, and port
hostname = "localhost"
port = "8080"
# UDMG API protocol, set to true to select https, default is false for http
secure = false
# UDMG Server Username/Password
username = "admin"
password = "admin_password"


Upgrading from a pre-1.3 UDMG release


Note
titleNote

If you are upgrading an installation of UDMG from any release before 1.3.0.0, you must uninstall the older version before installing the new version.

Migration from releases before 1.2.1.1 (0.7.1-sb.3) is not supported anymore.


The installation packages, binaries, services, and environment variables have changed, and this does not allow for a standard upgrade.

CategoryPrior releasesRelease 1.3
user and groupmft:mftudmg:udmg
binaries/usr/local/bin/opt/udmg/bin
configuration files/etc/mft/opt/udmg/etc
log files/var/opt/udmg/logs/var/opt/udmg/logs
UDMG Admin UI assets/opt/udmg/var/www/mft/opt/udmg/var/www/udmg
Services
  • mft_waarp_gateway
  • mft_auth_proxy
  • nginx
  • mft_web_transfer_client
  • mft-agent-proxy-client
  • mft-agent-proxy-server
  • udmg-server
  • udmg-auth-proxy
  • nginx
  • udmg-web-transfer
  • udmg-agent-server
  • udmg-agent-client
Environment variables
  • WAARP_GATEWAY_ADDRESS
  • MFT_AUTH_PROXY_CONFIG
  • MFT_AGENT_PROXY_CONFIG
  • UDMG_SERVER_ADDRESS
  • UDMG_AUTH_PROXY_CONFIG
  • UDMG_AGENT_CONFIG

It may be required to modify the work and data directories ownership or access rights and to update UDMG Server transfer rules to use paths that are accessible by the 'udmg' user.

The configuration files must be reviewed and compared between the old and new locations.

Special attention is required for the AESpassphrase parameter for UDMG Server. It must be the path for the file that was used by the previous release and must be accessible by the new service user. It is recommended to set an absolute path in the configuration file.

...

Pre-Installation / Upgrade Backups

The installation process overwrites the current files (exception: the configuration files are kept), this may affect your modifications.

Backing up the configuration files optimizes the time it takes you to get up and running after installing or upgrading.

/opt/udmg/etc/udmg/nginx/udmg.conf
/opt/udmg/etc/udmg/agent/client.toml
/opt/udmg/etc/udmg/agent/server.toml
/opt/udmg/etc/udmg/auth-proxy/config.toml
/opt/udmg/etc/udmg/web-transfer/config.toml
/opt/udmg/etc/udmg-server/server.ini

After upgrading RPM or DEB packages, review the new configuration file templates (with the extension .rpmnew or .dpk-new) and edit the current configuration files to add new parameters or remove deprecated parameters.

Anchor
db_migration
db_migration

Release Migration

The UDMG release version is stored in the database to ensure the data structure is compatible with the version of the UDMG components.

After upgrading the component binaries and before starting the UDMG Server, it is required to perform the release migration step.

Note
titleNote

The release migration is altering the database structure and requires the database user to have the DDL privileges fore creating, altering and deleting database its own objects.

Please refer to Installing a Database section for the requires privileges for each the database vendor. 


The udmg-server "migrate" command handles the necessary database updates and the setting of the internal version.

...

Upgrading with Linux Software Packages

Step 1

Contact your Stonebranch representative or Customer Support to receive the software package for the intended operating system.

Step 2

Perform the recommended backup of configuration files.

Step 3

Stop the components services.

The exact steps depend on the system architecture and the deployed components, for example:

On the main UDMG host:

sudo systemctl stop nginx
sudo systemctl stop udmg-auth-proxy
sudo systemctl stop udmg-agent-client
sudo systemctl stop udmg-server

On the Proxy host:
sudo systemctl stop udmg-agent-server

Step 4

Upgrade the UDMG packages (RPM or DEB). 

For RPM based Linux:

On the main UDMG host:

sudo rpm -Uvh udmg-agent-2.0.0.0.build.3.x86_64.rpm
sudo rpm -Uvh udmg-admin-ui-2.0.0.0.build.29.x86_64.rpm
sudo rpm -Uvh udmg-admin-ui-nginx-2.0.0.0.build.29.x86_64.rpm
sudo rpm -Uvh udmg-auth-proxy-2.0.0.0.build.10.x86_64.rpm
sudo rpm -Uvh --force udmg-server-2.0.0.2.build.2.x86_64.rpm

On the Proxy host:

sudo rpm -Uvh udmg-agent-2.0.0.0.build.3.x86_64.rpm


For Debian based Linux:

On the main UDMG host:

dpkg --install udmg-agent-2.0.0.0.build.3.x86_64.deb
dpkg --install udmg-admin-ui-2.0.0.0.build.29.x86_64.deb
dpkg --install udmg-admin-ui-nginx-2.0.0.0.build.29.x86_64.deb
dpkg --install udmg-auth-proxy-2.0.0.0.build.10.x86_64.deb
dpkg --install udmg-server-2.0.0.2.build.2.x86_64.deb

On the Proxy host:

dpkg --install udmg-agent-2.0.0.0.build.3.x86_64.deb

Step 5

Review the component configuration files.
Refer to each component installation section below for the list of parameters.

On the main UDMG host:
/opt/udmg/etc/udmg/nginx/udmg.conf
/opt/udmg/etc/udmg/agent/client.toml
/opt/udmg/etc/udmg/auth-proxy/config.toml
/opt/udmg/etc/udmg-server/server.ini

On the Proxy host:

/opt/udmg/etc/udmg/agent/server.toml


Note that new configuration file templates (with the extension .rpmnew or .dpk-new) that contain all the allowed parameters are added during the software package upgrade.


Note
titleNote

The default upstream port to reach the UDMG Authentication Proxy is set to 5775 in /opt/udmg/etc/udmg/nginx/udmg.conf starting from version 1.5.0.1.

Please review and make sure that the same port (either 5000 or 5775) is also defined in /opt/udmg/etc/udmg/auth-proxy/config.toml .


Step 6Perform the release migration.
Step 7

Review the component service configuration files.

For instance, the UDMG Authentication Proxy service file (/etc/systemd/system/udmg-auth-proxy.service) was installed with these parameters in version 1.5

Code Block
titleudmg-auth-proxy.service (Version 1.5)
[Unit]
Description=Stonebranch UDMG Authentication Proxy

[Service]
Type=simple
User=udmg
Group=udmg
WorkingDirectory=/home/udmg
Environment="UDMG_AUTH_PROXY_CONFIG=/opt/udmg/etc/udmg/auth-proxy/config.toml"
ExecStart=/bin/sh -c 'exec /opt/udmg/bin/udmg-auth-proxy'
Restart=on-failure
SyslogIdentifier=udmg-auth-proxy
SyslogFacility=local0

[Install]
WantedBy=multi-user.target

Note the addition of the start command on the ExecStart directive in the version 2.0:

Code Block
titleudmg-auth-proxy.service (Version 2.0)
[Unit]
Description=Stonebranch UDMG Authentication Proxy

[Service]
Type=simple
User=udmg
Group=udmg
WorkingDirectory=/home/udmg
Environment="UDMG_AUTH_PROXY_CONFIG=/opt/udmg/etc/udmg/auth-proxy/config.toml"
ExecStart=/bin/sh -c 'exec /opt/udmg/bin/udmg-auth-proxy start'
Restart=on-failure
SyslogIdentifier=udmg-auth-proxy
SyslogFacility=local0

[Install]
WantedBy=multi-user.target


The same applies for the other components:

/etc/systemd/system/udmg-auth-proxy.service

/etc/systemd/system/udmg-agent-client.service

/etc/systemd/system/udmg-agent-server.service

Step 8

Start the components services.

The exact steps depend on the system architecture and the deployed components, for example:

On the Proxy host:

sudo systemctl start udmg-agent-server

On the main UDMG host:

sudo systemctl start udmg-server
sudo systemctl start udmg-agent-client
sudo systemctl start udmg-auth-proxy
sudo
systemctl start nginx

Step 9Verify or apply the license; see UDMG Licensing.


...

Step 1

Contact your Stonebranch representative or Customer Support to receive the software package for the intended operating system.

Step 2

Perform the recommended backup of configuration files.

Step 3

Stop the components services.

The exact steps depend on the system architecture and the deployed components, for example:

On the main UDMG host:
sudo systemctl stop nginx
sudo systemctl stop udmg-server
sudo systemctl stop udmg-auth-proxy
sudo systemctl stop udmg-agent-client

On the Proxy host:
sudo systemctl stop udmg-agent-server

Step 4

Replace the component binaries:

sudo cp udmg-server.bin /opt/udmg/bin/udmg-server
sudo cp udmg-client.bin /opt/udmg/bin/udmg-client
sudo cp udmg-auth-proxy.bin /opt/udmg/bin/udmg-auth-proxy

Change ownership/permissions on new files:

sudo chown -R root:udmg /opt/udmg/bin/
sudo chmod -R 750 /opt/udmg/bin/

Upgrade the Admin UI:

sudo mv /opt/udmg/var/www/udmg mv /opt/udmg/var/www/udmg_BACKUP
sudo unzip -d /opt/udmg/var/www/udmg 'udmg-admin-ui-2.0.0.3.build.21.zip'
sudo chown -R root:udmg /opt/udmg/var/www/udmg

Step 5

Review the component configuration files.

Refer to each component installation section below for the list of parameters.
/opt/udmg/etc/udmg/nginx/udmg.conf
/opt/udmg/etc/udmg/agent/client.toml
/opt/udmg/etc/udmg/agent/server.toml
/opt/udmg/etc/udmg/auth-proxy/config.toml
/opt/udmg/etc/udmg-server/server.ini

Step 6Perform the release migration.
Step 7

Review the component service configuration files.

For instance, the UDMG Authentication Proxy service file (/etc/systemd/system/udmg-auth-proxy.service) was installed with these parameters in version 1.5

Code Block
titleudmg-auth-proxy.service (Version 1.5)
[Unit]
Description=Stonebranch UDMG Authentication Proxy

[Service]
Type=simple
User=udmg
Group=udmg
WorkingDirectory=/home/udmg
Environment="UDMG_AUTH_PROXY_CONFIG=/opt/udmg/etc/udmg/auth-proxy/config.toml"
ExecStart=/bin/sh -c 'exec /opt/udmg/bin/udmg-auth-proxy'
Restart=on-failure
SyslogIdentifier=udmg-auth-proxy
SyslogFacility=local0

[Install]
WantedBy=multi-user.target

Note the addition of the start command on the ExecStart directive in the version 2.0:

Code Block
titleudmg-auth-proxy.service (Version 2.0)
[Unit]
Description=Stonebranch UDMG Authentication Proxy

[Service]
Type=simple
User=udmg
Group=udmg
WorkingDirectory=/home/udmg
Environment="UDMG_AUTH_PROXY_CONFIG=/opt/udmg/etc/udmg/auth-proxy/config.toml"
ExecStart=/bin/sh -c 'exec /opt/udmg/bin/udmg-auth-proxy start'
Restart=on-failure
SyslogIdentifier=udmg-auth-proxy
SyslogFacility=local0

[Install]
WantedBy=multi-user.target


The same applies for the other components:

/etc/systemd/system/udmg-auth-proxy.service

/etc/systemd/system/udmg-agent-client.service

/etc/systemd/system/udmg-agent-server.service

Step 8

Start the components services.

The exact steps depend on the system architecture and the deployed components, for example:

On the Proxy host:

sudo systemctl start udmg-agent-server

On the main UDMG host:

sudo systemctl start udmg-server
sudo systemctl start udmg-auth-proxy
sudo systemctl start udmg-agent-client

sudo systemctl start nginx

Step 9Verify or apply the license; see UDMG Licensing.

...

Panel

#
# (c) Copyright 2024 Stonebranch, Inc., All rights reserved.
#
# Stonebranch, Inc.
# Universal Data Mover Gateway Server Configuration File
#
# This configuration file specifies global options for the
# udmg-server program.
#
# The configuration file is organized with the grouping of options under
# different section that are marked by brackets: [section_name]
# This organization should be maintained when modifying the file.
#
# The file syntax is:
#
# - Lines starting with a # or a ; are comments.
# - Blank lines are ignored.
# - Option lines are 'keyword = value' format.
# - keywords are not case sensitive.
# - keywords can start in any column.
# - Case sensitivity of the value depends on the value being specified.
# For example, a yes or no option is not case sensitive, but a file
# or directory name is.
# - Values must be enclosed in quotations marks (") or apostrophes (')
# if the value contains a space or tab.
# - File or folder path on Windows platform must be written with one of the following syntaxes:
# LogPath = "C:/UDMG/UDMG Server/logs"
# LogPath = "C://UDMG//UDMG Server//logs"
# LogPath = "C:\\UDMG\\UDMG Server\\logs"
# LogPath = C:\UDMG\UDMG Server\logs
# LogPath = C:/UDMG/UDMG Server/logs
#
#####################################################################


[global]
; The name given to identify this gateway instance. If the database is shared between multiple gateways, this name MUST be unique across these gateways.
GatewayName = sb-mft-01

; Default OS permission for created files
; FilePermissions = 770

; Default OS permission for created directories
; DirPermissions = 770

[paths]

; The root directory of the gateway. By default, it is the working directory of the process.
GatewayHome = /home/udmg/udmg-server

; The directory for all incoming files.
; DefaultInDir = in

; The directory for all outgoing files.
; DefaultOutDir = out

; The directory for all running transfer files.
; DefaultTmpDir = tmp


[log]
; All messages with a severity above this level will be logged. Possible values are TRACE, DEBUG, INFO, WARNING, ERROR and CRITICAL.
Level = INFO

; The path to the file where the logs must be written. Special values 'stdout' and 'syslog' log respectively to the standard output and to the syslog daemon.
; LogTo = stdout

; If LogTo is set on 'syslog', the logs will be written to this facility.
; SyslogFacility = local0

; The directory for the log files of the local servers, partners, and transfers.

; No default, if not provided then the detailed log feature is disabled. If not present, the directory is created with DirPermissions.

LogPath = /var/opt/udmg/logs
; Set to true to have a distinct log file for each local server and remote partner. The file is <LogPath>/<agent name>.<hostname>.log. The default is false to disable the feature.
LogAgent = false


[admin]
; The address used by the admin interface.
Host = 0.0.0.0

; The port used by the admin interface. If the port is 0, a free port will automatically be chosen.
Port = 18080

; Path of the TLS certificate for the admin interface.
; TLSCert =

; Path of the key of the TLS certificate.
; TLSKey =

; Password for the key of the TLS Certificate (if key is encrypted).
; TLSPassphrase =

; API rate limiter: number of allowed requests per client IP, per second. After that HTTP code 429 is returned. Disabled if 0 or not provided.
; RateLimit = 0

[database]

; Type of the RDBMS used for the UDMG Server database. Possible values: sqlite, mysql (default), postgresql, oracle, mssql
Type = postgresql

; Address (URL:port) of the database. The default port depends on the type of database used (PostgreSQL: 5432, MySQL: 3306, MS SQL: 1433, Oracle: 1521, SQLite: none).
Address = localhost:5432

; The name of the database
Name = udmg

; The name of the gateway database user
User = udmg_user

; The password of the gateway database user
Password = udmg_password

; Path of the database TLS certificate file (only supported for mysql, postgresql).
; TLSCert =

; Path of the key of the TLS certificate file (only supported for mysql, postgresql).
; TLSKey =

; The path to the file containing the passphrase used to encrypt account passwords using AES. Recommended to be a full absolute path, if the file does not exist, a new passphrase is generated the first time.
AESPassphrase = /opt/udmg/etc/udmg-server/passphrase.aes

; Maximum number of database connections, the default is 0 (unlimited)
; MaxConnections = 0

; Maximum number of transactions retries, the default is 3.
; MaxRetries = 3

; Delay in milliseconds between retries, the default is 100.
; MaxRetriesWait = 100

; Threshold before warning for long-running queries, the default is 10 seconds

WarningTimeout=10s


[controller]
; The frequency at which the database will be probed for new transfers
; Delay = 5s

; The maximum number of concurrent incoming transfers allowed on the gateway (0 = unlimited).
; MaxTransferIn = 0

; The maximum number of concurrent outgoing transfers allowed on the gateway (0 = unlimited).
; MaxTransferOut = 0

; The frequency at which the heartbeat will be updated
; Heartbeat = 10s

; The deadline to determine if this instance will be active
; Deadline = 5m0s


[sftp]
; Set to true to allow legacy and weak cipher algorithms: 3des-cbc, aes128-cbc, arcfour, arcfour128, arcfour256
; AllowLegacyCiphers = false


[tasks]
; Set to true to disable the COPY task.
DisableCopy = false

; Set to true to disable the MOVE task.
DisableMove = false

; Set to true to disable the COPYRENAME task.
DisableCopyRename = false

; Set to true to disable the MOVERENAME task.
DisableMoveRename = false

; Set to true to disable the DELETE task.
DisableDelete = false

; Set to true to disable the RENAME task.
DisableRename = false

; Set to true to disable the CHECKREGEX task.
DisableCheckRegex = false

; Set to true to disable the PUBLISHEVENT task.
DisablePublishEvent = false

; Set to true to disable the ICAP task.
DisableIcap = false

; Set to true to disable the TRANSFER task.
DisableTransfer = false

; Set to true to disable the EXECMOVE task.
DisableExecMove = false

; Set to true to disable the EXECOUTPUT task.
DisableExecOutput = false

; Set to true to disable the EXEC task.
DisableExec = true


[rule]
; Disables global rules, requiring rules to be explicitly allowed to be used.
; ExplicitRuleAssignment = false

...

Setup the Systemd Services


Note
titleNote

If you are upgrading an installation of UDMG from any release before 2.0.0.0, the start parameters for several services have changed.

Please review them carefully, especially for manual installation or if the Systemd service files have been edited. 
The service configuration is updated automatically for a standard upgrade with the provided Linux packages.

The following modules now require a 'start' command in server mode. 

udmg-auth-proxy start -f configuration_file

udmg-web-transfer start -f configuration_file

udmg-agent-client start -f configuration_file

udmg-agent-server start -f configuration_file

Without it, they will only display the command line usage information.

In addition, a 'test' command allows the syntax of the configuration file to be verified without starting the server.

UDMG Server

Create a new service definition:

...

Anchor
selinux
selinux
Using UDMG with SELinux

Security-Enhanced Linux (SELinux) is enabled by default on modern RHEL and CentOS servers. Each operating system object (process, file descriptor, file, etc.) is labeled with an SELinux context that defines the permissions and operations the object can perform. In RHEL 6.6/CentOS 6.6 and later, NGINX is labeled with the httpd_t context.

When SELinux is enabled, the UDMG Admin UI shows "403 access denied" and "404 page not found" errors on the landing page, and permission errors are reported in the NGINX log files:

Panel

...
2023/09/19 12:51:38 [error] 108236#108236: *1 "/opt/udmg/var/www/udmg/index.html" is forbidden (13: Permission denied), client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", host: "localhost
...

...